- improved hibench: this benchmark no longer requires a password for cloning hibridon, now (it's actually been the case for more than one year now) that the repository of hibridon is public.
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3958]
- split tests into 3:
- test_benchmarks.py: tests all benchmarks with the most basic resultsdb backend
- test_resultsdb: tests all resultsdb backends with the most basic benchmark
- test_clusterbench: tests clusterbench_submit
- made tests more robust (deletes the results foder if it already exists)
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3958]
note: as the production benchmarks are currently stored at IPR on a database server accessed via ssh, this allows iprbench to store its results to IPR benchmark database.
work rleated to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3958]
note: as the production benchmarks are currently stored at IPR on a database server accessed via ssh, this allows iprbench to store its results to IPR benchmark database.
work rleated to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3958]
these parameters are very much needed as they allow to identify the system on which the benchmark is run. Each performance benchmark highly depends on the host it's run on.
work rleated to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3958]
- At the moment, the database backend used is a set of tsv files, but the system is flexible to accomodate other dabase backends (mariadb, sqlite, etc.).
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3958]
This decoupling allows to write benchmarks as modules that can be used in various situations (from a benchmark job or directly from a user), but this design will allow automatic registering of the benchmark results in a user selectable form (sql database, stdout, etc.)
- separated `hibenchonphysix.py` into `clusterbench.py` (tool to run a benchmark on a cluster) and `hibench.py` (hibridon benchmark module) so that `clusterbench.py` no longer has a knowledge about hibridon.
- there are currently 2 ways to run a bechmark:
1. as a simple run through `clusterbench-run` command (which will eventually be renamed as iprbench-run since it might be completely independent from the concept of cluster)
2. as cluster jobs through `clusterbench-submit` command
- added unit test
- added another benchmark `mamul1` that is used as a unittest because it has 2 benefits over `hibench` benchmark:
1. it's standalone (no external resources needed)
2. it's quicker to execute
note: this refactoring work is not complete yet, but the concept proof is complete (the 2 unittests pass):
- still need to provide the user a way to switch between IpRCluster and DummyCluster(which is only intended to only be used for testing clusterbench))
- still need to run multiple configs of the same benchmark in one run (as hibenchonphysix did)
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3958] and [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3372]
- although still hardcoded, made it easier to switch between alambix and physix clusters
- although still hardcoded, made it easier to switch to test mode (quick test)
- removed hardcoded value for all_hosts_groups as it is retreived from cluster node database
nb: changes made on 08/10/2024
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3372]
This hosts description table is still hardcoded, though... at some point it will be better to use external data to make the code more generic (an attempt was made with pandas, but this introduced complexity in the setup so I decided to keep it hardcoded for the moment).
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3372]
since starbench is now an installable package, there is no starbench.py script anymore; starbench is no longer a simple python script; the existing code that assumed starbench was a simple python needed to be adapted.
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3372]
I had the case where on_exit() was never called because proc had no value and therefore the attempt to pass proc.pid to on_exit caused an exception before on_exit was called. As a result, the mater thread was waiting its children threads forever, as these child never signaled that they finished.
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3372]
- choosing different rsults dir allows to run the same benchmark more than once to have more than one measure
- this makes `hibenchonphysix.py` more decoupled from hibench, in the hope that it will be completely unaware of hibench at some poing (therefore reusable with other benchmarks)
- added the ability to show the performance per clock cycle (eg to see if avx512 has an effect on the computation)
- the cpus are now sorted on x axis depending on their generation
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3372]
Guillaume Raffy
g-raffy