- sqldb.ISqlConnection provides the same functionality as SimpaDbUtil.ISqlConnection, with much greater performance thanks to python contexts. However the interface is not compatible, so codes wanting to switch from SimpaDbUtil.ISqlConnection to sqldb.ISqlConnection need adjusting.
- migrated quman to use sqldb.ISqlConnection (as a result the sync now takes 3seconds instead of 60, because of the drastic reductin of calls to mysql and ssh)
- SimpaDbUtil.ISqlConnection will be deleted once every code using SimpaDbUtil.ISqlConnection has been migrated to sqldb.ISqlConnection
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
- replaced ISqlDatabaseBackend.get_last_insert_id() with ISqlDatabaseBackend.query_insert(), as ISqlDatabaseBackend.get_last_insert_id() cannot work with SshAccessedMysqlDb (each call always returns 0 because mysql starts a new session each time)
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
- splitted ISqlDatabaseBackend.query() into ISqlDatabaseBackend.query() and the more specialized SqlDatabaseBackend.query_select ()
- this way the return type of query_select is explicit (a Table).
- it also allows robust sql query parsing (usin json).
- replaced calls to ISqlDatabaseBackend.query('select ...') with calls to ISqlDatabaseBackend.query_select()
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
- added the method ISqlDatabaseBackend.get_last_insert_id because the actual sql function is different in sqlite (last_insert_rowid) and mysql (last_insert_id), the query is different for both backends
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
- this option allows the quman to work with user configurable database system as a backend. the default uses the database for alambix cluster
- also improved the output of show-disable-requests
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
```
WARNING: Missing build requirements in pyproject.toml for file:///home/graffy/work/simpaweb/cocluto.git.
WARNING: The project does not specify a build backend, and pip cannot fall back to setuptools without 'wheel'.
```
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
eg problem visible with the command:
> /usr/bin/python3 -m pip install --root=/ git+https://git.ipr.univ-rennes.fr/cellinfo/cocluto to
then:
```sh
root@alambix50:~# quman
Traceback (most recent call last):
File "/usr/local/bin/quman", line 5, in <module>
from cocluto.quman import main
File "/usr/local/lib/python3.11/dist-packages/cocluto/quman.py", line 13, in <module>
from cocluto.ClusterController.QstatParser import QstatParser
```
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
- quman's argument parsing now uses subparsers to allow for more actions than enable and disable
- added the show-disables action to quman
- added a test mode for the command line quman, that allows the user to test quman using quman's user interface
- renamed disable tag as disable id, as it's more meaningful (identfies a disable request)
- made user errors more user-friendly (ie not polluted with call stack)
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
Whith this feature, it's now possible to disable an entire queue (and not only one queue machine), with
```sh
quman d main.q --disable-tag admin.graffy.bug4242 --reason 'preparing cluster to shutdown for power shortage, see bug 4242'
```
- also replaced the term queue name with queue machine for clarity, now that we have both QueueMachineId and QueueId
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
- the userid and the host fqdn of are now recorded
- found better names (more appropriate and easier to understand)
- disable reason -> disable request
- requester -> disable tag
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
- fixed bug in request_queue_activation, which caused it to always enable the queue, even if there are other disables
- added a synchronization mechanism to quman which patches the database to ensure the database is coherent with the current activation of the queues
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3093]
- fixed bug that caused `draw_machine_age_pyramid_graph` to fail when some cluster machines get older than 20 years. Increased the age limit to 25 years, and added clipping to the graph to cope with some machines even older than this
fixes [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=4101]
- fixed bug in SshAccessedMysqlDb which didn't handle the fields with is_autoinc_index properly
- fixed bug in SshAccessedMysqlDb which failed when a field description contained double quotes
- improved SshAccessedMysqlDb.query(), which now displays the error message on failure
- fixed bug in SshAccessedMysqlDb.table_exists,which failed when the table didn't exist.
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3958]
- added to the class `ISqlDatabaseBackend` the ability to delete a sql table
- also added `SshAccessedMysqlDb`: an implementation of `ISqlDatabaseBackend` that implements the sql database as a remote sql database accessed using ssh
- added unit tests to validate the backends SshAccessedMysqlDb and SqliteDb
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3958]
- added to the class `ISqlDatabaseBackend` the ability to create a sql table
- also added `SqliteDb`: an implementation of `ISqlDatabaseBackend` that implements the sql database as a sqlite3 database
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3958]
- all styling errors are now fixed, but there are still warnings and information
- most functions have been converted to snake case
- most functions now have type hinting
work related to [https://bugzilla.ipr.univ-rennes.fr/show_bug.cgi?id=3873]
when Job2 was introduced, it was actually representing a job, but the class Job (which actually represented a Task, and this was misleading) already existed
also:
- added type hinting to ease code understanding
- took this opportunity to fix styling issues
- this bug was found when working on Bug 3575 - la page physix status renvoit internal server error
- in order to ease the detection of such regressions, added a unittest that covers this code
Guillaume Raffy