- fixed a bug : now, the displayed memory is correct even if the mem_available sge attribute returns the memory in other units than gigabyte
- also, this commit includes an old improvement that makes executeCommandOn handle an optional user, which was previously slinetly ignored in the case of a local command
- for the sake of debugging, improved the handling of missing data in SimpaDb.sql : now when a data is missing, the exception handler displays a message describing what's missing.
tentative de correction d'un bug qui cause l'erreur suivante d'apparaitre de temps en temps, à l'occasion du chargement de la page clusteruserguide :
SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 139672342353664 and this is thread id 139672333960960
- désormais, la page clusteruserguide est générique : elle genere le user guide de simpatix et physix
- désormais, la page clusteruserguide est mise à jour automatiquement : elle génère la liste les noeuds et les gflops dynamiquement en regardant la sortie de qstat et en utilisant la base de données SimpaDb.sql (on a réutilisé les classes qui avaent été écrites pour clusterstatusview)
- désormais, la page physix cluster status est disponible. Pour cela, il a fallu :
- ajouter un mécanisme d'export/import de clef publique ssh utilisateur (pour que intranet.ipr.univ-rennes1.fr puisse exécuter des commandes sge sur physix-master)
- adapter la view clusterstatus pour qu'elle sache gérer les commandes sge déportées. Au final, cette vue permet d'afficher le pages cluterstatus de simpatix (commandes sge locales) et physix (commandes sge déportées)
- now sge uses ssh mode for qlogin (it was the case on simpatix too but it wasn't yet scripted). qlogin using ssh works provided the ports are open.
- also set qsh and qrsh to use ssh mode to be consistent. (although not tested yet)
- fixed bug introduced recently where the name of the machine was used instead of its fully qualified domain name. This caused ClusetrConfigurator to fail on exec nodes
Bug 2230 - Unifier les flags de ressource gridengine en supprimant les ressources "has_<app>"
- migrated the following complexes:
- has_matlab2011b -> matlab_version
- has_dsmcmodfoam -> dsmcmodfoam_version
- has_meep -> meep_version
- openfoam_version (int encoding) -> openfoam_version (float encoding)
- also disabled obsolete resources (has_molpro2006 and has_molpro2010) on physix as they've been replaced with the better molpro_version.
fixed bug that causes SgeConfigurator to fail with sge 8.1.9 because a new parameter qsort_args has been added to parallel environment:
command failed : qconf -Ap '/tmp/make'
root@physix-master:/home/graffy# qconf -Ap '/tmp/make'
error: required attribute "qsort_args" is missing
error reading in file
- mise à la norme pep8 de SgeConfigurator en vue de l'améliorer pour qu'il puisse gérer :
- 2 clusters différents, dans un court terme, pour mettre physix88 en production dans un nouveau cluster sge (physix))
- 2 backends différents (sge et slurm), dans un plus long terme
nb: pour la validation pep8, j'ai été obligé d'ajouter un projet eclipse python appelé simpapy pour SimpaScripts/Python
- ajout de l'application DebtComputer qui permet de calculer l'argent injecté par un utilisateur dans le pot commun, et l'usage dont il en bénéficie
nb: les maintenances ne sont pas encore prises en compte
- the cluster evolution graphs are now displayed on the intranet :
- I had to make python graphing code handle debian 7 version of matplotlib; this was rather painful)
- I had to handle the case where the default matplotlib backend doesn't work (because of no display)
- improvement : the figures can now be saved as svg files. This mechanism will be used by the intranet to display the cluster evolution graphs
- also removed debug prints that pollute stdout
- refactoring : improved so that now PowerDiagram also uses the Inventory class instead of duplicating code. This will:
1. make the code easier to maintain when there is a change in the inventory database tables
2. make the code easier to adapt when we switch from a sql file to a mysql server
- refactored so that it now abstracts the database provider ; this way, accessing the inventory database through a database server instead of a sdl dump file is mostly transparent
From now on, the servers power diagram is available on http://intranet.ipr.univ-rennes1.fr/simpaweb/itteam/PowerDiagram/
- note : I initially thought I would use the jit (JavaScript InfoVis Toolkit) to display sexy interactive diagrams but I finally decided not to (although I managed to run an example) because:
- svg is more printer-friendly, and could be saved as a file
- generating jit data would have required more work, which doesn't add much value
So, in a further commit, I will clean up the jit-related mess
- drastically improved the performance of JobsState.AddJob call (used for example in clusterstatus page) in case of big job arrays. As an example : clusterstatus page took 30 seconds when there was a job array of 500 elements ; this now takes 3 seconds. The culprit was a conflicting hash (I don't remember why I didn't make a non conflicting hash in the first place), that was the same for each job array element.
- slightly improved speed by preventing unnecessary requests of job array element details
However, the big hit in performance is still in the dictionary of jobs in JobsState
Guillaume Raffy