Bug 1458 - la page clusterstatus ne répond plus

- drastically improved the performance of JobsState.AddJob call (used for example in clusterstatus page) in case of big job arrays. As an example : clusterstatus page took 30 seconds when there was a job array of 500 elements ; this now takes 3 seconds. The culprit was a conflicting hash (I don't remember why I didn't make a non conflicting hash in the first place), that was the same for each job array element.
This commit is contained in:
Guillaume Raffy 2016-09-06 09:47:56 +00:00
parent 1cff78822e
commit 3ad6206363
1 changed files with 6 additions and 1 deletions

View File

@ -27,7 +27,9 @@ class JobId:
A single integer is no longer enough to identify a job because all elements in a job array A single integer is no longer enough to identify a job because all elements in a job array
share the same sge job identifier. To uniquely define a job array element, we also use the task id. share the same sge job identifier. To uniquely define a job array element, we also use the task id.
""" """
MAX_NUM_JOBS_IN_ARRAY = 1000000
def __init__( self, iJobId, iJobArrayElementId = None): def __init__( self, iJobId, iJobArrayElementId = None):
assert iJobArrayElementId <= self.MAX_NUM_JOBS_IN_ARRAY
self.m_iJobId = iJobId self.m_iJobId = iJobId
self.m_iJobArrayElementId = iJobArrayElementId # None if this identifier does not refer to a job array element self.m_iJobArrayElementId = iJobArrayElementId # None if this identifier does not refer to a job array element
@ -35,7 +37,10 @@ class JobId:
""" """
required to use a JobId as a dict hash key required to use a JobId as a dict hash key
""" """
return self.m_iJobId # very simple hashing that conflicts only for job array elements hash = self.m_iJobId * self.MAX_NUM_JOBS_IN_ARRAY
if self.m_iJobArrayElementId is not None:
hash += self.m_iJobArrayElementId
return hash
def __eq__( self, other ): def __eq__( self, other ):
""" """