Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.0] adding possibility to bulk insert in JobLoggingDB #7866

Merged
merged 5 commits into from
Nov 20, 2024

Conversation

fstagni
Copy link
Contributor

@fstagni fstagni commented Nov 1, 2024

BEGINRELEASENOTES

*WMS
NEW: for parametric jobs, added the possibility to bulk insert records in JobLoggingDB

ENDRELEASENOTES

@DIRACGridBot DIRACGridBot added the alsoTargeting:integration Cherry pick this PR to integration after merge label Nov 1, 2024
@fstagni fstagni force-pushed the 80_bulkInsertJobLoggingDB branch 8 times, most recently from d5579d0 to 8fc7b77 Compare November 4, 2024 10:27
@fstagni fstagni changed the title feat: adding possibility to bulk insert in JobLoggingDB [8.0] adding possibility to bulk insert in JobLoggingDB Nov 4, 2024
@fstagni fstagni marked this pull request as ready for review November 4, 2024 10:29
@fstagni fstagni force-pushed the 80_bulkInsertJobLoggingDB branch from 8fc7b77 to 931e6d7 Compare November 4, 2024 11:31
@fstagni fstagni force-pushed the 80_bulkInsertJobLoggingDB branch 2 times, most recently from 6dc37a4 to 2ffbfed Compare November 8, 2024 17:23
@fstagni fstagni force-pushed the 80_bulkInsertJobLoggingDB branch 2 times, most recently from f16a1de to a765366 Compare November 18, 2024 15:16
@fstagni fstagni closed this Nov 19, 2024
@fstagni fstagni reopened this Nov 19, 2024
@fstagni fstagni force-pushed the 80_bulkInsertJobLoggingDB branch from 35f2e3e to d834c6c Compare November 19, 2024 15:55
Comment on lines +214 to +223
# For parametric jobs I can insert logging records in a bulk
if parametricJob and len(set(jobIDList)) == len(jobIDList):
result = self.jobLoggingDB.addLoggingRecord(
jobIDList, statusList, minorStatusList, date=timeStampList, source="JobManager"
)
else:
for jobID, status, minorStatus, timeStamp in zip(jobIDList, statusList, minorStatusList, timeStampList):
result = self.jobLoggingDB.addLoggingRecord(
jobID, status, minorStatus, date=timeStamp, source="JobManager"
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there still a need for this if? I think the first branch would work for all cases?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a trigger in JobLoggingDB:

https://github.com/DIRACGrid/DIRAC/blob/rel-v8r0/src/DIRAC/WorkloadManagementSystem/DB/JobLoggingDB.sql#L39-L45

I am not 100% sure this would be OK if len(set(jobIDList)) != len(jobIDList)

@fstagni fstagni force-pushed the 80_bulkInsertJobLoggingDB branch from d834c6c to cd14009 Compare November 20, 2024 09:03
@fstagni fstagni merged commit ec01262 into DIRACGrid:rel-v8r0 Nov 20, 2024
26 checks passed
@DIRACGridBot DIRACGridBot added sweep:done All sweeping actions have been done for this PR sweep:failed Sweeping failed and needs manual intervention labels Nov 20, 2024
@DIRACGridBot
Copy link

Sweep summary

Sweep ran in https://github.com/DIRACGrid/DIRAC/actions/runs/11938158973

Failed:

  • integration
    cherry-pick ec01262 into integration failed
    check merge conflicts on a local copy of this repository
    git fetch upstream
    git checkout upstream/integration -b cherry-pick-2-ec0126231-integration
    git cherry-pick -x -m 1 ec0126231
    # Fix the conflicts
    git cherry-pick --continue
    git commit --amend -m 'sweep: #7866 adding possibility to bulk insert in JobLoggingDB' --author='Federico Stagni <[email protected]>'
    git push -u origin cherry-pick-2-ec0126231-integration
    
    # If you have the GitHub CLI installed the PR can be made with
    gh pr create \
         --label 'sweep:from rel-v8r0' \
         --base integration \
         --repo DIRACGrid/DIRAC \
         --title '[sweep:integration] adding possibility to bulk insert in JobLoggingDB' \
         --body 'Sweep #7866 `adding possibility to bulk insert in JobLoggingDB` to `integration`.
    
    Adding original author @fstagni as watcher.
    
    BEGINRELEASENOTES
    
    *WMS
    NEW: for parametric jobs, added the possibility to bulk insert records in JobLoggingDB
    
    ENDRELEASENOTES
    Closes #7904'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alsoTargeting:integration Cherry pick this PR to integration after merge sweep:done All sweeping actions have been done for this PR sweep:failed Sweeping failed and needs manual intervention
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants