Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨Dask sidecar: pass job origin and show child computational service logs in workbench #5054

Merged

Conversation

sanderegg
Copy link
Member

@sanderegg sanderegg commented Nov 20, 2023

What do these changes do?

This PR brings:

  • refactoring/cleaning of the dask-sidecar remote function signature
  • allows to pass optional parent_node_id and parent_project_id arguments to the computational job, which is also passed back through the logs/progress events
  • director-v2 then sends the logs of the node in the logs exchange in RabbitMQ using both the topic of the current project id and of the parent project id if it exists, which means that the logs pane in the GUI shows the logs of a computational service started by a parent service through the public API (sim4life use-case)

Related issue/s

How to test

Dev Checklist

DevOps Checklist

@sanderegg sanderegg added the a:dask-service Any of the dask services: dask-scheduler/sidecar or worker label Nov 20, 2023
@sanderegg sanderegg added this to the 7peaks milestone Nov 20, 2023
@sanderegg sanderegg self-assigned this Nov 20, 2023
Copy link

codecov bot commented Nov 20, 2023

Codecov Report

Merging #5054 (faadab7) into master (cc6b89e) will increase coverage by 2.2%.
The diff coverage is 96.3%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #5054      +/-   ##
=========================================
+ Coverage    85.1%   87.3%    +2.2%     
=========================================
  Files        1263     924     -339     
  Lines       51910   40711   -11199     
  Branches     1114     207     -907     
=========================================
- Hits        44188   35559    -8629     
+ Misses       7489    5103    -2386     
+ Partials      233      49     -184     
Flag Coverage Δ
integrationtests 64.9% <84.4%> (+9.0%) ⬆️
unittests 84.5% <95.6%> (-0.5%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...dask_task_models_library/container_tasks/events.py 100.0% <100.0%> (+4.8%) ⬆️
...src/dask_task_models_library/container_tasks/io.py 100.0% <100.0%> (ø)
...service_dask_sidecar/computational_sidecar/core.py 100.0% <100.0%> (ø)
...car/src/simcore_service_dask_sidecar/dask_utils.py 97.6% <100.0%> (+0.1%) ⬆️
...-sidecar/src/simcore_service_dask_sidecar/tasks.py 92.5% <100.0%> (-0.2%) ⬇️
...r-v2/src/simcore_service_director_v2/utils/dask.py 89.7% <100.0%> (+1.5%) ⬆️
...sk_task_models_library/container_tasks/protocol.py 96.1% <97.3%> (+96.1%) ⬆️
...simcore_service_director_v2/modules/dask_client.py 93.1% <97.0%> (+0.5%) ⬆️
...rector_v2/modules/comp_scheduler/dask_scheduler.py 90.3% <57.1%> (+1.9%) ⬆️

... and 405 files with indirect coverage changes

@sanderegg sanderegg force-pushed the dask-sidecar/pass-job-origin branch 3 times, most recently from 18a09c9 to f4a599c Compare November 21, 2023 20:28
@sanderegg sanderegg marked this pull request as ready for review November 21, 2023 20:34
@sanderegg sanderegg changed the title ✨Dask sidecar: pass job origin ✨Dask sidecar: pass job origin and show child computational service logs in workbench Nov 21, 2023
Copy link
Contributor

@bisgaard-itis bisgaard-itis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Just a few comments

Copy link

sonarcloud bot commented Nov 22, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

Copy link

codeclimate bot commented Nov 22, 2023

Code Climate has analyzed commit faadab7 and detected 0 issues on this pull request.

View more on Code Climate.

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice cleanup!

@sanderegg sanderegg merged commit 6911102 into ITISFoundation:master Nov 22, 2023
56 checks passed
@sanderegg sanderegg deleted the dask-sidecar/pass-job-origin branch November 22, 2023 07:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:dask-service Any of the dask services: dask-scheduler/sidecar or worker
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants