-
Notifications
You must be signed in to change notification settings - Fork 300
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fast register for dynamic tasks #437
Conversation
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Signed-off-by: Katrina Rogan <[email protected]>
Codecov Report
@@ Coverage Diff @@
## master #437 +/- ##
==========================================
+ Coverage 84.25% 84.27% +0.01%
==========================================
Files 338 338
Lines 25528 25577 +49
Branches 2095 2104 +9
==========================================
+ Hits 21509 21555 +46
- Misses 3449 3451 +2
- Partials 570 571 +1
Continue to review full report at Codecov.
|
Signed-off-by: Katrina Rogan <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just a question for now, still thinking about stuff, sorry
with ctx.new_execution_context( | ||
mode=ExecutionState.Mode.TASK_EXECUTION, | ||
additional_context={ | ||
"dynamic_addl_distro": "s3::/my-s3-bucket/fast/123", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"dynamic_addl_distro": "s3::/my-s3-bucket/fast/123", | |
"dynamic_addl_distro": "s3://my-s3-bucket/fast/123", |
"distribution could be retrieved" | ||
) | ||
logger.warn(f"ctx.execution_state.additional_context {ctx.execution_state.additional_context}") | ||
sanitized_tasks = set() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
isn't tasks already a set? why do we need another set?
Signed-off-by: Max Hoffman <[email protected]>
Type
Are all requirements met?
Complete description
Passes in fast register arguments and substitutes them at dynamic task compilation time.
Tracking Issue
flyteorg/flyte#824
Follow-up issue
NA