This repository has been archived by the owner on Oct 9, 2023. It is now read-only.
Introduce Template-based task logging plugin #144
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kumare3
reviewed
Jan 19, 2021
kumare3
reviewed
Jan 19, 2021
| DefaultSparkConfig map[string]string `json:"spark-config-default" pflag:"-,Key value pairs of default spark configuration that should be applied to every SparkJob"` | ||
| SparkHistoryServerURL string `json:"spark-history-server-url" pflag:",URL for SparkHistory Server that each job will publish the execution history to."` | ||
| Features []Feature `json:"features" pflag:"-,List of optional features supported."` | ||
| LogConfig LogConfig `json:"logs" pflag:",Config for log links for spark applications."` |
There was a problem hiding this comment.
I'm not sure I like the * to indicate optional fields in logs... the added, mostly redundant, nil checks alone are just tiring... I think we might want to add something on the tags to indicate requirement...
kumare3
reviewed
Jan 19, 2021
Codecov Report
@@ Coverage Diff @@
## master #144 +/- ##
==========================================
+ Coverage 61.62% 61.91% +0.28%
==========================================
Files 106 109 +3
Lines 6090 6244 +154
==========================================
+ Hits 3753 3866 +113
- Misses 1954 1979 +25
- Partials 383 399 +16
Continue to review full report at Codecov.
|
akhurana001
reviewed
Jan 19, 2021
akhurana001
approved these changes
Jan 20, 2021
|
Yeah that won't work... the configured token doesn't have permissions to
push... I won't worry about it... once we move the repos over, I'll fix
these
…On Tue, Jan 19, 2021 at 4:48 PM akhurana001 ***@***.***> wrote:
***@***.**** approved this pull request.
LGTM, this looks like backward compatible with the current setup so we
should be fine.
Also the build is failing due to some access error, not sure why .
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#144 (review)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAGUF3DZJJDTWT6PVW46RTS2YR6XANCNFSM4WIABKCQ>
.
--
Thanks,
Haytham Abuelfutuh.
|
3 tasks
eapolinario
pushed a commit
that referenced
this pull request
Sep 6, 2023
* New task log interface and template plugin * Update spark and pytorch plugins * fix unit tests * Make tests deterministic * PR Comments * unit test sorted * Ensure deterministic order of task logs * Use mixed logs for driver pod * PR Comments
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TL;DR
Add Template-based task logging plugin and update usage to match.
Type
Are all requirements met?
Tracking Issue
flyteorg/flyte#549
flyteorg/flyte#448