-
Notifications
You must be signed in to change notification settings - Fork 300
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds neptune plugin for experiment tracking #2686
Adds neptune plugin for experiment tracking #2686
Conversation
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2686 +/- ##
===========================================
- Coverage 78.69% 50.73% -27.97%
===========================================
Files 187 192 +5
Lines 19257 19425 +168
Branches 4029 3805 -224
===========================================
- Hits 15155 9855 -5300
- Misses 3404 9065 +5661
+ Partials 698 505 -193 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
def neptune_init_run( | ||
project: str, | ||
secret: Union[Secret, Callable], | ||
host: str = "https://app.neptune.ai", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any particular reason host
has to be passed explicitly?
This can be extracted from the NEPTUNE_API_TOKEN
json.loads(base64.b64decode(os.getenv("NEPTUNE_API_TOKEN")).decode("utf-8"))["api_url"]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In a IRL conversation with @SiddhantSadangi, we discussed how this is a limitation of Flyte and dynamic log links.
Context: The Flyte UI needs to know the host URL during registration time. There are two options:
- Have the host passed in explicitly (this PR)
- Force users to have
NEPTUNE_API_TOKEN
defined during registration time.
…ne_pr Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
assert run_mock["Flyte Execution ID"] == host_name | ||
assert run_mock["Flyte Execution URL"] == execution_url |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
need to update these assertions.
Signed-off-by: Thomas J. Fan <[email protected]>
…ne_pr Signed-off-by: Thomas J. Fan <[email protected]>
Why are the changes needed?
This PR adds a Neptune.ai plugin for experiment tracking. This is similar to the Comet and Weights & Biases plugin.
What changes were proposed in this pull request?
This PR adds a
neptune_init_run
decorator that starts a Neptune Run. Given the Neptune integration with third party libraries passes therun
object around, this PR injects theneptune_run
object intocurrent_context()
.How was this patch tested?
This PR includes unit tests. I also ran the example in flyteorg/flytesnacks#1723:
Docs link
[
](flyteorg/flytesnacks#1723)