Add deckURI to NodeExecutionData

[pingsutw]

Signed-off-by: Kevin Su [email protected]

TL;DR

Add deckURI to NodeExecutionData, so that Flyteconsole can retrieve the pre-signed deck URI and read the file.

Type

• Bug Fix
• Feature
• Plugin

Are all requirements met?

• Code completed
• Smoke tested
• Unit tests added
• Code documentation added
• Any pending items have an associated Issue

Complete description

How did you fix the bug, make the feature etc. Link to any design docs etc

Tracking Issue

flyteorg/flyte#2175

Follow-up issue

NA

[codecov]

Codecov Report

Merging #413 (c4c95d2) into master (77b83f6) will increase coverage by 0.03%.
The diff coverage is 67.74%.

@@            Coverage Diff             @@
##           master     #413      +/-   ##
==========================================
+ Coverage   61.33%   61.37%   +0.03%     
==========================================
  Files         156      156              
  Lines       11095    11142      +47     
==========================================
+ Hits         6805     6838      +33     
- Misses       3586     3595       +9     
- Partials      704      709       +5     

Flag	Coverage Δ
unittests	60.30% <66.66%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/config/config.go	20.00% <ø> (ø)
pkg/manager/impl/util/data.go	73.07% <ø> (ø)
dataproxy/service.go	47.61% <65.51%> (+9.43%)	⬆️
pkg/config/serverconfig_flags.go	58.69% <100.00%> (+0.91%)	⬆️
pkg/repositories/transformers/node_execution.go	72.39% <100.00%> (+0.14%)	⬆️
...cloudevent/implementations/cloudevent_publisher.go	81.81% <0.00%> (-4.63%)	⬇️
pkg/manager/impl/execution_manager.go	74.64% <0.00%> (ø)
pkg/runtime/application_config_provider.go	33.33% <0.00%> (ø)
scheduler/core/gocron_scheduler.go	83.07% <0.00%> (+0.08%)	⬆️
pkg/manager/impl/node_execution_manager.go	72.54% <0.00%> (+0.31%)	⬆️
... and 1 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 77b83f6...c4c95d2. Read the comment docs.

[pingsutw]

cc @EngHabu @katrogan one question, I'm able to get a signed URL when using flytekit, but it doesn't work when I use curl. Anything I missed in this PR?

(base) ➜  ~ curl http://34.201.131.50:30081//api/v1/dataproxy/artifact_urn              
Not Found
(base) ➜  ~ curl http://34.201.131.50:30081//api/v1/dataproxy/artifact_urn\?native_url\=deck.html
Not Found

[katrogan]

@pingsutw why are there two slashes before api in the path?

[pingsutw]

same error if I use http://34.201.131.50:30081/api/v1/dataproxy/artifact_urn

[EngHabu]

Try this:

curl http://34.201.131.50:30081//api/v1/dataproxy/artifact_urn?native_url=s3://bucket/deck.html

[pingsutw]

Fixed it, we should register the data proxy handler in the HTTP server. https://github.com/flyteorg/flyteadmin/pull/413/files#diff-4ce88803abf59984cddcf6ab336c97b3231b0cb87f90e07f2f42e622e330f325R197-R201

[pmahindrakar-oss]

Is this valid that we can accurately say here . The library sets this time for signed url if its google
https://github.com/flyteorg/stow/blob/9c3f5f9ea24966206f0b2edc608da6ca803420e3/google/container.go#L55

[pingsutw]

it's not accurate, but I think it's okay for now because we can't get the expiry time from SignedURLResponse
I can add it in a separate PR.