Implement list job and get job methods for Dataproc launcher #1106
Conversation
khorshuheng
requested review from
davidheryanto,
pyalex,
woop and
zhilingc
as code owners
| status = self.block_polling(timeout_sec=timeout_sec) | ||
| if status == SparkJobStatus.COMPLETED: | ||
| return self._output_file_uri | ||
| raise SparkJobFailure("Job fails") |
There was a problem hiding this comment.
job fails? is that the only information we provide?
|
@khorshuheng where can I see the tests that validate whether these methods work? |
Once we expose list job and get job method on Feast client, we can add tests for them under end to end test. Alternatively, this would be covered when we add Job service to the end to end test, since job service will use this two method. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: khorshuheng, pyalex The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Signed-off-by: Khor Shu Heng <[email protected]>
Signed-off-by: Khor Shu Heng <[email protected]>
Signed-off-by: Khor Shu Heng <[email protected]>
Signed-off-by: Khor Shu Heng <[email protected]>
khorshuheng
force-pushed
the
dataproc-listjob
branch
from
369a62c to
dab9261
Compare
Signed-off-by: Khor Shu Heng <[email protected]>
Signed-off-by: Khor Shu Heng <[email protected]>
Signed-off-by: Khor Shu Heng <[email protected]>
|
/lgtm |
Signed-off-by: Khor Shu Heng [email protected]
What this PR does / why we need it:
get_by_id and list_jobs methods were added to the launcher interface on #1095, but was not implemented for Dataproc launcher. This PR provides the implementation.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?: