Optional IngestionJob parameters passed by Spark Launcher #1130
Conversation
pyalex
requested review from
davidheryanto,
khorshuheng,
woop and
zhilingc
as code owners
pyalex
changed the title
|
/test test-end-to-end-gcp |
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
pyalex
force-pushed
the
optional-job-options
branch
from
ff95b5a
to
5c2e162
Compare
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
Signed-off-by: Oleksii Moskalenko <[email protected]>
|
/test test-end-to-end-gcp |
| FEAST_SPARK_STAGING_LOCATION: file:///shared/staging | ||
| FEAST_HISTORICAL_FEATURE_OUTPUT_LOCATION: file:///shared/historical_feature_output | ||
| FEAST_HISTORICAL_FEATURE_OUTPUT_FORMAT: parquet | ||
| FEAST_REDIS_HOST: redis |
There was a problem hiding this comment.
FEAST_REDIS_PORT is a separate options, and its default value 6379 was not changed in this case. So I believe it was omitted to save 1 line)
| @@ -27,4 +27,4 @@ RUN wget -q https://github.com/grpc-ecosystem/grpc-health-probe/releases/downloa | |||
| -O /usr/bin/grpc-health-probe && \ | |||
| chmod +x /usr/bin/grpc-health-probe | |||
|
|
|||
| CMD ["feast", "server"] | |||
| CMD ["python", "-m", "feast.cli", "server"] | |||
There was a problem hiding this comment.
container was failing with
pkg_resources.DistributionNotFound: The 'Click==7.*' distribution was not found and is required by feast
It could be due to any reason, since it's conda.
Signed-off-by: Oleksii Moskalenko <[email protected]>
|
/test test-end-to-end |
Signed-off-by: Oleksii Moskalenko <[email protected]>
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: pyalex, woop The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/lgtm |
|
/test test-end-to-end-gcp |
Signed-off-by: Oleksii Moskalenko <[email protected]>
|
New changes are detected. LGTM label has been removed. |
Signed-off-by: Oleksii Moskalenko <[email protected]>
|
/test test-end-to-end-aws |
|
@pyalex: The following test failed, say
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
Signed-off-by: Oleksii Moskalenko [email protected]
What this PR does / why we need it:
SDK / JobService should be able to configure
statsd,deadletterPathandstencilURLoptional parameters of IngestionJobs.Non-default project passed to Job via SDK / JobService
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?: