Skip to content

Commit

Permalink
make EMR jar uploader work the same as k8s one
Browse files Browse the repository at this point in the history
Signed-off-by: Oleg Avdeev <[email protected]>
  • Loading branch information
oavdeev committed Jan 23, 2021
1 parent 775d673 commit 80d379e
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion sdk/python/feast/pyspark/launchers/aws/emr_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,11 @@ def _random_string(length) -> str:


def _upload_jar(jar_s3_prefix: str, jar_path: str) -> str:
if jar_path.startswith("https://"):
if (
jar_path.startswith("s3://")
or jar_path.startswith("s3a://")
or jar_path.startswith("https://")
):
return jar_path
with open(jar_path, "rb") as f:
uri = urlparse(os.path.join(jar_s3_prefix, os.path.basename(jar_path)))
Expand Down

0 comments on commit 80d379e

Please sign in to comment.