Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SageMaker Training Backend #3140

Merged
merged 13 commits into from
Dec 16, 2023
Merged

Add SageMaker Training Backend #3140

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
6baa4b5
Extract AmazonSageMakerBaseExecutor
jmsmkn Dec 14, 2023
0ec6ac8
Add implementation of the SageMaker Training Backend
jmsmkn Dec 14, 2023
7a67e72
Update the demo algorithm
jmsmkn Dec 15, 2023
fe95f52
Update sagemaker shim
jmsmkn Dec 16, 2023
f497970
Add taking directory ownership on SageMaker Training
jmsmkn Dec 16, 2023
b6a6597
Update SageMaker Shim
jmsmkn Dec 16, 2023
4705786
Add training output prefix
jmsmkn Dec 16, 2023
b6fffae
Fix component tests
jmsmkn Dec 16, 2023
21099d1
Fix docker tests
jmsmkn Dec 16, 2023
e22cccd
Fix cost fixtures
jmsmkn Dec 16, 2023
0ceeee2
Fix test fixture
jmsmkn Dec 16, 2023
32f40b3
Fix test fixture
jmsmkn Dec 16, 2023
01e3ab3
Fix undeletable directories
jmsmkn Dec 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 2 additions & 8 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,21 +113,15 @@ retina_archive_structures:
bash -c "python manage.py runscript create_retina_archive_structures"


scripts/algorithm_io.tar:
docker buildx build --platform linux/amd64 -t algorithm_io app/tests/resources/gc_demo_algorithm/
docker save algorithm_io -o scripts/algorithm_io.tar
chmod a+r scripts/algorithm_io.tar


algorithm_evaluation_fixtures: scripts/algorithm_io.tar
algorithm_evaluation_fixtures:
docker compose run \
-v $(shell readlink -f ./scripts/):/app/scripts:ro \
--rm \
web \
python manage.py runscript algorithm_evaluation_fixtures


cost_fixtures: scripts/algorithm_io.tar
cost_fixtures:
docker compose run \
-v $(shell readlink -f ./scripts/):/app/scripts:ro \
--rm \
Expand Down
3 changes: 3 additions & 0 deletions app/config/settings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1077,6 +1077,9 @@
COMPONENTS_DOCKER_TASK_AWS_SECRET_ACCESS_KEY = os.environ.get(
"COMPONENTS_DOCKER_TASK_AWS_SECRET_ACCESS_KEY", "componentstask123"
)
COMPONENTS_DOCKER_KEEP_CAPS_UNSAFE = strtobool(
os.environ.get("COMPONENTS_DOCKER_KEEP_CAPS_UNSAFE", "False")
)
COMPONENTS_PUBLISH_PORTS = strtobool(
os.environ.get("COMPONENTS_PUBLISH_PORTS", "False")
)
Expand Down
Loading