Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support custom git clone branch in docker #343

Merged
merged 8 commits into from
Oct 5, 2024
Merged

Support custom git clone branch in docker #343

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ba1e507
fixing imagenet 500 url
gfursin Oct 5, 2024
73b9b3a
Merge branch 'main' of https://github.com/flexaihq/cm4mlops
gfursin Oct 5, 2024
6ab2217
Merge pull request #7 from mlcommons/main
gfursin Oct 5, 2024
861c4aa
restoring old links to dataset and win bin tools; however need to fin…
gfursin Oct 5, 2024
a7d560e
Merge pull request #340 from flexaihq/main
ctuning-admin Oct 5, 2024
7838076
Merge pull request #342 from mlcommons/main
gfursin Oct 5, 2024
1f27d0a
Commit to support custom git clone branch in docker
anandhu-eng Oct 5, 2024
c4f2ffd
Merge branch 'mlperf-inference' into customBranchDocker
arjunsuresh Oct 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions automation/script/module_misc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1470,6 +1470,8 @@ def dockerfile(i):
run_cmd = r['run_cmd_string']

cm_repo = i.get('docker_cm_repo', docker_settings.get('cm_repo', 'mlcommons@cm4mlops'))
cm_repo_branch = i.get('docker_cm_repo_branch', docker_settings.get('cm_repo_branch', 'mlperf-inference'))

cm_repo_flags = i.get('docker_cm_repo_flags', docker_settings.get('cm_repo_flags', ''))

docker_base_image = i.get('docker_base_image', docker_settings.get('base_image'))
Expand Down Expand Up @@ -1542,6 +1544,7 @@ def dockerfile(i):
'automation': 'script',
'tags': 'build,dockerfile',
'cm_repo': cm_repo,
'cm_repo_branch': cm_repo_branch,
'cm_repo_flags': cm_repo_flags,
'docker_base_image': docker_base_image,
'docker_os': docker_os,
Expand Down
2 changes: 1 addition & 1 deletion script/app-mlperf-inference/_cm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1631,7 +1631,7 @@ docker:
extra_run_args: ' --ulimit memlock=-1 --cap-add SYS_ADMIN --cap-add SYS_TIME --security-opt apparmor=unconfined --security-opt seccomp=unconfined'
os: ubuntu
cm_repo: mlcommons@cm4mlops
cm_repo_flags: ' --branch=mlperf-inference '
cm_repo_branch: mlperf-inference
real_run: False
os_version: '22.04'
docker_input_mapping:
Expand Down
2 changes: 2 additions & 0 deletions script/build-dockerfile/_cm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,15 @@ default_env:
'
CM_DOCKER_OS: ubuntu
CM_DOCKER_NOT_PULL_UPDATE: False
CM_MLOPS_REPO_BRANCH: mlperf-inference

input_mapping:
build: CM_BUILD_DOCKER_IMAGE
cache: CM_DOCKER_CACHE
cm_repo: CM_MLOPS_REPO
cm_repo_flags: CM_DOCKER_ADD_FLAG_TO_CM_MLOPS_REPO
cm_repos: CM_DOCKER_EXTRA_CM_REPOS
cm_repo_branch: CM_MLOPS_REPO_BRANCH
comments: CM_DOCKER_RUN_COMMENTS
copy_files: CM_DOCKER_COPY_FILES
docker_base_image: CM_DOCKER_IMAGE_BASE
Expand Down
9 changes: 3 additions & 6 deletions script/build-dockerfile/customize.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,8 @@ def preprocess(i):
else:
cm_mlops_repo = "mlcommons@cm4mlops"

if env.get("CM_MLOPS_REPO_BRANCH", '') != '':
cm_mlops_repo_branch_string = f" --branch {env['CM_MLOPS_REPO_BRANCH']}"
else:
cm_mlops_repo_branch_string = ""

cm_mlops_repo_branch_string = f" --branch={env['CM_MLOPS_REPO_BRANCH']}"

if env.get('CM_DOCKERFILE_WITH_PATH', '') == '':
env['CM_DOCKERFILE_WITH_PATH'] = os.path.join(os.getcwd(), "Dockerfile")

Expand Down Expand Up @@ -206,7 +203,7 @@ def preprocess(i):
x = env.get('CM_DOCKER_ADD_FLAG_TO_CM_MLOPS_REPO','')
if x!='': x=' '+x

f.write('RUN cm pull repo ' + cm_mlops_repo + x + EOL)
f.write('RUN cm pull repo ' + cm_mlops_repo + cm_mlops_repo_branch_string + x + EOL)

# Check extra repositories
x = env.get('CM_DOCKER_EXTRA_CM_REPOS','')
Expand Down
4 changes: 2 additions & 2 deletions script/get-sys-utils-cm/_cm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ deps:
env:
CM_CLEAN_DIRS: bin
CM_PACKAGE_WIN_URL: https://zenodo.org/records/13868077/files/cm-artifact-os-windows-32.zip?download=1
; https://www.dropbox.com/scl/fi/cj7xmgmk4wu1r0iuoedcq/zlib123dllx64-bin.zip?rlkey=sqmldotiste2zhk6nhmijg09e&st=i5w7esjx&dl=1
; https://www.dropbox.com/scl/fi/x94qvrmac55ht03257139/xz-5.2.9-win64.zip?rlkey=adzixlsqrapncrusy9ndd2qji&st=tqobzlfo&dl=1
; https://cKnowledge.org/ai/data/zlib123dllx64-bin.zip
; https://cKnowledge.org/ai/data/xz-5.2.9-win64.zip
CM_SUDO: sudo

input_mapping:
Expand Down
Loading