Skip to content

Commit

Permalink
Merge branch 'mainline' into rmvh/maya_fix_multiple_submitters
Browse files Browse the repository at this point in the history
  • Loading branch information
rmv authored Sep 20, 2023
2 parents d5c6845 + dee1fe9 commit 304e5be
Show file tree
Hide file tree
Showing 25 changed files with 131 additions and 150 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/e2e_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
id-token: write
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_CODEBUILD_MAINLINE_E2E_ROLE }}
aws-region: us-west-2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/integ_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
id-token: write
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_CODEBUILD_MAINLINE_INTEG_ROLE }}
aws-region: us-west-2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
id-token: write
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_CODEBUILD_RELEASE_PUBLISH_ROLE }}
aws-region: us-west-2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ jobs:
with:
python-version: '3.9'
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_CODEARTIFACT_ROLE }}
aws-region: us-west-2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/reuse_python_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ jobs:
python-version: ${{ matrix.python-version }}

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v3
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_CODEARTIFACT_ROLE }}
aws-region: us-west-2
Expand Down
13 changes: 6 additions & 7 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,17 +48,16 @@ your build of the adaptor for the one in the service.

1. Use the development location from the Submitter Development Workflow.
Make sure you're running Maya with `set DEADLINE_ENABLE_DEVELOPER_OPTIONS=true` enabled.
2. Build wheels for `openjd`, `deadline` and `deadline-cloud-for-maya`.
2. Build wheels for `openjd`, `deadline` and `deadline-cloud-for-maya`, place them in a "wheels"
folder in `deadline-cloud-for-maya`. A script is provided to do this, just execute from `deadline-cloud-for-maya`:
```
# If you don't have the build package installed already
$ pip install build
...
$ mkdir wheels; \
rm wheels/*; \
for dir in ../openjd ../deadline-cloud ../deadline-cloud-for-maya; \
do python -m build --wheel --outdir ./wheels --skip-dependency-check $dir; \
done
...
$ ./scripts/build_wheels.sh
```
Wheels should have been generated in the "wheels" folder:
```
$ ls ./wheels
deadline_cloud_for_maya-<version>-py3-none-any.whl
deadline-<version>-py3-none-any.whl
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ assetReferences:
outputs:
directories:
- /normalized/job/bundle/dir/images
referencedPaths: []
Original file line number Diff line number Diff line change
@@ -1,12 +1,4 @@
parameterValues:
- name: deadline:priority
value: 50
- name: deadline:targetTaskRunStatus
value: READY
- name: deadline:maxFailedTasksCount
value: 20
- name: deadline:maxRetriesPerTask
value: 5
- name: MayaSceneFile
value: /normalized/job/bundle/dir/cube.ma
- name: Frames
Expand All @@ -23,5 +15,11 @@ parameterValues:
value: /normalized/job/bundle/dir/images
- name: RenderSetupIncludeLights
value: 'true'
- name: RezPackages
value: mayaIO mtoa deadline_cloud_for_maya
- name: deadline:targetTaskRunStatus
value: READY
- name: deadline:maxFailedTasksCount
value: 20
- name: deadline:maxRetriesPerTask
value: 5
- name: deadline:priority
value: 50
Original file line number Diff line number Diff line change
Expand Up @@ -66,15 +66,6 @@ parameterDefinitions:
allowedValues:
- 'true'
- 'false'
- name: RezPackages
type: STRING
userInterface:
control: LINE_EDIT
label: Rez Packages
groupLabel: Software Environment
description: A space-separated list of Rez packages to install. Requires a Queue
Environment to handle the env creation.
default: mayaIO mtoa deadline_cloud_for_maya
- name: OutputFilePrefix
type: STRING
userInterface:
Expand Down
8 changes: 4 additions & 4 deletions job_bundle_output_tests/cube/scene/cube.ma
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
//Codeset: 1252
file -rdi 1 -ns "scene_file_to_reference" -rfn "scene_file_to_referenceRN" -op
"VERS|2023|UVER|undef|MADE|undef|CHNG|Wed, Aug 30, 2023 02:59:28 PM|ICON|undef|INFO|undef|OBJN|21|INCL|undef(|LUNI|cm|TUNI|film|AUNI|deg|TDUR|141120000|"
-typ "mayaBinary" "C:/Users/markw/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests//cube/scene_file_to_reference.mb";
-typ "mayaBinary" "D:/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests//cube/scene_file_to_reference.mb";
file -r -ns "scene_file_to_reference" -dr 1 -rfn "scene_file_to_referenceRN" -op
"VERS|2023|UVER|undef|MADE|undef|CHNG|Wed, Aug 30, 2023 02:59:28 PM|ICON|undef|INFO|undef|OBJN|21|INCL|undef(|LUNI|cm|TUNI|film|AUNI|deg|TDUR|141120000|"
-typ "mayaBinary" "C:/Users/markw/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests//cube/scene_file_to_reference.mb";
-typ "mayaBinary" "D:/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests//cube/scene_file_to_reference.mb";
requires maya "2023";
requires "stereoCamera" "10.0";
requires -nodeType "aiOptions" -nodeType "aiAOVDriver" -nodeType "aiAOVFilter" "mtoa" "5.1.0";
Expand Down Expand Up @@ -188,13 +188,13 @@ createNode materialInfo -n "materialInfo1";
rename -uid "B803D1DE-4EE6-CFD3-DEC1-BD90C863A0EA";
createNode file -n "file1";
rename -uid "1BAA2EE5-46EB-ED01-BE45-9888E95529F2";
setAttr ".ftn" -type "string" "C:/Users/markw/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests//cube/deadline_logo_48x48.png";
setAttr ".ftn" -type "string" "D:/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests//cube/deadline_logo_48x48.png";
setAttr ".cs" -type "string" "sRGB";
createNode place2dTexture -n "place2dTexture1";
rename -uid "CEF90DC0-44C4-11AF-47D0-97B5269B63A4";
createNode reference -n "scene_file_to_referenceRN";
rename -uid "9FAAF201-4C9E-4E6C-9DC3-359F06356F1E";
setAttr ".ed" -type "dataReferenceEdits"
setAttr ".ed" -type "dataReferenceEdits"
"scene_file_to_referenceRN"
"scene_file_to_referenceRN" 0;
setAttr ".ptag" -type "string" "";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ assetReferences:
- /normalized/job/bundle/dir/images/renderSetupLayer2/camera3
- /normalized/job/bundle/dir/images/renderSetupLayer4/camera1
- /normalized/job/bundle/dir/images/renderSetupLayer4/persp
referencedPaths: []
Original file line number Diff line number Diff line change
@@ -1,12 +1,4 @@
parameterValues:
- name: deadline:priority
value: 50
- name: deadline:targetTaskRunStatus
value: READY
- name: deadline:maxFailedTasksCount
value: 20
- name: deadline:maxRetriesPerTask
value: 5
- name: MayaSceneFile
value: /normalized/job/bundle/dir/layers.ma
- name: layerDefaultFramesFrames
Expand Down Expand Up @@ -57,5 +49,11 @@ parameterValues:
value: 'true'
- name: ArnoldErrorOnLicenseFailure
value: 'false'
- name: RezPackages
value: mayaIO mtoa deadline_cloud_for_maya
- name: deadline:targetTaskRunStatus
value: READY
- name: deadline:maxFailedTasksCount
value: 20
- name: deadline:maxRetriesPerTask
value: 5
- name: deadline:priority
value: 50
Original file line number Diff line number Diff line change
Expand Up @@ -58,15 +58,6 @@ parameterDefinitions:
allowedValues:
- 'true'
- 'false'
- name: RezPackages
type: STRING
userInterface:
control: LINE_EDIT
label: Rez Packages
groupLabel: Software Environment
description: A space-separated list of Rez packages to install. Requires a Queue
Environment to handle the env creation.
default: mayaIO mtoa deadline_cloud_for_maya
- name: layerDefaultFramesFrames
type: STRING
userInterface:
Expand Down
10 changes: 5 additions & 5 deletions job_bundle_output_tests/layers/scene/layers.ma
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
//Codeset: 1252
file -rdi 1 -ns "scene_file_to_reference" -rfn "scene_file_to_referenceRN" -op
"VERS|2023|UVER|undef|MADE|undef|CHNG|Wed, Aug 30, 2023 02:59:28 PM|ICON|undef|INFO|undef|OBJN|21|INCL|undef(|LUNI|cm|TUNI|film|AUNI|deg|TDUR|141120000|"
-typ "mayaBinary" "C:/Users/markw/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests/layers/scene/scene_file_to_reference.mb";
-typ "mayaBinary" "D:/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests/layers/scene/scene_file_to_reference.mb";
file -r -ns "scene_file_to_reference" -dr 1 -rfn "scene_file_to_referenceRN" -op
"VERS|2023|UVER|undef|MADE|undef|CHNG|Wed, Aug 30, 2023 02:59:28 PM|ICON|undef|INFO|undef|OBJN|21|INCL|undef(|LUNI|cm|TUNI|film|AUNI|deg|TDUR|141120000|"
-typ "mayaBinary" "C:/Users/markw/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests/layers/scene/scene_file_to_reference.mb";
-typ "mayaBinary" "D:/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests/layers/scene/scene_file_to_reference.mb";
requires maya "2023";
requires "stereoCamera" "10.0";
requires -nodeType "aiOptions" -nodeType "aiAOVDriver" -nodeType "aiAOVFilter" "mtoa" "5.1.0";
Expand Down Expand Up @@ -383,7 +383,7 @@ createNode simpleSelector -n "collection1Selector";
setAttr ".tf" 4;
createNode reference -n "scene_file_to_referenceRN";
rename -uid "C918C559-4A3A-DEDF-08EC-F796E312D2DE";
setAttr ".ed" -type "dataReferenceEdits"
setAttr ".ed" -type "dataReferenceEdits"
"scene_file_to_referenceRN"
"scene_file_to_referenceRN" 0;
setAttr ".ptag" -type "string" "";
Expand Down Expand Up @@ -521,7 +521,7 @@ createNode absOverride -n "renderable5";
setAttr ".es" yes;
createNode absUniqueOverride -n "imageFormat";
rename -uid "BF6B15F6-4AF8-A704-31D6-4CB52DCE4F3E";
addAttr -ci true -sn "atv" -ln "attrValue" -min 0 -max 63 -en "GIF:SoftImage:RLA:Tiff:Tiff16:SGI:Alias PIX:Maya IFF:JPEG:EPS:Maya16 IFF:Quantel=12:SGI16:Targa=19:Windows Bitmap:SGI Movie:Quicktime:AVI:MacPaint=30:PSD:PNG:QuickDraw:QuickTime Image:DDS:PSD Layered:EXR(exr)=40:IMF plugin=50:Custom Image Format:Macromedia SWF (swf)=60:Adobe Illustrator (ai):SVG (svg):Swift3DImporter (swft)"
addAttr -ci true -sn "atv" -ln "attrValue" -min 0 -max 63 -en "GIF:SoftImage:RLA:Tiff:Tiff16:SGI:Alias PIX:Maya IFF:JPEG:EPS:Maya16 IFF:Quantel=12:SGI16:Targa=19:Windows Bitmap:SGI Movie:Quicktime:AVI:MacPaint=30:PSD:PNG:QuickDraw:QuickTime Image:DDS:PSD Layered:EXR(exr)=40:IMF plugin=50:Custom Image Format:Macromedia SWF (swf)=60:Adobe Illustrator (ai):SVG (svg):Swift3DImporter (swft)"
-at "enum";
setAttr ".atr" -type "string" "imageFormat";
setAttr ".tgName" -type "string" "defaultRenderGlobals";
Expand Down Expand Up @@ -564,7 +564,7 @@ createNode simpleSelector -n "RenderSettingsCollection4Selector";
setAttr ".tf" 0;
createNode absUniqueOverride -n "imageFormat1";
rename -uid "370CC408-469D-C882-B66D-35B5096546AB";
addAttr -ci true -sn "atv" -ln "attrValue" -min 0 -max 63 -en "GIF:SoftImage:RLA:Tiff:Tiff16:SGI:Alias PIX:Maya IFF:JPEG:EPS:Maya16 IFF:Quantel=12:SGI16:Targa=19:Windows Bitmap:SGI Movie:Quicktime:AVI:MacPaint=30:PSD:PNG:QuickDraw:QuickTime Image:DDS:PSD Layered:EXR(exr)=40:IMF plugin=50:Custom Image Format:Macromedia SWF (swf)=60:Adobe Illustrator (ai):SVG (svg):Swift3DImporter (swft)"
addAttr -ci true -sn "atv" -ln "attrValue" -min 0 -max 63 -en "GIF:SoftImage:RLA:Tiff:Tiff16:SGI:Alias PIX:Maya IFF:JPEG:EPS:Maya16 IFF:Quantel=12:SGI16:Targa=19:Windows Bitmap:SGI Movie:Quicktime:AVI:MacPaint=30:PSD:PNG:QuickDraw:QuickTime Image:DDS:PSD Layered:EXR(exr)=40:IMF plugin=50:Custom Image Format:Macromedia SWF (swf)=60:Adobe Illustrator (ai):SVG (svg):Swift3DImporter (swft)"
-at "enum";
setAttr ".atr" -type "string" "imageFormat";
setAttr ".tgName" -type "string" "defaultRenderGlobals";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ assetReferences:
- /normalized/job/bundle/dir/images/layerTheFIrst
- /normalized/job/bundle/dir/images/layerTheSecond
- /normalized/job/bundle/dir/images/masterLayer
referencedPaths: []
Original file line number Diff line number Diff line change
@@ -1,12 +1,4 @@
parameterValues:
- name: deadline:priority
value: 50
- name: deadline:targetTaskRunStatus
value: READY
- name: deadline:maxFailedTasksCount
value: 20
- name: deadline:maxRetriesPerTask
value: 5
- name: MayaSceneFile
value: /normalized/job/bundle/dir/layers_no_variation.ma
- name: Frames
Expand All @@ -25,5 +17,11 @@ parameterValues:
value: 'true'
- name: ArnoldErrorOnLicenseFailure
value: 'false'
- name: RezPackages
value: mayaIO mtoa deadline_cloud_for_maya
- name: deadline:targetTaskRunStatus
value: READY
- name: deadline:maxFailedTasksCount
value: 20
- name: deadline:maxRetriesPerTask
value: 5
- name: deadline:priority
value: 50
Original file line number Diff line number Diff line change
Expand Up @@ -66,15 +66,6 @@ parameterDefinitions:
allowedValues:
- 'true'
- 'false'
- name: RezPackages
type: STRING
userInterface:
control: LINE_EDIT
label: Rez Packages
groupLabel: Software Environment
description: A space-separated list of Rez packages to install. Requires a Queue
Environment to handle the env creation.
default: mayaIO mtoa deadline_cloud_for_maya
- name: OutputFilePrefix
type: STRING
userInterface:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
//Codeset: 1252
file -rdi 1 -ns "scene_file_to_reference" -rfn "scene_file_to_referenceRN" -op
"VERS|2023|UVER|undef|MADE|undef|CHNG|Wed, Aug 30, 2023 02:59:28 PM|ICON|undef|INFO|undef|OBJN|21|INCL|undef(|LUNI|cm|TUNI|film|AUNI|deg|TDUR|141120000|"
-typ "mayaBinary" "C:/Users/markw/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests/layers_no_variation/scene/scene_file_to_reference.mb";
-typ "mayaBinary" "D:/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests/layers_no_variation/scene/scene_file_to_reference.mb";
file -r -ns "scene_file_to_reference" -dr 1 -rfn "scene_file_to_referenceRN" -op
"VERS|2023|UVER|undef|MADE|undef|CHNG|Wed, Aug 30, 2023 02:59:28 PM|ICON|undef|INFO|undef|OBJN|21|INCL|undef(|LUNI|cm|TUNI|film|AUNI|deg|TDUR|141120000|"
-typ "mayaBinary" "C:/Users/markw/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests/layers_no_variation/scene/scene_file_to_reference.mb";
-typ "mayaBinary" "D:/deadline-clients/deadline-cloud-for-maya/job_bundle_output_tests/layers_no_variation/scene/scene_file_to_reference.mb";
requires maya "2023";
requires "stereoCamera" "10.0";
requires -nodeType "aiOptions" -nodeType "aiAOVDriver" -nodeType "aiAOVFilter" "mtoa" "5.1.0";
Expand Down Expand Up @@ -202,7 +202,7 @@ createNode script -n "sceneConfigurationScriptNode";
setAttr ".st" 6;
createNode reference -n "scene_file_to_referenceRN";
rename -uid "E80FD48A-47F8-9D5B-2E45-52AD9FE11F56";
setAttr ".ed" -type "dataReferenceEdits"
setAttr ".ed" -type "dataReferenceEdits"
"scene_file_to_referenceRN"
"scene_file_to_referenceRN" 0;
setAttr ".ptag" -type "string" "";
Expand Down
12 changes: 10 additions & 2 deletions maya_submitter_plugin/plug-ins/DeadlineCloudForMaya.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@

__log__ = logging.getLogger("Deadline")
_registered_mel_commands: List[str] = []
_first_initialization: bool = True


def reload_modules(mod):
Expand All @@ -50,11 +51,18 @@ def initializePlugin(plugin):
"""
Initialize the DeadlineSubmitter plugin.
"""
global _registered_mel_commands
global _registered_mel_commands, _first_initialization
try:
plugin_obj = om.MFnPlugin(plugin, VENDOR, VERSION)

reload_modules(deadline.maya_submitter)
if _first_initialization:
_first_initialization = False
else:
# If a user unloaded and then reloaded the plugin, refresh
# some key module dependencies.
reload_modules(deadline.job_attachments)
reload_modules(deadline.client)
reload_modules(deadline.maya_submitter)

command_name = "DeadlineCloudSubmitter"
plugin_obj.registerCommand(command_name, mel_commands.DeadlineCloudSubmitterCmd)
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ license = ""
requires-python = ">=3.7"

dependencies = [
"deadline == 0.23.*",
"deadline == 0.26.*",
"openjd == 0.10.*",
]

Expand Down
2 changes: 1 addition & 1 deletion requirements-testing.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@ pytest-xdist ~= 3.3
twine ~= 4.0
mypy ~= 1.5
black ~= 23.9
ruff ~= 0.0.287
ruff ~= 0.0.290
moto[s3,sts] ~= 4.2
14 changes: 14 additions & 0 deletions scripts/build_wheels.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/bin/bash
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

set -euo pipefail

if [ ! -d wheels ]; then
mkdir wheels
fi
rm -f wheels/*

for dir in ../openjobio ../deadline-cloud ../deadline-cloud-for-maya; do
echo "Building $dir..."
python -m build --wheel --outdir ./wheels --skip-dependency-check $dir
done
8 changes: 2 additions & 6 deletions src/deadline/maya_submitter/data_classes.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,14 @@ class RenderSubmitterUISettings:
"""

submitter_name: str = field(default="Maya")

name: str = field(default="", metadata={"sticky": True})
description: str = field(default="", metadata={"sticky": True})
initial_status: str = field(default="READY", metadata={"sticky": True})
max_failed_tasks_count: int = field(default=20, metadata={"sticky": True})
max_retries_per_task: int = field(default=5, metadata={"sticky": True})
priority: int = field(default=50, metadata={"sticky": True})

override_frame_range: bool = field(default=False, metadata={"sticky": True})
frame_list: str = field(default="", metadata={"sticky": True})
project_path: str = field(default="")
output_path: str = field(default="")
override_rez_packages: bool = field(default=True)
rez_packages: str = field(default="")

input_filenames: list[str] = field(default_factory=list, metadata={"sticky": True})
input_directories: list[str] = field(default_factory=list, metadata={"sticky": True})
Expand Down
8 changes: 0 additions & 8 deletions src/deadline/maya_submitter/default_maya_job_template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -67,14 +67,6 @@ parameterDefinitions:
allowedValues:
- 'true'
- 'false'
- name: RezPackages
type: STRING
userInterface:
control: LINE_EDIT
label: Rez Packages
groupLabel: Software Environment
description: A space-separated list of Rez packages to install. Requires a Queue Environment to handle the env creation.
default: mayaIO mtoa deadline_cloud_for_maya
steps:
- name: Render
parameterSpace:
Expand Down
Loading

0 comments on commit 304e5be

Please sign in to comment.