Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add frame range to Blender command in CueSubmit #1337

Merged
merged 12 commits into from
May 21, 2024
+82 −4
Merged

Add frame range to Blender command in CueSubmit #1337

Changes from 3 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
2009e12
Add frame range to Blender render command
n-jay Jan 19, 2024
2ff33f5
Fix linting issue
n-jay Jan 19, 2024
29fdbeb
Merge branch 'master' of https://github.com/AcademySoftwareFoundation…
n-jay Jan 28, 2024
507a11a
Add variables for frame range tokens
n-jay Feb 2, 2024
ce618a0
Update Blender command with tokens for
n-jay Feb 2, 2024
3c0f2ff
Remove unused import
n-jay Feb 2, 2024
d3d7c6d
Add suffix to frame start/end variables
n-jay Feb 17, 2024
2d254c7
Update comment
n-jay Feb 18, 2024
2d4ba88
Add unit tests
n-jay Feb 26, 2024
a17be62
Add missing services line to blender unit test layer data
n-jay May 10, 2024
ee2af1e
Merge branch 'master' of https://github.com/AcademySoftwareFoundation…
n-jay May 10, 2024
41b6e81
Merge branch 'master' of https://github.com/AcademySoftwareFoundation…
n-jay May 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions cuesubmit/cuesubmit/Submission.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
from __future__ import absolute_import

from builtins import str
import re

import outline
import outline.cuerun
Expand Down Expand Up @@ -63,6 +64,7 @@ def buildBlenderCmd(layerData):
blenderFile = layerData.cmd.get('blenderFile')
outputPath = layerData.cmd.get('outputPath')
outputFormat = layerData.cmd.get('outputFormat')
frameRange = layerData.layerRange
if not blenderFile:
raise ValueError('No Blender file provided. Cannot submit job.')

Expand All @@ -72,8 +74,19 @@ def buildBlenderCmd(layerData):
renderCommand += ' -o {}'.format(outputPath)
if outputFormat:
renderCommand += ' -F {}'.format(outputFormat)
# The render frame must come after the scene and output
renderCommand += ' -f {frameToken}'.format(frameToken=Constants.FRAME_TOKEN)
if frameRange:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great if you could add unit tests to cover both the single-frame and start/end frame cases.

https://github.com/AcademySoftwareFoundation/OpenCue/blob/master/cuesubmit/tests/Submission_tests.py

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in 2d4ba88

@bcipriano however, I'm having a bit of trouble with lines #178 and #153 of Submission_tests.py
The test fails when I set it to blender but passes when I set it to shell. This is despite the layerType being set as Blender. Any thoughts on this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping @bcipriano

# Checks if frame range is in the correct format
if re.match(r'^\d+$', frameRange):
renderCommand += ' -f {}'.format(frameRange)
elif re.match(r'^\d+-\d+$', frameRange):
startFrame, endFrame = map(int, frameRange.split("-"))
renderCommand += (' -s {startFrame} -e {endFrame} -a'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm reading this correctly, I think this will result in a layer command with the start/end frames hardcoded. I.e., if the frame range is 1-5 the command will contain -s 1 -e 5 -a.

Compare this to the code prior to this change, which instead inserts the #IFRAME# token to act as a placeholder for the frame.

I think instead, we want to use tokens for start frame / end frame:

OpenCue/cuesubmit/cuesubmit/Constants.py

Lines 45 to 54 in 5b9defd

COMMAND_TOKENS = {'#ZFRAME#': 'Current frame with a padding of 4',
'#IFRAME#': 'Current frame',
'#FRAME_START#': 'First frame of chunk',
'#FRAME_END#': 'Last frame of chunk',
'#FRAME_CHUNK#': 'Chunk size',
'#FRAMESPEC#': 'Full frame range',
'#LAYER#': 'Name of the Layer',
'#JOB#': 'Name of the Job',
'#FRAME#': 'Name of the Frame'
}

These tokens should allow Cuebot to swap in the correct frame numbers for each task based on the chunk size.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thread relate error on cuebot is normal, it's related to a race condition that's handled first come first.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bcipriano noted.
I was under the impression that the Frame Spec field solely defined the frame range (1-5), hence why I got it directly from that but had no idea about the Chunk Size.

...should allow Cuebot to swap in the correct frame numbers for each task based on the chunk size.

Got it.
To make sure I follow correctly; Frame Spec can be populated with a frame range such as 1-5, and then Chunk Size can be set as an integer value (eg: 2) which defines the number of frames within that 1-5 range that will be processed per RQD node, yes?
And the dedicated #tokens can be used to substitute the respective values.

Sorry, I'm not very familiar with the concept of chunk sizes in render farms, hence the confusion 😅

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you've got that right.

To extend the same example, if the frame range is 1-5 and the chunk size is 2, the set of tasks for that layer should be:

  • Frames 1-2
  • Frames 3-4
  • Frame 5 (don't go past the last frame, despite the chunk size setting)

Cuebot should handle all of this already -- dividing up the frame range into chunks and the replacement of the #FRAME_START# / #FRAME_END# tokens -- so no need to do any of those calculations at this point in the code.

Kind of confusingly, in OpenCue each task is called a "frame". This is because when OpenCue was originally written, it only operated on single frames -- chunk size was always assumed to be 1. We added the chunk size concept later, but the naming stayed the same.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bcipriano I've resolved this in ce618a0.

Verified on test animation.
image

.format(startFrame=startFrame, endFrame=endFrame))
else:
raise ValueError('Invalid frameRange format: {}'.format(frameRange))
else:
# The render frame must come after the scene and output
renderCommand += ' -f {frameToken}'.format(frameToken=Constants.FRAME_TOKEN)
return renderCommand


Expand Down
Loading