-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(job_attachments)!: use correct profile for GetStorageProfileForQueue API #296
fix(job_attachments)!: use correct profile for GetStorageProfileForQueue API #296
Conversation
0167a9c
to
b698da6
Compare
if storage_profile_id: | ||
create_job_args["storageProfileId"] = storage_profile_id | ||
storage_profile_response = deadline.get_storage_profile_for_queue( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: since we're doing this in two places, we might want to make a function wrapper like what we've done in other places, like queue params https://github.com/aws-deadline/deadline-cloud/blob/mainline/src/deadline/client/api/_queue_parameters.py
Not blocking though, since we didn't hit the rule of 3 ;)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for the advise. I have dded a function wrapper get_storage_profile_for_queue
in client/api/_get_storage_profile_for_queue.py.
@@ -1333,8 +1333,9 @@ def test_upload_bucket_wrong_account(external_bucket: str, job_attachment_test: | |||
|
|||
# WHEN | |||
with pytest.raises( | |||
AssetSyncError, match=f"Error checking if object exists in bucket '{external_bucket}'" | |||
AssetSyncError, match=f"Error uploading binary file in bucket '{external_bucket}'" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm curious about the exception. When I ran this test, I received an "access denied" error. Was the related logic updated in this PR?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I also saw the "access denied" error message. There were no updates related to this in this PR. I found it a bit strange that this integ test was failing... The error message that's currently showing up seems correct though - it's expected that PutObject would be denied if we try to upload a manifest file to a bucket that we don't have access to.
2ebad98
to
6bd5c17
Compare
…eue API Signed-off-by: Gahyun Suh <[email protected]>
6bd5c17
to
6cc7596
Compare
What was the problem/requirement? (What/Why)
When submitting a job with a Storage Profile ID, the
GetStorageProfileForQueue
API was using the default AWS profile instead of the Deadline Cloud submitter's configured profile. If the two profiles pointed to different regions, the API couldn't find the storage profile resource, resulting in a job submission failureWhat was the solution? (How)
Modified the code to use the correct boto3 session (using the Deadline Cloud submitter's profile) when calling the
GetStorageProfileForQueue
API. The API is no longer called fromsrc/deadline/job_attachments/upload.py
. Instead, the API is called from the below places:and the
prepare_paths_for_upload()
function now receivesstorage_profile
object itself instead ofstorage_profile_id
.What is the impact of this change?
Fixes the job submission issue when involving a Storage Profile in a different region than the default AWS profile.
How was this change tested?
[default]
profile and Deadline Cloud default profile, where each profile is pointing to different regions.Was this change documented?
No.
Is this a breaking change?
Yes, there are changes in the public interface (
S3AssetManager
'sprepare_paths_for_upload()
).By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.