Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checksum Matches Existing Object - Json submit endpoint #340

Closed
brynnz22 opened this issue Oct 31, 2023 · 4 comments · Fixed by #347 or #371
Closed

Checksum Matches Existing Object - Json submit endpoint #340

brynnz22 opened this issue Oct 31, 2023 · 4 comments · Fixed by #347 or #371
Assignees
Labels
X SMALL Less than 1 day

Comments

@brynnz22
Copy link
Contributor

When trying to re-submit a JSON that was previously deleted from production, I encounter the error:

detail": "provided checksum matches existing object: {'checksums.type': 'sha256', 'checksums.checksum': '9eaece59f58b4d58008f2efcf556736c0f5a0cbe16311206d7f6b127e9cfc580'}

@dwinston
Copy link
Collaborator

There is a workaround — I can locate the past dagster job that sourced your submitted json and then re-run it (e.g. via the dagster dagit UI). However, I recognize that re-submitting is a legitimate use case, so I’d rather change the code to e.g. include a timestamp in the submission so that the resulting payload has a unique checksum.

@dwinston dwinston moved this to Bench in Polyneme mixset Nov 2, 2023
@dwinston dwinston moved this from Bench to Lineup in Polyneme mixset Nov 2, 2023
dwinston added a commit that referenced this issue Nov 2, 2023
re-submission of "same" changes is a valid use case

closes #340
@dwinston dwinston moved this from Lineup to On base in Polyneme mixset Nov 2, 2023
dwinston added a commit that referenced this issue Nov 2, 2023
re-submission of "same" changes is a valid use case

closes #340
@github-project-automation github-project-automation bot moved this from On base to Scored in Polyneme mixset Nov 2, 2023
dwinston added a commit that referenced this issue Nov 3, 2023
re-submission of "same" changes is a valid use case

closes #340
@brynnz22
Copy link
Contributor Author

brynnz22 commented Nov 3, 2023

Thank you @dwinston apologies for not reviewing this sooner. I attempted to resubmit a json just now that I had previously submitted and then deleted from MongoDb weeks ago and I get the same error:

{ "detail": "provided checksum matches existing object: {'checksums.type': 'sha256', 'checksums.checksum': '9eaece59f58b4d58008f2efcf556736c0f5a0cbe16311206d7f6b127e9cfc580'}" }

@eecavanna
Copy link
Collaborator

eecavanna commented Nov 3, 2023

I'm posting this comment to confirm that the production Runtime API is currently at commit: 0c69b60

image

That commit comes after (on the main branch) the commit that introduced the fix for this issue, so I expect the fix to be present in the production Runtime API.

@dwinston dwinston moved this from Scored to Lineup in Polyneme mixset Nov 4, 2023
@dwinston dwinston moved this from Lineup to On base in Polyneme mixset Nov 4, 2023
@dwinston dwinston moved this from On base to Scored in Polyneme mixset Nov 4, 2023
@dwinston
Copy link
Collaborator

dwinston commented Nov 8, 2023

plan: add an optional timestamp kwarg to

def drs_metadata_for(filepath, base=None):
, which will pass it to a optional-timestamp-kwarg-added
def sha256hash_from_file(file_path: str):
, so that the same content submitted at a different time generates a different checksum. Finally, add a str of the current time to the invocation at .

@dwinston dwinston moved this from Scored to Lineup in Polyneme mixset Nov 8, 2023
@dwinston dwinston moved this from Lineup to At bat in Polyneme mixset Nov 13, 2023
@dwinston dwinston added the X SMALL Less than 1 day label Nov 13, 2023
@PeopleMakeCulture PeopleMakeCulture linked a pull request Nov 13, 2023 that will close this issue
@github-project-automation github-project-automation bot moved this from At bat to Scored in Polyneme mixset Nov 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
X SMALL Less than 1 day
Projects
Archived in project
3 participants