-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Look into User Datasets issue "dataset does not contain both a dataset.json file and a meta.json file" #39
Comments
Here's the code that is throwing. It could be some kind of eventual consistency issue with IRODS is it seems like the code isn't finding the files but we're finding them after the fact. WDK/Model/src/main/java/org/gusdb/wdk/model/user/dataset/irods/IrodsUserDatasetSession.java Lines 625 to 626 in 6d9815f
|
It looks like the tarball is unpacked into a staging area and then rsynced into the users directory. https://github.com/VEuPathDB/EuPathDBIrods/blob/master/Scripts/ud.re#L119-L155 We assume when getting a list of user datasets for a user, that if the directory exists, it is a valid dataset. It's possible that the rsync is partially complete without the WDK/Model/src/main/java/org/gusdb/wdk/model/user/dataset/irods/IrodsUserDatasetSession.java Lines 192 to 194 in 6d9815f
It's worth checking if this particular dataset was listed soon after it's creation, or to check the IRODS rules logs if we have them to see if they corroborate this picture. |
Looks like the error came in on |
Looks like the write to IRODS came in 2 seconds earlier:
|
We confirmed that the Next step is to follow up with front-end folks to see if they have any idea why we might be running into this issue now since theoretically this bug would have been there forever. |
i experienced this myself.. i think when you go quickly after upload to the All datasets page... it is clearly some kind of our of sync situation.. need more detail |
E-mail from Cristina with details:
The text was updated successfully, but these errors were encountered: