Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable the download of raw data files #334

Open
dalonsoa opened this issue Sep 20, 2024 · 4 comments
Open

Enable the download of raw data files #334

dalonsoa opened this issue Sep 20, 2024 · 4 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@dalonsoa
Copy link
Collaborator

The file cannot be downloaded because Django in production, by default, prevents from downloading raw data files for security reasons. They need to be served in a different way. If downloading the raw data files is a thing, I'll create an issue about that.

Originally posted by @dalonsoa in #332 (comment)

@ICHydro
Copy link
Collaborator

ICHydro commented Sep 20, 2024

There is no need for the user to download the raw data file. In fact, there is probably no need to keep the raw file after the data have been processed. Similarly, there is probably no need for the user to be able to change the data file of the import after it is created; they can just create a new import.

@dalonsoa
Copy link
Collaborator Author

If the raw data file is not to be kept, is there any reason for not deleting the whole data import object altogether once the import is successful? I don’t think it has much use if there’s no file.

@ICHydro
Copy link
Collaborator

ICHydro commented Sep 20, 2024

Indeed, I am reconsidering this. There is probably value in keeping the file to enable future reprocessing, and it can also serve as a place for users to keep the raw data files to ensure that they do not get lost, which enhances reproducibility and traceability. So let's keep this as an enhancement, including the possibility for users to download the original file.

@dalonsoa
Copy link
Collaborator Author

dalonsoa commented Oct 8, 2024

If we are going to keep the raw data and enable the user to download it, we might want to consider using a proper external service for this rather than storing the data in the VM, something like AWS S3 buckets or Azure Blob storage or something along those lines.

@ICHydro I'd suggest we leave this on hold until we can discuss the options and implications.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants