Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set expectations about large datasets #41

Open
mekline opened this issue Aug 7, 2024 · 0 comments
Open

Set expectations about large datasets #41

mekline opened this issue Aug 7, 2024 · 0 comments
Assignees
Labels
GeneralIssue Issue concerning the general psych-DS project

Comments

@mekline
Copy link
Collaborator

mekline commented Aug 7, 2024

TLDR

At some point, datasets become long enough that they cause high latency or even timeout on the webapp implementation.

We don't need to provide super duper duper duper performance (very large datasets probably want CLI validation for other reasons, maybe?) but we should give people an indication of what to expect.

Details

Once low-hanging fruit speed improvements are done, run some large datasets and add (somewhere on website, probably documentation?) some information like "Datasets with files larger than X may take Y amount of time", "Datasets with more than X files may take Z amount of time."

@mekline mekline added the GeneralIssue Issue concerning the general psych-DS project label Aug 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GeneralIssue Issue concerning the general psych-DS project
Projects
None yet
Development

No branches or pull requests

2 participants