Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run sqllogictests multiple times with random fuzzed configurations #9746

Open
devinjdangelo opened this issue Mar 23, 2024 · 1 comment
Open
Labels
enhancement New feature or request

Comments

@devinjdangelo
Copy link
Contributor

Is your feature request related to a problem or challenge?

A thought coming off of fixing #9736 is that we could likely proactively identify issues like this by modifying sqllogictests to run multiple times with a variety of randomized configs.

Describe the solution you'd like

Perhaps this function https://github.com/apache/arrow-datafusion/blob/6c6305159711fbca43ff7798faa52aaebcb2e7d3/datafusion/sqllogictest/src/test_context.rs#L67
could be modified to set random configs over a range of valid values.

Then this could be modified to run in a loop a set number of times based on a passed option with a default.

https://github.com/apache/arrow-datafusion/blob/6c6305159711fbca43ff7798faa52aaebcb2e7d3/datafusion/sqllogictest/bin/sqllogictests.rs#L90

Describe alternatives you've considered

If this results in tests taking too long, the default could be 1 run, but occasionally (prior to a release perhaps) a longer test with many runs could be run.

Additional context

No response

@devinjdangelo devinjdangelo added the enhancement New feature or request label Mar 23, 2024
@devinjdangelo devinjdangelo changed the title Run sqllogictests with random fuzzed configurations Run sqllogictests multiple times with random fuzzed configurations Mar 23, 2024
@Omega359
Copy link
Contributor

related: #913

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants