-
Notifications
You must be signed in to change notification settings - Fork 539
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding git workflows with test matrix and common workflows #363
base: master
Are you sure you want to change the base?
Conversation
Hi, Thanks for introducing these changes! We will have a look within the next two weeks. |
Sure, I will update it. There are few changes, will push it in weekend [when I get some time] |
@twollnik Also, When will be Spark 3.1.1 supported by deequ? |
see #366 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks super useful overall, thanks! We would like to add just the testing workflow for now to keep the scope of the PR small (and to be honest because we are no github experts and don't feel comfortable maintaining a lot of github config code). Thanks!
.github/workflows/build_test.yml
Outdated
hadoop: [2.7, 3.2] | ||
java: [11] | ||
spark-version: [2.4.7, 3.0.2] # 3.1.1 - not supported yet | ||
python-version: [3.7, 3.8] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please clarify why we need different python version and why do we need python at all?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Internally I had mono repo both deequ and python-deequ, I will remove it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Side note, Having one mono repo for both is better
Thanks so much for your quick reaction on this and for making changes. It will likely take some time for us to react to the updated PR, but we will get around to it! |
@twollnik Any update? and Also waiting for Spark 3.1 support as well. |
Ping!. |
Thanks for your patience on this. @sscdotopen could you please take a look and let us know what you think about the proposed changes? |
cc: @twollnik @TammoR