Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Subtask] integrate pytest with gradle to run python test with CICD #2518

Closed
Tracked by #2113
shaofengshi opened this issue Mar 13, 2024 · 4 comments · Fixed by #2684 or #2753
Closed
Tracked by #2113

[Subtask] integrate pytest with gradle to run python test with CICD #2518

shaofengshi opened this issue Mar 13, 2024 · 4 comments · Fixed by #2684 or #2753
Assignees
Labels
good first issue Good for newcomers subtask Subtasks of umbrella issue

Comments

@shaofengshi
Copy link
Contributor

shaofengshi commented Mar 13, 2024

Describe the subtask

As discussed in #2280, integrate pytest with gradle, so that we can use gradle to build/test the python client module

This might be a good reference: https://attx-project.github.io/Building-with-Gradle.html

Parent issue

#2229

@shaofengshi shaofengshi added the subtask Subtasks of umbrella issue label Mar 13, 2024
@shaofengshi shaofengshi added the good first issue Good for newcomers label Mar 14, 2024
@coolderli
Copy link
Contributor

coolderli commented Mar 18, 2024

@shaofengshi Hi, Why do we use gradle to integrate pytest. We can use the Github CI to finish it. If we develop the python module, we can use pytest to have a test locally.

@shaofengshi
Copy link
Contributor Author

shaofengshi commented Mar 19, 2024

@shaofengshi Hi, Why do we use gradle to integrate pytest. We can use the Github CI to finish it. If we develop the python module, we can use pytest to have a test locally.

@coolderli , with gradle integration, user can run the build in local, without dependency on Github. Is that reasonable? Just let me know if that will bring potential issue.

@coolderli
Copy link
Contributor

@shaofengshi Hi, Why do we use gradle to integrate pytest. We can use the Github CI to finish it. If we develop the python module, we can use pytest to have a test locally.

@coolderli , with gradle integration, user can run the build in local, without dependency on Github. Is that reasonable? Just let me know if that will bring potential issue.

@shaofengshi Users can use pytest to have a test in local env. Some Python users may not be familiar with gradle. I'd like to use some Python tools to build and test the Python project.

@shaofengshi
Copy link
Contributor Author

@shaofengshi Hi, Why do we use gradle to integrate pytest. We can use the Github CI to finish it. If we develop the python module, we can use pytest to have a test locally.

@coolderli , with gradle integration, user can run the build in local, without dependency on Github. Is that reasonable? Just let me know if that will bring potential issue.

@shaofengshi Users can use pytest to have a test in local env. Some Python users may not be familiar with gradle. I'd like to use some Python tools to build and test the Python project.

Hi Peidian, I believe python user still use pytest to run the test. What Jerry suggests is, when run a "gradle build" command, it will call the "pytest" command in the client-python module. Not sure whether this solves your concern. @coolderli

jerryshao pushed a commit that referenced this issue Mar 27, 2024
### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: #2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
shaofengshi added a commit that referenced this issue Mar 28, 2024
yuqi1129 pushed a commit that referenced this issue Mar 28, 2024
@shaofengshi shaofengshi reopened this Mar 29, 2024
xunliu pushed a commit to xunliu/gravitino that referenced this issue Mar 30, 2024
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
@jerryshao jerryshao added this to the Gravitino 0.5.0 milestone Apr 1, 2024
xunliu pushed a commit to xunliu/gravitino that referenced this issue Apr 1, 2024
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
xunliu pushed a commit to xunliu/gravitino that referenced this issue Apr 2, 2024
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
xunliu pushed a commit to xunliu/gravitino that referenced this issue Apr 3, 2024
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
xunliu pushed a commit to xunliu/gravitino that referenced this issue Apr 3, 2024
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
xunliu pushed a commit to xunliu/gravitino that referenced this issue Apr 3, 2024
…e#2684)

### What changes were proposed in this pull request?

 - use Gradle to manage Python build.
 - add Github CI
 - `./gradlew clean clients:client-python:test`

### Why are the changes needed?

Fix: apache#2518 

### Does this PR introduce _any_ user-facing change?

- no

### How was this patch tested?

- `./gradlew clean clients:client-python:test`
xunliu added a commit that referenced this issue Apr 3, 2024
…st with CICD (#2753)

### What changes were proposed in this pull request?

1. Use Gradle python plugin to manage python module
2. Automatic download conda and create python env in
`gravitino/.gradle/python` directory
3. Use multiple python version `3.8`, `3.9`, `3.10`,and `3.11` test ITs
in Github Action
4. Support `gradlew clean` & `gradlew build` & `gradlew
:client:client-python:test -PpythonVersion=3.8`

### Why are the changes needed?

Use Gradle manager and development Python client module.

Fix: #2518 

### Does this PR introduce _any_ user-facing change?

1. Use `pythonVersion` Gradle parameter setting Python version to
compile and test Python client, `gradlew :client:client-python:test
-PpythonVersion=3.8`
2. Use `skipPythonITs` Gradle parameter to control if skip Python test
cases.

### How was this patch tested?

CI Passed.

---------

Co-authored-by: Peidian li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment