Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-47505][INFRA][3.4] Fix Pyspark-errors test jobs for branch-3.4 #45624

Closed

Conversation

panbingkun
Copy link
Contributor

@panbingkun panbingkun commented Mar 21, 2024

What changes were proposed in this pull request?

The pr aims to fix pyspark-errors test jobs for branch-3.4.

Why are the changes needed?

Fix pyspark-errors test jobs for branch-3.4.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass GA.

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the BUILD label Mar 21, 2024
@github-actions github-actions bot added the INFRA label Mar 21, 2024
@panbingkun
Copy link
Contributor Author

panbingkun commented Mar 21, 2024

@panbingkun
Copy link
Contributor Author

@dongjoon-hyun @HyukjinKwon
I finally found the reason for pyspark-errors's failure,
The root cause is that the disk space is full, so we need to backport this PR #42253 (@zhengruifeng ) to branch-3.4,

At present, the test of pyspark-errors on branch-3.4 has passed, as follows:
https://github.com/panbingkun/spark/actions/runs/8370751714/job/22918653065
image

@zhengruifeng
Copy link
Contributor

if disk space is full is the cause, then I think we can backport it.

@panbingkun
Copy link
Contributor Author

@panbingkun panbingkun changed the title [Don't Review] Only for address pyspark-errors for branch-3.4 [SPARK-47505][INFRA][3.4] Fix Pyspark-errors test for branch-3.4 Mar 21, 2024
@panbingkun panbingkun changed the title [SPARK-47505][INFRA][3.4] Fix Pyspark-errors test for branch-3.4 [SPARK-47505][INFRA][3.4] Fix Pyspark-errors test jobs for branch-3.4 Mar 21, 2024
@panbingkun panbingkun marked this pull request as ready for review March 21, 2024 13:32
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind. Let's merge this. Thank you, @panbingkun and @zhengruifeng .

dongjoon-hyun pushed a commit that referenced this pull request Mar 21, 2024
### What changes were proposed in this pull request?
The pr aims to fix `pyspark-errors` test jobs for branch-3.4.

### Why are the changes needed?
Fix `pyspark-errors` test jobs for branch-3.4.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Pass GA.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes #45624 from panbingkun/branch-3.4_fix_pyerrors.

Authored-by: panbingkun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
@dongjoon-hyun
Copy link
Member

Merged to branch-3.4.

zhengruifeng pushed a commit that referenced this pull request Jul 9, 2024
…ee_disk_space_container`

### What changes were proposed in this pull request?
This PR removed the check for the existence of `./dev/free_disk_space_container` before execution,  because `./dev/free_disk_space_container` has already been backported to branch-3.4 and branch-3.5 through #45624 and #43381,  so there is no need to check its existence before execution.

### Why are the changes needed?
Remove unnecessary existence check for `./dev/free_disk_space_container`.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #47263 from LuciferYang/SPARK-48840.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
ericm-db pushed a commit to ericm-db/spark that referenced this pull request Jul 10, 2024
…ee_disk_space_container`

### What changes were proposed in this pull request?
This PR removed the check for the existence of `./dev/free_disk_space_container` before execution,  because `./dev/free_disk_space_container` has already been backported to branch-3.4 and branch-3.5 through apache#45624 and apache#43381,  so there is no need to check its existence before execution.

### Why are the changes needed?
Remove unnecessary existence check for `./dev/free_disk_space_container`.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#47263 from LuciferYang/SPARK-48840.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
jingz-db pushed a commit to jingz-db/spark that referenced this pull request Jul 22, 2024
…ee_disk_space_container`

### What changes were proposed in this pull request?
This PR removed the check for the existence of `./dev/free_disk_space_container` before execution,  because `./dev/free_disk_space_container` has already been backported to branch-3.4 and branch-3.5 through apache#45624 and apache#43381,  so there is no need to check its existence before execution.

### Why are the changes needed?
Remove unnecessary existence check for `./dev/free_disk_space_container`.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#47263 from LuciferYang/SPARK-48840.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
szehon-ho pushed a commit to szehon-ho/spark that referenced this pull request Aug 7, 2024
### What changes were proposed in this pull request?
The pr aims to fix `pyspark-errors` test jobs for branch-3.4.

### Why are the changes needed?
Fix `pyspark-errors` test jobs for branch-3.4.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
Pass GA.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#45624 from panbingkun/branch-3.4_fix_pyerrors.

Authored-by: panbingkun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
attilapiros pushed a commit to attilapiros/spark that referenced this pull request Oct 4, 2024
…ee_disk_space_container`

### What changes were proposed in this pull request?
This PR removed the check for the existence of `./dev/free_disk_space_container` before execution,  because `./dev/free_disk_space_container` has already been backported to branch-3.4 and branch-3.5 through apache#45624 and apache#43381,  so there is no need to check its existence before execution.

### Why are the changes needed?
Remove unnecessary existence check for `./dev/free_disk_space_container`.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Pass GitHub Actions

### Was this patch authored or co-authored using generative AI tooling?
No

Closes apache#47263 from LuciferYang/SPARK-48840.

Authored-by: yangjie01 <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants