Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Free up disk space #6047

Closed
wants to merge 8 commits into from
Closed

[CI] Free up disk space #6047

wants to merge 8 commits into from

Conversation

pan3793
Copy link
Member

@pan3793 pan3793 commented Feb 5, 2024

🔍 Description

Issue References 🔗

I saw the K8s IT failed with

Error: No space left on device : '/home/runner/runners/2.312.0/_diag/pages/131e7057-723d-4281-b296-b876237f4318_21774d93-c106-57cd-810b-f0e6bf96d865_1.log'

Describe Your Solution 🔧

Borrow the solution from Apache Spark, just uninstall the un-used large pre-installed packages.

Types of changes 🔖

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Wait CI report

Checklist 📝

Be nice. Be informative.

@github-actions github-actions bot added the kind:infra license, community building, project builds, asf infra related, etc. label Feb 5, 2024
@pan3793 pan3793 changed the title [CI] Free up disk space for K8s IT [CI] Free up disk space Feb 5, 2024
pan3793
pan3793 commented Feb 5, 2024
View reviewed changes
.github/scripts/free_disk_space.sh
sudo apt-get remove --purge -y '^dotnet-.*'
sudo apt-get remove --purge -y '^llvm-.*'
sudo apt-get remove --purge -y 'php.*'
sudo apt-get remove --purge -y '^temurin-\d{n,}.*'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it removes temurin-11-jdk, temurin-17-jdk, temurin-21-jdk while skips the temurin-8-jdk, to avoid breaking other packages' dependency requirements

@pan3793 pan3793 self-assigned this Feb 5, 2024
@pan3793 pan3793 added this to the v1.7.4 milestone Feb 5, 2024
@pan3793 pan3793 closed this in 5f99ec8 Feb 5, 2024
pan3793 added a commit that referenced this pull request Feb 5, 2024
@pan3793
[KYUUBI #6047] [CI] Free up disk space
054db88 
I saw [the K8s IT](https://github.com/apache/kyuubi/actions/runs/7776268590/job/21203214430?pr=6045) failed with

```
Error: No space left on device : '/home/runner/runners/2.312.0/_diag/pages/131e7057-723d-4281-b296-b876237f4318_21774d93-c106-57cd-810b-f0e6bf96d865_1.log'
```

Borrow the solution from Apache Spark, just uninstall the un-used large pre-installed packages.

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

Wait CI report

---

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6047 from pan3793/ci-space.

Closes #6047

272df27 [Cheng Pan] nit
8c78115 [Cheng Pan] nit
a9c8b03 [Cheng Pan] nit
ab4c3e4 [Cheng Pan] keep jdk8
9d1ec4e [Cheng Pan] extend
47f90b9 [Cheng Pan] nit
30b82cd [Cheng Pan] fix
352689f [Cheng Pan] [CI] Free up disk space for K8s IT

Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit 5f99ec8)
Signed-off-by: Cheng Pan <[email protected]>
pan3793 added a commit that referenced this pull request Feb 5, 2024
@pan3793
[KYUUBI #6047] [CI] Free up disk space
06cc3db 
I saw [the K8s IT](https://github.com/apache/kyuubi/actions/runs/7776268590/job/21203214430?pr=6045) failed with

```
Error: No space left on device : '/home/runner/runners/2.312.0/_diag/pages/131e7057-723d-4281-b296-b876237f4318_21774d93-c106-57cd-810b-f0e6bf96d865_1.log'
```

Borrow the solution from Apache Spark, just uninstall the un-used large pre-installed packages.

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

Wait CI report

---

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6047 from pan3793/ci-space.

Closes #6047

272df27 [Cheng Pan] nit
8c78115 [Cheng Pan] nit
a9c8b03 [Cheng Pan] nit
ab4c3e4 [Cheng Pan] keep jdk8
9d1ec4e [Cheng Pan] extend
47f90b9 [Cheng Pan] nit
30b82cd [Cheng Pan] fix
352689f [Cheng Pan] [CI] Free up disk space for K8s IT

Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit 5f99ec8)
Signed-off-by: Cheng Pan <[email protected]>
@pan3793
Copy link
Member Author

pan3793 commented Feb 5, 2024

Merged to master/1.8/1.7

pan3793 added a commit that referenced this pull request Feb 17, 2024
@pan3793
[KYUUBI #6047] [CI] Free up disk space
9740b75 
I saw [the K8s IT](https://github.com/apache/kyuubi/actions/runs/7776268590/job/21203214430?pr=6045) failed with

```
Error: No space left on device : '/home/runner/runners/2.312.0/_diag/pages/131e7057-723d-4281-b296-b876237f4318_21774d93-c106-57cd-810b-f0e6bf96d865_1.log'
```

Borrow the solution from Apache Spark, just uninstall the un-used large pre-installed packages.

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

Wait CI report

---

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6047 from pan3793/ci-space.

Closes #6047

272df27 [Cheng Pan] nit
8c78115 [Cheng Pan] nit
a9c8b03 [Cheng Pan] nit
ab4c3e4 [Cheng Pan] keep jdk8
9d1ec4e [Cheng Pan] extend
47f90b9 [Cheng Pan] nit
30b82cd [Cheng Pan] fix
352689f [Cheng Pan] [CI] Free up disk space for K8s IT

Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit 5f99ec8)
Signed-off-by: Cheng Pan <[email protected]>
pan3793 added a commit that referenced this pull request Feb 22, 2024
@pan3793
[KYUUBI #6074] Add a script to simplify the process of creating relea…
2c70c67 
…se notes

# 🔍 Description
## Issue References 🔗

Currently, we use a rather primitive way to manually write release notes from scratch, and some of the mechanical and repetitive work can be simplified by the scripts.

## Describe Your Solution 🔧

Adds a script to simplify the process of creating release notes.

Note: it just simplifies some processes, the release manager still needs to tune the outputs by hand.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

```
RELEASE_TAG=v1.8.1 PREVIOUS_RELEASE_TAG=v1.8.0 build/release/pre_gen_release_notes.py
```

```
$ head build/release/commits-v1.8.1.txt
[KYUUBI #5981] Deploy Spark Hive connector with Scala 2.13 to Maven Central
[KYUUBI #6058] Make Jetty server stop timeout configurable
[KYUUBI #5952][1.8] Disconnect connections without running operations after engine maxlife time graceful period
[KYUUBI #6048] Assign serviceNode and add volatile for variables
[KYUUBI #5991] Error on reading Atlas properties composed of multi values
[KYUUBI #6045] [REST] Sync the AdminRestApi with the AdminResource Apis
[KYUUBI #6047] [CI] Free up disk space
[KYUUBI #6036] JDBC driver conditional sets fetchSize on opening session
[KYUUBI #6028] Exited spark-submit process should not block batch submit queue
[KYUUBI #6018] Speed up GetTables operation for Spark session catalog
```

```
$ head build/release/contributors-v1.8.1.txt
* Shaoyun Chen        -- [KYUUBI #5857][KYUUBI #5720][KYUUBI #5785][KYUUBI #5617]
* Chao Chen           -- [KYUUBI #5750]
* Flyangz             -- [KYUUBI #5832]
* Pengqi Li           -- [KYUUBI #5713]
* Bowen Liang         -- [KYUUBI #5730][KYUUBI #5802][KYUUBI #5767][KYUUBI #5831][KYUUBI #5801][KYUUBI #5754][KYUUBI #5626][KYUUBI #5811][KYUUBI #5853][KYUUBI #5765]
* Paul Lin            -- [KYUUBI #5799][KYUUBI #5814]
* Senmiao Liu         -- [KYUUBI #5969][KYUUBI #5244]
* Xiao Liu            -- [KYUUBI #5962]
* Peiyue Liu          -- [KYUUBI #5331]
* Junjie Ma           -- [KYUUBI #5789]
```
---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6074 from pan3793/release-script.

Closes #6074

3d5ec20 [Cheng Pan] credits
1765279 [Cheng Pan] Add a script to simplify the process of creating release notes

Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
zhaohehuhu pushed a commit to zhaohehuhu/incubator-kyuubi that referenced this pull request Mar 21, 2024
@pan3793 @zhaohehuhu
[KYUUBI apache#6047] [CI] Free up disk space
d73e022 
# 🔍 Description
## Issue References 🔗

I saw [the K8s IT](https://github.com/apache/kyuubi/actions/runs/7776268590/job/21203214430?pr=6045) failed with

```
Error: No space left on device : '/home/runner/runners/2.312.0/_diag/pages/131e7057-723d-4281-b296-b876237f4318_21774d93-c106-57cd-810b-f0e6bf96d865_1.log'
```

## Describe Your Solution 🔧

Borrow the solution from Apache Spark, just uninstall the un-used large pre-installed packages.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Wait CI report

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes apache#6047 from pan3793/ci-space.

Closes apache#6047

272df27 [Cheng Pan] nit
8c78115 [Cheng Pan] nit
a9c8b03 [Cheng Pan] nit
ab4c3e4 [Cheng Pan] keep jdk8
9d1ec4e [Cheng Pan] extend
47f90b9 [Cheng Pan] nit
30b82cd [Cheng Pan] fix
352689f [Cheng Pan] [CI] Free up disk space for K8s IT

Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
zhaohehuhu pushed a commit to zhaohehuhu/incubator-kyuubi that referenced this pull request Mar 21, 2024
@pan3793 @zhaohehuhu
[KYUUBI apache#6074] Add a script to simplify the process of creating…
9f3ab85 
… release notes

# 🔍 Description
## Issue References 🔗

Currently, we use a rather primitive way to manually write release notes from scratch, and some of the mechanical and repetitive work can be simplified by the scripts.

## Describe Your Solution 🔧

Adds a script to simplify the process of creating release notes.

Note: it just simplifies some processes, the release manager still needs to tune the outputs by hand.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

```
RELEASE_TAG=v1.8.1 PREVIOUS_RELEASE_TAG=v1.8.0 build/release/pre_gen_release_notes.py
```

```
$ head build/release/commits-v1.8.1.txt
[KYUUBI apache#5981] Deploy Spark Hive connector with Scala 2.13 to Maven Central
[KYUUBI apache#6058] Make Jetty server stop timeout configurable
[KYUUBI apache#5952][1.8] Disconnect connections without running operations after engine maxlife time graceful period
[KYUUBI apache#6048] Assign serviceNode and add volatile for variables
[KYUUBI apache#5991] Error on reading Atlas properties composed of multi values
[KYUUBI apache#6045] [REST] Sync the AdminRestApi with the AdminResource Apis
[KYUUBI apache#6047] [CI] Free up disk space
[KYUUBI apache#6036] JDBC driver conditional sets fetchSize on opening session
[KYUUBI apache#6028] Exited spark-submit process should not block batch submit queue
[KYUUBI apache#6018] Speed up GetTables operation for Spark session catalog
```

```
$ head build/release/contributors-v1.8.1.txt
* Shaoyun Chen        -- [KYUUBI apache#5857][KYUUBI apache#5720][KYUUBI apache#5785][KYUUBI apache#5617]
* Chao Chen           -- [KYUUBI apache#5750]
* Flyangz             -- [KYUUBI apache#5832]
* Pengqi Li           -- [KYUUBI apache#5713]
* Bowen Liang         -- [KYUUBI apache#5730][KYUUBI apache#5802][KYUUBI apache#5767][KYUUBI apache#5831][KYUUBI apache#5801][KYUUBI apache#5754][KYUUBI apache#5626][KYUUBI apache#5811][KYUUBI apache#5853][KYUUBI apache#5765]
* Paul Lin            -- [KYUUBI apache#5799][KYUUBI apache#5814]
* Senmiao Liu         -- [KYUUBI apache#5969][KYUUBI apache#5244]
* Xiao Liu            -- [KYUUBI apache#5962]
* Peiyue Liu          -- [KYUUBI apache#5331]
* Junjie Ma           -- [KYUUBI apache#5789]
```
---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes apache#6074 from pan3793/release-script.

Closes apache#6074

3d5ec20 [Cheng Pan] credits
1765279 [Cheng Pan] Add a script to simplify the process of creating release notes

Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zwangsheng zwangsheng zwangsheng approved these changes

@ulysses-you ulysses-you ulysses-you approved these changes

Assignees

@pan3793 pan3793

Labels
kind:infra license, community building, project builds, asf infra related, etc.
Projects
None yet
Milestone
v1.7.4
Development

Successfully merging this pull request may close these issues.
3 participants
@pan3793 @ulysses-you @zwangsheng