Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] [spark-hive-connector] support scala 2.13 for spark 3.3 version #6431

Closed
3 of 4 tasks
FANNG1 opened this issue May 28, 2024 · 0 comments
Closed
3 of 4 tasks

[FEATURE] [spark-hive-connector] support scala 2.13 for spark 3.3 version #6431

FANNG1 opened this issue May 28, 2024 · 0 comments
Labels
kind:feature Feature request priority:major

Comments

@FANNG1
Copy link

FANNG1 commented May 28, 2024 •
edited
Loading

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the feature

support scala 2.13 for spark 3.3 version

Motivation

It's hard for downstream code using Kyuubi spark hive connector like Gravitino to support multi scala versions in apache/gravitino#3608

Describe the solution

compile with scala 2.13 and publish to Maven

Additional context

No response

Are you willing to submit PR?

  • Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
  • No. I cannot submit a PR at this time.
@pan3793 pan3793 mentioned this issue Jun 1, 2024
Closed
4 tasks
@pan3793 pan3793 closed this as completed in 3ed912f Jun 1, 2024
pan3793 pushed a commit that referenced this issue Jun 1, 2024
@zhouyifan279 @pan3793
[KYUUBI #6247] Make KSHC binary compatible with multiple Spark versions
126ff4e 
# 🔍 Description
## Issue References 🔗

This pull request closes #6247

This also closes #6431

## Describe Your Solution 🔧
Add a job `spark-connector-cross-version-test` in GitHub Actions to:
1. Build KSHC package with maven opt `-Pspark-3.5`
2. Run KSHC tests with maven opt `-Pspark-3.3` and `-Pspark-3.4` and KSHC package built in step 1
3. Fix the binary-compatible issue via reflection.

## Types of changes 🔖

- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)

## Test Plan 🧪

Pass GHA.

---

# Checklist 📝

- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)

**Be nice. Be informative.**

Closes #6436 from zhouyifan279/kshc-cross-version-test.

Closes #6247

d3ac2ef [zhouyifan279] Tune the KSHC code to fix binary-compatible issues
4e14edc [zhouyifan279] Fix invalid unit-tests-log name
56ca45d [zhouyifan279] Fix invalid unit-tests-log name
4c5ab7b [zhouyifan279] Update test log name
8a84e88 [zhouyifan279] Add matrix scala
17cb671 [zhouyifan279] [KYUUBI #6247] KSHC cross-version test

Authored-by: zhouyifan279 <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
(cherry picked from commit 3ed912f)
Signed-off-by: Cheng Pan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind:feature Feature request priority:major
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[KYUUBI #6247] Make KSHC binary compatible with multiple Spark versions zhouyifan279/kyuubi
1 participant
@FANNG1