update actions version #3
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Keep deps (JNI + private) dependencies as 24.04-SNAPSHOT util they're available next week. Added TODO (NVIDIA#10256) to remind us to bump up deps version to 24.06.0-SNAPSHOT. Signed-off-by: Tim Liu <[email protected]>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
Signed-off-by: YanxuanLiu <[email protected]>
* Add fix for removing internal metadata information from 350 shim Signed-off-by: Partho Sarthi <[email protected]> * Create a shim helper class wth the relevant change instead of duplicating code Signed-off-by: Partho Sarthi <[email protected]> * Remove extra whitespace Signed-off-by: Partho Sarthi <[email protected]> --------- Signed-off-by: Partho Sarthi <[email protected]>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
* Use new kernel for getJsonObject Signed-off-by: Haoyang Li <[email protected]> * Use table to pass parsed path Signed-off-by: Haoyang Li <[email protected]> * use list/vector of instruction objects Signed-off-by: Haoyang Li <[email protected]> * fallback when nested too long Signed-off-by: Haoyang Li <[email protected]> * cancel xfail cases Signed-off-by: Haoyang Li <[email protected]> * cancel xfail cases Signed-off-by: Haoyang Li <[email protected]> * generated and modified docs Signed-off-by: Haoyang Li <[email protected]> * wip Signed-off-by: Haoyang Li <[email protected]> * wip Signed-off-by: Haoyang Li <[email protected]> * apply jni change and remove xpass Signed-off-by: Haoyang Li <[email protected]> * Adds test cases Signed-off-by: Haoyang Li <[email protected]> --------- Signed-off-by: Haoyang Li <[email protected]>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
Signed-off-by: Andy Grove <[email protected]>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
Signed-off-by: Jim Brennan <[email protected]>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
…10637) * Cleanup async state when multi-threaded shuffle readers fail Signed-off-by: Alessandro Bellina <[email protected]> --------- Signed-off-by: Alessandro Bellina <[email protected]>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
To fix: NVIDIA#10624 As latest maven-gpg-plugin has the issue "error 401 Unauthorized" when deploy files onto Sonatype repo, we pin it to the stable version 3.1.0 to unblock our release process. Signed-off-by: Tim Liu <[email protected]>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
)" (NVIDIA#10657) This reverts commit c28c7fa. Signed-off-by: Jason Lowe <[email protected]>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
Fixes NVIDIA#10487 Signed-off-by: Gera Shegalov <[email protected]>
Signed-off-by: Tim Liu <[email protected]>
Signed-off-by: Tim Liu <[email protected]>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
Signed-off-by: Niranjan Artal <[email protected]>
To fix: NVIDIA#10665 Signed-off-by: Tim Liu <[email protected]>
Signed-off-by: Jason Lowe <[email protected]>
Signed-off-by: liurenjie1024 <[email protected]>
Signed-off-by: YanxuanLiu <[email protected]>
…/spark-rapids into update-action-version
Fix merge conflict with branch-24.04 [skip ci]
* Add an config to switch back to old impl for getJsonObject Signed-off-by: Haoyang Li <[email protected]> * Update sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala Co-authored-by: Gera Shegalov <[email protected]> * update doc Signed-off-by: Haoyang Li <[email protected]> * update doc Signed-off-by: Haoyang Li <[email protected]> * verify and address comments Signed-off-by: Haoyang Li <[email protected]> --------- Signed-off-by: Haoyang Li <[email protected]> Co-authored-by: Gera Shegalov <[email protected]>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
…0681) Wait for the pre-merge CI job to SUCCEED Signed-off-by: jenkins <jenkins@localhost> Co-authored-by: jenkins <jenkins@localhost>
* Update latest changelog [skip ci] Update change log with CLI: \n\n scripts/generate-changelog --token=<GIT_TOKEN> --releases=24.02,24.04 Signed-off-by: jenkins <jenkins@localhost> * Update according to review Signed-off-by: Tim Liu <[email protected]> * Update change log to latest Signed-off-by: Tim Liu <[email protected]> * Update change log Update change log Change roadmap to 24.04 for PRs NVIDIA#10158,NVIDIA#10369,NVIDIA#10358,NVIDIA#10360, to move them into 24.04 PR list from 24.02 Signed-off-by: Tim Liu <[email protected]> --------- Signed-off-by: jenkins <jenkins@localhost> Signed-off-by: Tim Liu <[email protected]> Co-authored-by: jenkins <jenkins@localhost> Co-authored-by: Tim Liu <[email protected]>
…-10704 Fix auto merge conflict 10704 [skip ci]
Signed-off-by: Chong Gao <[email protected]> Signed-off-by: Robert (Bobby) Evans <[email protected]> Co-authored-by: Chong Gao <[email protected]>
Signed-off-by: Hongbin Ma (Mahone) <[email protected]>
* Removing some authorizations for departed users Signed-off-by: Mike Wilson <[email protected]> Co-authored-by: Sameer Raheja <[email protected]>
* Upgrade to jucx 1.16.0 Signed-off-by: Alessandro Bellina <[email protected]> --------- Signed-off-by: Alessandro Bellina <[email protected]>
…10718) * Refactor Parquet reader Signed-off-by: Nghia Truong <[email protected]> * Update config * Add back the deprecated config Signed-off-by: Nghia Truong <[email protected]> * Fix config Signed-off-by: Nghia Truong <[email protected]> * Change message for the deprecated config Signed-off-by: Nghia Truong <[email protected]> * Rename variable Signed-off-by: Nghia Truong <[email protected]> * Change the logic of reading conf Signed-off-by: Nghia Truong <[email protected]> * Add example and mark conf as `internal()` Signed-off-by: Nghia Truong <[email protected]> * Reformat code Signed-off-by: Nghia Truong <[email protected]> * Update docs Signed-off-by: Nghia Truong <[email protected]> * Change configs Signed-off-by: Nghia Truong <[email protected]> * Update docs Signed-off-by: Nghia Truong <[email protected]> * Change variables into functions Signed-off-by: Nghia Truong <[email protected]> * Change functions back into `lazy val` Signed-off-by: Nghia Truong <[email protected]> --------- Signed-off-by: Nghia Truong <[email protected]>
…debugging UT in IDEA (NVIDIA#10733) * wip for test Signed-off-by: Haoyang Li <[email protected]> * update comment Signed-off-by: Haoyang Li <[email protected]> --------- Signed-off-by: Haoyang Li <[email protected]>
NVIDIA#10716) Signed-off-by: Robert (Bobby) Evans <[email protected]>
* generate shims * Generate Scala 2.13 poms Signed-off-by: Raza Jafri <[email protected]> * undo bad change to the supportedExprs.csv * Fixed copyrights and removed snapshot * Update copyrights on SparkShimsSuite --------- Signed-off-by: Raza Jafri <[email protected]>
* set fixed seed for some random failed tests Signed-off-by: Haoyang Li <[email protected]> * add import Signed-off-by: Haoyang Li <[email protected]> --------- Signed-off-by: Haoyang Li <[email protected]>
Signed-off-by: Haoyang Li <[email protected]>
* Shimplify 400 Signed-off-by: Raza Jafri <[email protected]> * Added 400 to all.buildvers * addressed review comments, moved the shim declaration to all.buildvers * added 2.13 pom --------- Signed-off-by: Raza Jafri <[email protected]>
…ldcard path (NVIDIA#10734) Signed-off-by: Chong Gao <[email protected]> Co-authored-by: Chong Gao <[email protected]>
* Use parse_url kernel for PATH parsing Signed-off-by: Haoyang Li <[email protected]> * verify Signed-off-by: Haoyang Li <[email protected]> --------- Signed-off-by: Haoyang Li <[email protected]>
Signed-off-by: Robert (Bobby) Evans <[email protected]>
…0755) * minor doc change Signed-off-by: cindyyuanjiang <[email protected]> * better wording Co-authored-by: Jason Lowe <[email protected]> --------- Signed-off-by: cindyyuanjiang <[email protected]> Co-authored-by: Jason Lowe <[email protected]>
Related to NVIDIA/spark-rapids-jni#1843 Signed-off-by: Gera Shegalov <[email protected]>
Closes NVIDIA#10721 ```json { "Event": "com.nvidia.spark.rapids.SparkRapidsBuildInfoEvent", "sparkRapidsBuildInfo": { "url": "https://github.com/NVIDIA/spark-rapids", "branch": "gerashegalov/issue10721", "date": "2024-04-30T04:47:11Z", "cudf_version": "24.06.0-SNAPSHOT", "user": "gshegalov", "revision": "d321e6b5555b1ad16dc515e70d1082e65172cefd", "version": "24.06.0-SNAPSHOT" }, "sparkRapidsJniBuildInfo": { "url": "https://github.com/NVIDIA/spark-rapids-jni.git", "branch": "HEAD", "gpu_architectures": "70;75;80;86;90", "date": "2024-04-28T05:18:44Z", "user": "root", "revision": "f3d5933601860c766a15b34968fa79af21ecaa86", "version": "24.06.0-SNAPSHOT" }, "cudfBuildInfo": { "url": "https://github.com/rapidsai/cudf.git", "branch": "HEAD", "gpu_architectures": "70;75;80;86;90", "date": "2024-04-28T05:18:41Z", "user": "root", "revision": "064dd7b02166cc67e882b708d66621bc3fafd70b", "version": "24.06.0-SNAPSHOT" }, "sparkRapidsPrivateBuildInfo": { "url": "https://gitlab-master.nvidia.com/nvspark/spark-rapids-private.git", "branch": "HEAD", "date": "2024-04-28T04:40:09Z", "user": "", "revision": "b8ec681d75ce850404b9340dc39d290ef735b3ec", "version": "" } } ``` Signed-off-by: Gera Shegalov <[email protected]>
…tabricks] (NVIDIA#10756) * Use cached ThreadPoolExecutor * Revert "Fix Multithreaded Readers working with Unity Catalog on Databricks [databricks] (NVIDIA#8296)" * Signing off Signed-off-by: Raza Jafri <[email protected]> * Removed spark311 version of ReaderUtils.scala --------- Signed-off-by: Raza Jafri <[email protected]>
…VIDIA#10644) * Add loggings to Integration test runs - Set explain all - Working logs - Added logger.FileHandler only for xdist workers * undo setting explain * added documentation and did some refactoring * Signing off Signed-off-by: Raza Jafri <[email protected]> * addressed review comments * added documentation to clarify ambiguity in multiple log4j props file * renamed xdist_it_log4j.properties to be more generic * Added log4j to the flow with TEST_PARALLEL < 2 * Added comments * Updated README.md * added more comments and updated copyrights * Made changes so the worker logs only get generated in local mode * Updated docs * added logs to local-cluster mode as well * Update integration_tests/run_pyspark_from_build.sh Co-authored-by: Gera Shegalov <[email protected]> * Update integration_tests/run_pyspark_from_build.sh Co-authored-by: Gera Shegalov <[email protected]> --------- Signed-off-by: Raza Jafri <[email protected]> Co-authored-by: Gera Shegalov <[email protected]>
Signed-off-by: Partho Sarthi <[email protected]>
Signed-off-by: Robert (Bobby) Evans <[email protected]>
…0767) fix NVIDIA#10751 A cuDF Table requires non empyt columns, so need to check the number of columns when converting a batch to a cuDF table. This PR adds the support for rows-only batches in RebatchingRoundoffIterator. --------- Signed-off-by: Firestarman <[email protected]>
…IA#10743) * initial PR for the framework resuing Vanila Spark's UTs (borrow from Gluten) Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * fix shim complain Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * fix other shim compile issue Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * fix Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * fix for scala 2.13 Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * more to shim Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * clean pom Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * move everything to shim Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * 2.13 Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * move to spark330 Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * fix Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * fix bug Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * fix blossom ci Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * address license comment Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * address suites util package Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * address reason comment Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * address other comments Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * fix bug Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * address 2rd round of comments Signed-off-by: Hongbin Ma (Mahone) <[email protected]> * get rid of UNKNOW_ISSUE Signed-off-by: Hongbin Ma (Mahone) <[email protected]> --------- Signed-off-by: Hongbin Ma (Mahone) <[email protected]>
Signed-off-by: YanxuanLiu <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.