Add Shim JSON Headers for Databricks 13.3

[razajafri]

This pull request introduces support for Databricks 13.3. This PR is part of a series of PRs that will address the complete support for Databricks 13.3. The primary focus of this PR is the addition of necessary shims for 341db.

Changes Made:

Shims for 341db: This PR includes the implementation of shims to support this version of Databricks, ensuring that our codebase remains compatible.

Testing:

Tests will be addressed in subsequent PRs

contributes towards #9175

[razajafri]

build

[jlowe]

The PR headline should be updated to be a bit more descriptive. As it is, it implies this PR adds support for Databricks 13.3 which isn't quite accurate.

[gerashegalov]

If this PR merely shimplify-generated it would help if we can see the command line used in the description.

Should spark341db.RapidsShuffleManager be part of the PR?

[razajafri]

If this PR merely shimplify-generated it would help if we can see the command line used in the description.

Should spark341db.RapidsShuffleManager be part of the PR?

Thank you for the comment. This isn't just shimplify-generated code but a commit from a branch that is already building. I have just broken the commits down so they are easier to review.

You can see the branch here.

The RapidsShuffleManager will be a part of the boilerplate that will follow in subsequent PRs.

[razajafri]

build

[gerashegalov]

If this PR merely shimplify-generated it would help if we can see the command line used in the description.
Should spark341db.RapidsShuffleManager be part of the PR?

Thank you for the comment. This isn't just shimplify-generated code but a commit from a branch that is already building. I have just broken the commits down so they are easier to review.

You can see the branch here.

The RapidsShuffleManager will be a part of the boilerplate that will follow in subsequent PRs.

In the future It is easier for the reviews if we follow the standard procedure that also takes care of the RapidsShuffleManager

spark-rapids/docs/dev/shimplify.md

Lines 185 to 187 in 97089e6

	mvn generate-sources -Dshimplify=true \
	-Dshimplify.move=true -Dshimplify.overwrite=true \
	-Dshimplify.add.shim=324 -Dshimplify.add.base=323

[razajafri]

If this PR merely shimplify-generated it would help if we can see the command line used in the description.
Should spark341db.RapidsShuffleManager be part of the PR?

Thank you for the comment. This isn't just shimplify-generated code but a commit from a branch that is already building. I have just broken the commits down so they are easier to review.
You can see the branch here.
The RapidsShuffleManager will be a part of the boilerplate that will follow in subsequent PRs.

In the future It is easier for the reviews if we follow the standard procedure that also takes care of the RapidsShuffleManager

spark-rapids/docs/dev/shimplify.md

Lines 185 to 187 in 97089e6

	mvn generate-sources -Dshimplify=true \
	-Dshimplify.move=true -Dshimplify.overwrite=true \
	-Dshimplify.add.shim=324 -Dshimplify.add.base=323

I do see your point. In this case, however, breaking down the commits into smaller chunked PRs was an afterthought. I agree it would've helped to review the changes if it was just the shimplify-generated code or something repeatable by the reviewers

[NvTimLiu]

@razajafri

You can add 13.3 in the below line to run pre-merge CI with Databricks 13.3 against your PR, values '10.4', '11.3', '12.2', '13.3'
https://github.com/NVIDIA/spark-rapids/blob/branch-23.12/jenkins/Jenkinsfile-blossom.premerge-databricks#L91

[razajafri]

@razajafri

You can add 13.3 in the below line to run pre-merge CI with Databricks 13.3 against your PR, values '10.4', '11.3', '12.2', '13.3' https://github.com/NVIDIA/spark-rapids/blob/branch-23.12/jenkins/Jenkinsfile-blossom.premerge-databricks#L91

Thanks, I will add it after the pom changes are in as we aren't building against 13.3 yet.