Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add EMR client to spark connector #1790

Merged
merged 41 commits into from
Jul 11, 2023
Merged
Changes from 1 commit
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
70588d6
Create Spark Connector
vamsimanohar Jun 8, 2023
2b8ceb7
Add spark client and engine
rupal-bq Jun 21, 2023
f595ff4
Remove vars
rupal-bq Jun 21, 2023
95a86d5
Merge branch 'main' into create-spark-connector
rupal-bq Jun 26, 2023
51700bd
Merge branch 'main' into create-spark-connector
rupal-bq Jun 26, 2023
d0d5042
Spark connector draft
rupal-bq Jun 26, 2023
5ac01b2
nit
rupal-bq Jun 26, 2023
4fcf4ba
Fix checkstyle errors
rupal-bq Jun 26, 2023
52a044d
nit
rupal-bq Jun 26, 2023
69ca116
Fix license header
rupal-bq Jun 26, 2023
901e762
Add spark storage test
rupal-bq Jun 26, 2023
1b81453
Update comments
rupal-bq Jun 26, 2023
106218e
Fix checkstyle in comments
rupal-bq Jun 26, 2023
6da93f3
Merge branch 'main' into create-spark-connector
rupal-bq Jun 26, 2023
b953aa6
Update tests
rupal-bq Jun 26, 2023
2f899cd
Add emr client
rupal-bq Jun 27, 2023
8ebab1a
Set default values for flint args
rupal-bq Jun 27, 2023
7dcad73
Validate emr auth type
rupal-bq Jun 27, 2023
23f195f
Add default constants for flint
rupal-bq Jun 27, 2023
63e3f6d
Update unit tests
rupal-bq Jun 28, 2023
0df37a8
Address PR comments
rupal-bq Jun 28, 2023
99d561a
tests draft
rupal-bq Jul 4, 2023
776bc36
Refactor class name
rupal-bq Jul 4, 2023
0eb5e53
Merge branch 'main' into create-spark-connector
rupal-bq Jul 4, 2023
680268a
Merge branch 'create-spark-connector' into emr-client
rupal-bq Jul 4, 2023
890c8c7
Rename classes and update tests
rupal-bq Jul 4, 2023
9fc496a
Update scan operator test
rupal-bq Jul 5, 2023
be15dae
Address PR comment
rupal-bq Jul 5, 2023
e74e036
Fix Connection pool shut down issue
rupal-bq Jul 5, 2023
dbf0367
Update emr unit tests
rupal-bq Jul 6, 2023
b8209bd
Merge branch 'create-spark-connector' into emr-client
rupal-bq Jul 6, 2023
2f73d40
Merge branch 'main' into emr-client
rupal-bq Jul 6, 2023
7f8c2ab
Update doc and tests
rupal-bq Jul 6, 2023
cd9a4fe
nit
rupal-bq Jul 6, 2023
9e34ce3
Update EMR clinet impl tests
rupal-bq Jul 6, 2023
c029ae5
Address PR comments
rupal-bq Jul 6, 2023
9d249ee
Make spark & flint jars configurable
rupal-bq Jul 6, 2023
f9d7382
Address comments
rupal-bq Jul 7, 2023
c6e0076
Add spark application id in logs
rupal-bq Jul 7, 2023
cc6d8cb
nit
rupal-bq Jul 7, 2023
5bbda31
Delete result when not required
rupal-bq Jul 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'main' into emr-client
Signed-off-by: Rupal Mahajan <maharup@amazon.com>
rupal-bq committed Jul 6, 2023
commit 2f73d40dfdcb9d7ccba8b2f137f2fdb1d3aa4d3b

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.