Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve] use generate_sink_sql=true for db2 upsert #233

Conversation

shashwatsai
Copy link
Contributor

Purpose of this pull request

Currently we enforce sending sink insert query, forcing user to not support upsert scenarios.
Please refer:
ISSUE: apache/seatunnel#4434
PR (Upsert DB2 Support): apache/seatunnel#7879

Check list

@arshadmohammad
Copy link
Collaborator

Please remove [ISSUE-4434] from the PR title. Instead, you can mention the issue reference in the PR description. The PR title can be named as [Improve][DB2]..."


connectorConfig =
connectorConfig.withValue(QUERY_KEY, ConfigValueFactory.fromAnyRef(sql));
if (selectTableFields.isAll()) {
Copy link
Collaborator

@arshadmohammad arshadmohammad Oct 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not see any reference of generate_sink_sql change or use. can you explain the code change, how it is related to db2 upsert

connectorConfig =
connectorConfig.withValue(
QUERY_KEY, ConfigValueFactory.fromAnyRef(sql));
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you attach snapshot or test cases for this change

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done @arshadmohammad!

Thanks for your time, have incorporated the review comments.

@shashwatsai shashwatsai force-pushed the feature/ISSUE-4434-db2-upsert-use-generate-sink-sql branch from f8c2baa to ca1f460 Compare October 21, 2024 11:48
@shashwatsai shashwatsai changed the title [Improve][ISSUE-4434] use generate_sink_sql=true for db2 upsert [Improve] use generate_sink_sql=true for db2 upsert Oct 21, 2024
@shashwatsai shashwatsai force-pushed the feature/ISSUE-4434-db2-upsert-use-generate-sink-sql branch from ca1f460 to 8f43cac Compare October 21, 2024 12:23
@shashwatsai
Copy link
Contributor Author

shashwatsai commented Oct 21, 2024

Manual Deployment:

Source: Hive JDBC (partitioned table)
image

Sink: DB2
image

Trigger for row updates:
image

generated job config:
image

Validations:
Difference for keys: 745, 766, 779
image

Copy link
Collaborator

@arshadmohammad arshadmohammad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1

@arshadmohammad arshadmohammad merged commit 33513dd into apache:main Oct 21, 2024
15 checks passed
@arshadmohammad
Copy link
Collaborator

Thanks @shashwatsai for working on db2 issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants