Fix Error Handling and Cleanup during Insert Bulk Process #2887
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pull Request Test Coverage Report for Build 10629449983Warning: This coverage report may be inaccurate.This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.
Details
💛 - Coveralls |
kuntalghosh
requested changes
Aug 29, 2024
| { | ||
| int ret = 0; | ||
|
|
||
| HOLD_CANCEL_INTERRUPTS(); |
There was a problem hiding this comment.
Let's move this to its parent function add a comment
hold interrupts during cleanup
There was a problem hiding this comment.
What difference does it make though? For both cases of cleanup we need to hold interrupts so it will merely avoid redundancy in code lines
There was a problem hiding this comment.
The cleanup function doesn't care whether you're pausing the interrupts or not. It's rather from where you're calling the function, it matters. In this case, since you're calling in catch block, you want to hold the interrupts.
Yeah, the comment part looks redundant. Please ignore.
surendravishnoi
previously approved these changes
Aug 29, 2024
kuntalghosh
requested changes
Aug 30, 2024
kuntalghosh
approved these changes
Aug 30, 2024
surendravishnoi
approved these changes
Aug 30, 2024
KushaalShroff
merged commit 721d4e0
into
babelfish-for-postgresql:BABEL_4_X_DEV
46 checks passed
KushaalShroff
added a commit
to amazon-aurora/babelfish_extensions
that referenced
this pull request
Aug 30, 2024
…for-postgresql#2887) This commit fixes an issue with the error handling and cleanup phase of the Insert Bulk Process. 1. For Error Handling there was a scenario where bulk_load_callback(0, 0, NULL, NULL) call for cleanup would flush the remaining rows during EndBulkCopy and could result in an error. To fix this, we move the transaction rollback logic from TSQL extension to TDS. We also improved it by using Savepoints in case of active transaction which is aligned with TSQL behaviour. 2. In this case, if we have a reset-connection after the Bulk Load TDS packet we werent cleaning up the Bulk Load state. To do so we reset the offset. 3. During Reset Connection TDS is not resetting any TSQL transaction semantic. To resolve this we introduce a wrapper of AbortOutOfAnyTransaction to reset NestedTranCount. Issues Resolved BABEL-5200, BABEL-5199, BABEL-5220 Authored-by: Kushaal Shroff [email protected] Signed-off-by: Kushaal Shroff [email protected]
tanscorpio7
reviewed
Aug 30, 2024
| if (values) | ||
| pfree(values); | ||
| if (nulls) | ||
| pfree(nulls); | ||
| } | ||
| PG_CATCH(); | ||
| { | ||
| if (TDS_DEBUG_ENABLED(TDS_DEBUG2)) |
There was a problem hiding this comment.
We should hold interrupts before doing any logging in catch blocks.
KushaalShroff
added a commit
to amazon-aurora/babelfish_extensions
that referenced
this pull request
Aug 30, 2024
…for-postgresql#2887) This commit fixes an issue with the error handling and cleanup phase of the Insert Bulk Process. 1. For Error Handling there was a scenario where bulk_load_callback(0, 0, NULL, NULL) call for cleanup would flush the remaining rows during EndBulkCopy and could result in an error. To fix this, we move the transaction rollback logic from TSQL extension to TDS. We also improved it by using Savepoints in case of active transaction which is aligned with TSQL behaviour. 2. In this case, if we have a reset-connection after the Bulk Load TDS packet we werent cleaning up the Bulk Load state. To do so we reset the offset. 3. During Reset Connection TDS is not resetting any TSQL transaction semantic. To resolve this we introduce a wrapper of AbortOutOfAnyTransaction to reset NestedTranCount. Issues Resolved BABEL-5200, BABEL-5199, BABEL-5220 Authored-by: Kushaal Shroff [email protected] Signed-off-by: Kushaal Shroff [email protected]
sharathbp
pushed a commit
to amazon-aurora/babelfish_extensions
that referenced
this pull request
Sep 3, 2024
…for-postgresql#2887) This commit fixes an issue with the error handling and cleanup phase of the Insert Bulk Process. 1. For Error Handling there was a scenario where bulk_load_callback(0, 0, NULL, NULL) call for cleanup would flush the remaining rows during EndBulkCopy and could result in an error. To fix this, we move the transaction rollback logic from TSQL extension to TDS. We also improved it by using Savepoints in case of active transaction which is aligned with TSQL behaviour. 2. In this case, if we have a reset-connection after the Bulk Load TDS packet we werent cleaning up the Bulk Load state. To do so we reset the offset. 3. During Reset Connection TDS is not resetting any TSQL transaction semantic. To resolve this we introduce a wrapper of AbortOutOfAnyTransaction to reset NestedTranCount. Issues Resolved BABEL-5200, BABEL-5199, BABEL-5220 Authored-by: Kushaal Shroff [email protected] Signed-off-by: Kushaal Shroff [email protected]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This commit fixes an issue with the error handling and cleanup phase of the Insert Bulk Process.
Issues Resolved
BABEL-5200, BABEL-5199, BABEL-5220
Authored-by: Kushaal Shroff [email protected]
Signed-off-by: Kushaal Shroff [email protected]
Test Scenarios Covered
a. Commit without error
b. Rollback without error
a. transaction testing during error scenarios
b. @@trancount test - error should not terminate transaction
c. Test CheckConstraint BCP Option Enabled
d. Test Reusing the same connection for BCP even after error scenarios
The above tests test the seq and index.
For Reset-connection, Although the tests in 6. do reset but for validation through logs I added a temp log to debug TdsResetConnection and with this log its evident that pid 21384 is being reset and reused for Bulk Load
Check List
By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.