Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the nvarchar-varbinary casting #3263

Open
wants to merge 5 commits into
base: BABEL_5_X_DEV
Choose a base branch
from

Conversation

pranavJ23
Copy link
Contributor

Description

Cherry-Pick: #3072
PROBLEM: while casting nvarchar to varbinary we were considering the encoding as input encoding in babelfish where as in tsql we use UTF16 encoding fir nvarchar irrespective of input encoding.

RCA: we were considering varchar and nvarchar as same, whereas we should use input encoding for varchar and UTF16 encoding for nvarchar.

FIX: So we need to identify that if the input is nvarchar then we will do the UTF16 encoding.

For a casting like nvarchar->varbinary->nvarchar, now since for the casting we are encoding the input string into UTF16 encoding via function nvarcharvarbinary, so while converting varbinary-> nvarchar we will use the function varbinarynvarchar where we will convert UTF16 encoding to UTF8 with null padding.
So we created a function nvarcharvarbinary and varbinarynvarchar to handle nvarchar<-> varbinary to and fro casting.
And for this casting we have specifically applied a condition that we will not convert the datatype to basetype before choosing the casting function

Issues Resolved

[List any issues this PR will resolve]

Test Scenarios Covered

  • Use case based -

  • Boundary conditions -

  • Arbitrary inputs -

  • Negative test cases -

  • Minor version upgrade tests -

  • Major version upgrade tests -

  • Performance tests -

  • Tooling impact -

  • Client tests -

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.

pranavJ23 and others added 3 commits December 16, 2024 13:18
PROBLEM: while casting nvarchar to varbinary we were considering the UTF8 encoding as input encoding in Babelfish
where as in TSQL we use UTF16 encoding fir nvarchar irrespective of input encoding.

RCA: we were considering varchar and nvarchar as same, whereas we should use input encoding for varchar and UTF16
encoding for nvarchar.

FIX: So we need to identify that if the input is nvarchar then we will do the UTF16 encoding.

For a casting like nvarchar->varbinary->nvarchar, now since for the casting we are encoding the input string into UTF16
encoding via function nvarcharvarbinary, so while converting varbinary-> nvarchar we will use the function
varbinarynvarchar where we will convert UTF16 encoding to UTF8 with null padding.

So we created a function nvarcharvarbinary and varbinarynvarchar to handle nvarchar<-> varbinary to and fro casting.
And for this casting we have specifically applied a condition that we will not convert the datatype to basetype before choosing the casting function

Task: BABEL-4891
Signed-off-by: Pranav Jain <[email protected]>
@coveralls
Copy link
Collaborator

coveralls commented Dec 16, 2024

Pull Request Test Coverage Report for Build 12387540169

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 146 of 181 (80.66%) changed or added relevant lines in 3 files are covered.
  • 448 unchanged lines in 3 files lost coverage.
  • Overall coverage increased (+0.02%) to 74.838%

Changes Missing Coverage Covered Lines Changed/Added Lines %
contrib/babelfishpg_common/src/varbinary.c 83 91 91.21%
contrib/babelfishpg_common/src/varchar.c 40 67 59.7%
Files with Coverage Reduction New Missed Lines %
contrib/babelfishpg_tsql/src/pl_handler.c 53 90.05%
contrib/babelfishpg_tsql/src/pltsql_coerce.c 112 82.47%
contrib/babelfishpg_tsql/src/hooks.c 283 83.91%
Totals Coverage Status
Change from base Build 12352168391: 0.02%
Covered Lines: 46618
Relevant Lines: 62292

💛 - Coveralls

Signed-off-by: pranav jain <[email protected]>
Signed-off-by: pranav jain <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants