Rel16 cherry pick before fast forward #50
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously when reading the text from file, the buffer doesn’t include any EOF so we get some random message in the end. Task: No JIRA Signed-off-by: Zhenye Li <[email protected]> Co-authored-by: Zhenye Li <[email protected]>
…abelfish-for-postgresql#1532) This change allows Information_schema_tsql to return PG base types and sp_columns_100 to return data for bytea columns as well. Signed-off-by: Sai Rohan Basa [email protected]
…#1533) Update datetimeoffset-timezone-* test to use set_config instead ofALTER DATABASE to set the GUC parameters. Signed-off-by: vasavi suthapalli <[email protected]>
Update Github action scripts Create upgrade test folder for last minor version. Bump Babelfish version string in babelfish_version.h Update T-SQL extension minor version by 1 in Version.config Add following new upgrade scripts for TSQL extension: Signed-off by: Ashish Prasad <[email protected]>
) Needed some changes in the error message when the server does not exist and when an invalid option is provided. This change updates both the error messages Issues Resolved BABEL-3730 Signed-off-by: Sai Rohan Basa [email protected]
* Support PARSENAME() T-SQL function The PARSENAME() function in T-SQL is used to parse a string representing a four-part SQL Server object name, such as "database.schema.object.column". Signed-off-by: pratikzode <[email protected]>
This commit adds license information in the build instructions for linked servers. Signed-off-by: Sharu Goel [email protected]
…s are not being tracked by the extensions which utilize processutility hook (babelfish-for-postgresql#1424) Currently statements like create proc/func/trigger/database, drop database and transaction statements are not being tracked by the extensions which utilises the processutility hooks in Babelfish. Above issue happens because bbf_processutility doesn't call subsequent/previous processutility hook for above statements due to which these subsequent extension hooks are not being executed. This commit resolves above issue by adding a custom processutility hook in engine to execute above Babelfish statements so that execution happens at last step of processutility hook chain which makes sure that subsequent extension hooks are executed. Before: processUtility_hooks()-> ... -> bbf_processUtility_hook() -> executes above statements -> execution completed/doesn't return to previous hook execution or doesn't execute subsequent hooks After: processUtility_hooks()-> ... -> bbf_processUtility_hook() -> ... -> processUtility_hooks()-> ... -> Standard_processUtility()-> executes above statements using custom hook -> returns to previous hook execution Task: BABEL-4218 Link for engine PR: babelfish-for-postgresql/postgresql_modified_for_babelfish#127 Signed-off-by: Kritika <[email protected]>
…led (babelfish-for-postgresql#1554) * Add instructions to build Babelfish with kerberos authentication enabled * Update installing intruction template Signedoff-by: Dipesh Dhameliya <[email protected]>
This commit adds license information in the build instructions for linked servers in INSTALLING.md.tmpl Signed-off-by: Sharu Goel <[email protected]>
…resql#1557) * test commit to check validity of modified isBabelfishDatabase func Signed-off-by: Aditya Verma <[email protected]> * enable tests earlier disabled in action.yml Signed-off-by: Aditya Verma <[email protected]> * Signed-off-by: Aditya Verma <[email protected]> * Signed-off-by: Aditya Verma <[email protected]> * BABEL-4185: Fix permission issues for babelfish dump/restore with non-superuser Signed-off-by: Aditya Verma <[email protected]> * add --no-role-passwords flag Signed-off-by: Aditya Verma <[email protected]> * empty commit to re-run tests Signed-off-by: Aditya Verma <[email protected]> * add --no-role-passwords for databasse level dump Signed-off-by: Aditya Verma <[email protected]> * Signed-off-by: Aditya Verma <[email protected]> * re-run tests Signed-off-by: Aditya Verma <[email protected]> * Signed-off-by: Aditya Verma <[email protected]> * empty commit to re-run tests Signed-off-by: Aditya Verma <[email protected]> --------- Signed-off-by: Aditya Verma <[email protected]> Co-authored-by: Aditya Verma <[email protected]>
We are crashing here when the implicit batching of packets is happening at the very first phase i.e. reading column metadata phase because the table itself has huge number of columns. Inherently TDS is in a "miscellaneous" mem context during FETCH phase and the allocated pointer thus seems to crash for invalid context during pfree. To fix this we are appending the new packet's data during the FETCH phase rather than freeing the message data which we do in the PROCESS phase. Signed-off-by: Kushaal Shroff <[email protected]>
…esql#1561) For Some reason BCP was reordering the list of columns and placing the LOB type columns at the end, maybe as part of optimisation. There could also be other scenarios where there is some rearrangement of column order and this was not accounted for and not supported. A mismatch in the attribute list and the tuple-desc list was resulting in a crash when creating the heap tuple using the tuple desc and the mismatched list of values. To fix this I have appropriately stored the Slots's values/nulls in the order of the columns received. Signed-off-by: Kushaal Shroff <[email protected]>
* Support EOMONTH Transact-SQL function The EOMONTH function is a Transact-SQL function in SQL Server that returns the last day of the month of a specified date, with an optional offset. Signed-off-by: pratikzode <[email protected]>
…-postgresql#1565) * Fix the bug: like expression will error out unexpectly in sublink/multiple case when/Join qual Previously, like expression unexpectly error out in certain cases: 1. Like expression inside Join on condition 2. Like expression inside one of the multiple case whens 3. Like expression as one of the sublink condition in sublink query This fix has solved those three issues by adding corresponding Like collation change transform into Like expression node. Task: BABEL-4046 Signed-off-by: Zhibai Song <[email protected]>
…riable (babelfish-for-postgresql#1569) when error handling does not properly close table variables on Error Tasks: BABEL-4225, BABEL-4226, BABEL-4227 Signed-off-by: Kristian Lejao <[email protected]>
…ish-for-postgresql#1503) Previously BBF allowed 'use <databasename>' inside procedure, function and trigger, this type of behaviour shouldn't be allowed. This commit doesn't support the syntax of 'use <databasename>' inside the procedure, function and trigger. Task: Babel-3308 Signed-off-by: vasavi suthapalli <[email protected]>
This commit tests working of pg_stat_statements from T-SQL endpoint. Tested all the DDLs, DMLs, TCLs, aggregate functions, constraints etc. TASK: BABEL-4218 Signed-off-by: Kritika <[email protected]>
Description In this commit, we add support for sp_testlinkedserver stored procedure which allows testing of linked servers against any possible exceptions in the connection process. The sp_testlinkedserver stored procedure have been implemented by recalling the functions used by TSQL-OPENQUERY() to establish linked server connections. Task: BABEL-4070 Signed-off-by: Shriramu A R [email protected]
…elfish-for-postgresql#1583) Description Currently users of allowed extensions will have to exit T-SQL and connect on PG port for installing the extension, altering or dropping the extension. This procedure will enable the users to execute certain PG statements from T-SQL endpoint. For now we are supporting it in a restricted manner by adding support for executing create extension, alter extension version and drop extension. Issues Resolved Task: BABEL-1361 Signed-off-by: Kritika <[email protected]>
…h-for-postgresql#1584) During cleanup phase, drop table variable would drain the relation refcnt hence when we close the cursors that references the table variables would error out. Fix is to change the order of operation in the cleanup phase, drop cursor before drop table variables. Task: BABEL-4268, BABEL-4269 Signed-off-by: Kristian Lejao <[email protected]>
…ostgresql#1588) This commit fixes the build issue that occured after the merge of the PR babelfish-for-postgresql#1551 . The issue is fixed by adding ENABLE_TDS_LIB flag check in the sp_testlinkedserver_internal() function. Task: BABEL-4070 Signed-off-by: Shriramu A R [email protected]
…postgresql#1541) * fix like with bracket work improper for some corner cases previously, like expression for bracket produce wrong result for : 1. [%, when there's no right bracket, % should not work as a wild character 2. the order range of character should not follow the ascii order but should follow the database natural order, for example : '='(ascii:62) should not between 0(ascii 48) to a(ascii 97) Task: BABEL-4193 Signed-off-by: Zhibai Song <[email protected]>
Starting 3rd July 2023, all the pre-cached .NET 3.1 versions will be removed from all operating systems. To fix this, we have updated the dotnet sdk version and TFM to use .NET 7.0. Signed-off-by: Sharu Goel [email protected]
Description In this commit, we add support for the connect timeout server option using the sp_serveroption stored procedure. It specifies the time to wait in seconds before returning from a failed attempt to login to a linked server. The connect timeout server option is implemented using the free_tds library API dbsetlogintime(). Task: BABEL-4274 Signed-off-by: Shriramu A R [email protected]
…h-for-postgresql#1594) This commit reverts a unwanted change introduced in minor-version-upgrade.yml github action by commit aaa7127. Task: BABEL-OSS Signed-off-by: Harsh Lunagariya [email protected]
Description At present, babelfish encompasses various testing frameworks such as JDBC, Python, .NET, and fuzz tests. Each of these frameworks serves specific purposes and relies on client drivers. However, we lack a framework dedicated to unit testing. This is the motivation behind this to create a unit testing framework for babelfish. In this commit, we add unit testing framework which enable us to test any piece of our codebase. The babelfishpg_unit directory is added as a new extension within the contrib folder present in babelfish_extensions. It contains the necessary files for the unit testing framework. Here's a brief explanation of each file: Makefile: This file specifies the build instructions for the babelfishpg_unit extension. babelfishpg_unit.control: This files contain metadata and control information about the extension. babelfishpg_unit.h: This header file contains any necessary macros, definitions, or utility functions required for the unit tests. babelfishpg_unit—1.0.0.sql: This SQL script defines the necessary functions and setup for running the tests. It acts as the entry point for invoking the unit tests. Once it is invoked, it will trigger the execution of all the tests or specific tests. babelfishpg_unit.c: This file serves as the main entry point for the unit tests. It includes the necessary headers and implements the logic to execute all the defined tests. test_money.c, ...: These are the individual test files which contains the test cases and assertions specific to a particular area of functionality or component being tested. log_files: It is a folder which contains log files created after each test run Issues Resolved I have developed the framework in such a way that we can run multiple tests by passing multiple parameters. With this framework, all test functions associated with multiple specified parameters can be successfully executed. Signed-off-by: Yenni Shashank [email protected]
…tgresql#1591) Previously the sqlcmd hangs for select query which has XML columns. This commit fixes by sending NBCRowToken when nullMapSize is equal to simpleRowSize. And for older clients of TDS 7.2 where NbcRowToken doesn't exist should send 0xffffffffffffffff specially for XML NULL values. Task: BABEL-4196 Signed-off-by: vasavi suthapalli <[email protected]>
…for-postgresql#1599) * Add instruction to build and configure Babelfish with SSL ODBC Driver 18 for SQL Server by default will encrypt the connection. So out of the box configuration of Babelfish cannot be connected to from SQLCMD using this ODBC Driver. This commit adds instructions on how to build and configure the babelfish server with SSL enabled. Signed-off-by: Sharu Goel <[email protected]>
…#1604) * Fix backward scan flag for parallel queries Currently there is a bug in Babelfish that allows the backward scan flag to be set for parallel queries. However, backward scans cannot be parallelized. This bug results in assertion failures in the Postgres engine and leads Babelfish queries to crash. In this commit, we fix the issue by disallowing backward scan flag to be set when a query is in parallel mode. This issue occurs during the implementation of full text search in Babelfish. Fixing the issue unblocks full text search implementation. Task: BABEL-4261, BABEL-4281 Signed-off-by: Robin Li <[email protected]>
…mat_numeric functions (babelfish-for-postgresql#1987) Currently, we are using the following arguments : sys.format = (anyelement, NVARCHAR, VARCHAR) sys.format_datetime = (anyelement, NVARCHAR, VARCHAR, VARCHAR) sys.format_numeric = (anyelement, NVARCHAR, VARCHAR, VARCHAR, int) This change modifies the datatypes of arguments to : sys.format = (anyelement, sys.NVARCHAR, sys.VARCHAR) sys.format_datetime = (anyelement, sys.NVARCHAR, sys.VARCHAR, sys.VARCHAR) sys.format_numeric = (anyelement, sys.NVARCHAR, sys.VARCHAR, sys.VARCHAR, int) Task: BABEL-4394 Signed-off-by: Sai Rohan Basa <[email protected]>
…sh-for-postgresql#1957) Binary data type simply stores hex codes but when transformed to and from varchar, use of correct encoding becomes important when we cast to and from string data types. For example the symbol '™' is stored as 0xE284A2 in UTF-8 encoding while 0x99 in WIN1252 encoding. From users perspective they must see the hex value which is congruent to their server encoding. To fix this we do necessary encoding to source data in varchar <--> varbinary internal functions. Length checks should be done when data (hex string) is in server encoding. We must also handle the case for string literal to binary types. So we explicitly call the varcharvarbinary conversion when a string literal is being casted to binary data type. Also added CAST functions for sys.BBF_VARBINARY to sys.BBF_BINARY and vice versa. These cast are being used in geography and geometry data types cast functions. Made necessary changes to geography and geometry casts as well i.e. CAST (CAST ($1 AS sys.VARCHAR) AS sys.bbf_varbinary) --> CAST ($1 AS sys.bbf_varbinary) Engine PR: babelfish-for-postgresql/postgresql_modified_for_babelfish#248 Extension PR: babelfish-for-postgresql#1957 Task: BABEL-1940 Signed-off-by: Tanzeel Khan [email protected] Co-authored-by: Rohit Bhagat [email protected]
…#2010) fix upgrade script for new casts Task: BABEL-1940 Signed-off-by: Tanzeel Khan <[email protected]>
…ostgresql#2006) Signed-off-by: Rohit Bhagat <[email protected]>
…ostgresql#2015) Previously the parallel mode check happens during GUC assignment for GUCs enable_pg_hint and identity_insert. This commit follows PG custom and moves the logic into the GUCs' check_hook function. Signed-off-by: Xiaohui Fanhe <[email protected]>
…-postgresql#1986) Currently babelfish doesn't support fulltext index creation and drop for fulltext search. This commit implements the logic of create and drop fulltext index and also handles the unsupported features of the CREATE/DROP FULLTEXT INDEX statements. Task: BABEL-4383 Signed-off-by: Roshan Kanwar <[email protected]>
Previously insert exec implement didn't consider the type cast during execution, this fix has add a implicit type cast between insert execution and exec execution if the type is mismatched. Engine pr : babelfish-for-postgresql/postgresql_modified_for_babelfish#256 Task: BABEL-2999, BABEL-4426 Signed-off-by: Zhibai Song <[email protected]>
…for-postgresql#2019) Currently, in upgrade scripts, server_collation_name is set to restored_server_collation_name in babelfishpg_common upgrade scripts only and then GUC restored_server_collation_name is being reset. Now this server_collation_name is being set only for that connection, when a new connection is created server_collation_name will contain the default value. This will cause issue when different connections are used to update babelfishpg_common and babelfishpg_tsql, as the server_collation_name contains default value in connection in which babelfishpg_tsql is getting updated, but expected value of server_collation_name is the value of restored_server_collation_name. Due to this, a failure is occurring in Major Version upgrade from 14.x to 15.x, when different connections are used to upgrade babelfishpg_common and babelfishpg_tsql extension and server collation name is set to non-default value in base version. This commit will resolve this issue by setting the server_collation_name to restored_server_collation_name in babelfishpg_tsql upgrade scripts as well. Task: BABEL-4519 Sign-off-by: Rohit Bhagat <[email protected]>
…s.format functions (babelfish-for-postgresql#2023) We are deprecating the older versions of sys.format_datetime, sys.format_numeric and sys.format functions in the version 2.7.0, Hence while doing major version upgrade, sys.format_datetime(anyelement, nvarchar, character varying, character varying) is not found. And similarly, sys.format_numeric(anyelement, NVARCHAR, VARCHAR, VARCHAR, int) and sys.format(anyelement, NVARCHAR, VARCHAR) are also deprecated in older version which may cause an error. This changes catches the above error and deprecates the format functions only when they exist. Task: BABEL-4394 Signed-off-by: Sai Rohan Basa <[email protected]>
1/ Update Github action scripts 2/ Create 15_5 upgrade test folder. 3/ Bump Babelfish version string in babelfish_version.h 4/ Add upgrade script babelfishpg_tsql--3.4.0--3.5.0.sql Signed-off-by: Sai Rohan Basa <[email protected]>
…sh-for-postgresql#1998) This change reimplements the datediff, datediff_big, and dateadd functions in C to improve performance by 65% compared to the original implementation. Task: BABEL-4496 Signed-off-by: Jake Owen <[email protected]>
Test BABEL-4281 gives the memory usage with STATISTICS PROFILE. During testing, this value can change, causing test failures. Switching to SHOWPLAN_ALL will give a plan without outputing memory usage. Signed-off-by: Walt Boettge <[email protected]>
…t exists". (babelfish-for-postgresql#2029) Some tests listed in issue BABEL-4540 failing with error " 'tds_fdw' extension does not exists". This is because of tds_fdw extension is not installed locally. However while running jdbc test check in GitHub we install 'tds_fdw' extension in GitHub check. These tests are running now as expected in parallel query mode. Removed these tests from ignore file. One of the test four-part-names-vu-verify still failing due to time-out. Included this test to appropriate group. Task: BABEL-4540 Signed-off-by: Sandeep Kumawat <[email protected]>
…or-postgresql#1905) sp_describe_undeclared_parameters is slow. The issue is reported here: babelfish-for-postgresql#1317. Initially, we used to create a query in C for every parameter passed as an argument to sp_describe_undeclared_parameters and then execute it. This was taking huge amount of time as it used JOINS among multiple views like sys.objects, sys.columns, sys.types T. In this commit, we have replaced the use of such views as much as possible with pg catalogs. This aids to the improvement in performance. Task: BABEL-3705 Signed-off-by: Shameem Ahmed <[email protected]>
Sairakan
force-pushed
the
rel16-cherry-pick-before-fast-forward
branch
from
2bf402d to
8810a19
Compare
added 5 commits
November 21, 2023 05:33
Signed-off-by: Jason Teng <[email protected]>
…sted both Sender and Receiver functions with JDBC, ODBC and DOTNET Drivers (babelfish-for-postgresql#1921)" This reverts commit 3a0259b.
…r Support (babelfish-for-postgresql#1819)" This reverts commit 00c87ad.
Signed-off-by: Jason Teng <[email protected]>
Sairakan
force-pushed
the
rel16-cherry-pick-before-fast-forward
branch
from
a582ddf to
f46174f
Compare
added 4 commits
November 21, 2023 05:42
Signed-off-by: Jason Teng <[email protected]>
Signed-off-by: Jason Teng <[email protected]>
Signed-off-by: Jason Teng <[email protected]>
Signed-off-by: Jason Teng <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.