[AD-608] Add support for nullable data (#38)

* [AD-608] refactor change * [AD-608] implement boost::optional for ResultSetGetString * both ResultSetGetString by index and by column name are modified * [AD-608] update java_test to accommodate boost::optional * if value should be null, checking for optional value being non-existent is enabled, and if value should not be null, checking for optional value being existent is enabled * (* operator) is use for boost::optional<std::string>. For details, see https://www.boost.org/doc/libs/1_78_0/libs/optional/doc/html/boost_optional/quick_start.html * [AD-608] update result_set GetString * update to accommodate boost::optional<std::string> * [AD-608] update jni_test to accommodate boost::optional * if value should be null, checking for optional value being non-existent is enabled, and if value should not be null, checking for optional value being existent is enabled * (* operator) is use for boost::optional<std::string>. For details, see https://www.boost.org/doc/libs/1_78_0/libs/optional/doc/html/boost_optional/quick_start.html * [AD-608] add include boost/optional for jni_test.cpp * [AD-608] implement boost::optional<std::string> in column_meta * catalogName, schemaName, tableName, columnName, remarks, columnDef are now boost::optional<std::string> * [AD-608] make ColumnMeta constructor accept std::string as parameters again * [AD-608] define PutOptString function in ApplicationDataBuffer namespace * [AD-608] add include headers for boost/optional * <boost/optional/optional_io.hpp> header is included because boost::optional<T> needs to be returned. * [AD-608] change std::string to boost::optional in table_meta.h * [AD-608] call PutOptString in metadata_query * replace PutString with PutOptString inside column_metatdata_query and table_metadata_query * [AD-608] update BOOST_CHECK * since nullable Strings are now supported, set BOOST_CHECK_EQUAL to true instead of false. * remove todo comments * [AD-608] bugfix include * [AD-608] implement nullable int in java.cpp * [AD-608] update java_test to support nullable int * [AD-608] update resultSet to support nullable int * [AD-608] refactor/format changes * [AD-608] add checks for nullable strings * [AD-608] remove empty string initialization * [AD-608] implement optional int in application_buffer * [AD-608] make type_traits accept optional parameters * [AD-608] change NullabilityToIsNullable to accept int32_t * [AD-608] make column_meta support nullable int * make column_metadata_query call putOptInt*. * change columnType from uint8_t to int16_t, because data_type could be a negative value. * replace currentColumn.GetDataType() with columnType in GetColumn * [AD-608] add check for optional values for log_msg * [AD-608] remove wasNull from JNI calls * remove wasNull parameter from JNI calls * remove wasNull parameter from resultSet calls * [AD-608] add if check for log_msg * add (*) operator for GetDataType, since dataType is a boost::optional variable * [AD-608] remove wasNull * wasNull is no longer required since we have nullable int/short/strings etc * remove references of wasNull in function calls * remove wasNull in DocumentDbDatabaseSchemaMetadataGetSchemaName * remove BOOST_CHECK / BOOST_REQUIRE for wasNull * add BOOST_CHECK for intValue in jni_test * [AD-608] bugfix make Nullability::ToSql always return * [AD-608] resolve build errors caused by boost::optional * add if statement checks that combine the initialization of the optional object and the test. * [AD-608] change decimalDigits to return a short * SQLColumns returns short for decimal digits on documentation * previously, unnecessary conversion from integer to short was done, so I've changed the SqlTypeDecimalDigits and the BinaryTypeDecimalDigits just to return shorts * [AD-608] call PutOpt* functions in type_info_query * [AD-608] re-add wasNull to DocumentDbDatabaseSchemaMetadataGetSchemaName * reason: it was a mistake to remove the wasNull in DocumentDbDatabaseSchemaMetadataGetSchemaName. * [AD-608] remove double const * [AD-608] update jni_test * fixed tests to check for correct boolean * changed test for char_octet_length since nullable ints are now supported * [AD-608] remove comments * [AD-608] bugfix BinaryTypeToSqlTypeName * change BinaryTypeToSqlTypeName to modify boost::optional< std::string >& value from returning a const boost::optional< std::string >, which caused issues. ** when returning boost::optional< std::string > to other spaces like column_metadata_query, the reference to the object is deleted because it is a boost::optional object. * I am considering replacing `void BinaryTypeToSqlTypeName` with ` const boost::optional< std::string >& BinaryTypeToSqlTypeName` altogether, but that's to be done for tomorrow * [AD-608] revert BinaryTypeToSqlTypeName back to returning an object * make BinaryTypeToSqlTypeName return boost::optional< std::string > instead of by reference (suggested by Bruce) * removed function "void BinaryTypeToSqlTypeName( boost::optional< int16_t > binaryType, boost::optional< std::string >& value)" because it is not needed Co-Authored-By: Bruce Irschick <[email protected]> * [AD-608] debug macOS build error * cast char[] to std::string * [AD-608] remove todo for wasNull(buffer_length_len) * [AD-608] add parameter for boost_check_equal * [AD-608] format java.h * [AD-608] refactor the format of result_set.h * [AD-608] remove extra line in application_data_buffer_.cpp * [AD-608] format result_set.cpp * [AD-608] reformat * [AD-608] reformat * remove comment * [AD-608] create NOTICE * [AD-608] update NOTICE to include line at end of file * [AD-608] update NOTICE from feedback Co-Authored-By: Alexey Temnikov <[email protected]> * [AD-608] test against local server * added new json file named meta_queries_test_002 for meta_queries_test. * add test TestGetDataWithColumnsReturnsOneFromLocalServer, which tests with meta_queries_test_002 * [AD-608] remove 'quiet' from import test data script * reason: it shows more details when imports are successful/fail. If we have 'quiet', we might not be able to know whether something has failed or not during the import of the test data. Co-Authored-By: Bruce Irschick <[email protected]> * [AD-608] remove --quiet in macOS import_test_data script * [AD-608] add back --quiet in macOS import_test_data script for clear database command * [AD-608] address code review comments * move boost::optional object initialization outside of if statement for readability. * [AD-608] remove unnecessary line in NOTICE * [AD-608] address code review comments and call get_value_or * call get_value_or to ensure log messages are always returned
aws · Mar 11, 2022 · 3205ee6 · 3205ee6
1 parent f660b4b
commit 3205ee6
Show file tree

Hide file tree

Showing 24 changed files with 738 additions and 476 deletions.
diff --git a/NOTICE b/NOTICE
@@ -0,0 +1,37 @@
+Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+The Amazon DocumentDB ODBC Driver uses third-party libraries and 
+lists the third-party software licensing below.
+
+In the event that we accidentally failed to list a required notice,
+please bring it to our attention through any of the ways detailed here:
+
+    https://github.com/aws/amazon-documentdb-odbc-driver/issues
+
+The attached notices are provided for information only.
+
+For any licenses that require disclosure of source, sources are available at
+https://github.com/aws/amazon-documentdb-odbc-driver.
+
+1) License Notice for the Boost library
+---------------------------------------
+
+Copyright (c) 2003 - 2021 Boost Software
+
+Distributed under the Boost Software License, Version 1.0.
+See http://www.boost.org/LICENSE_1_0.txt
+
+2) License Notice for Apache Ignite
+-----------------------------------------
+
+Copyright (c) 2015 - 2021 Apache Ignite
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not
+use this software except in compliance with the License. You may obtain a
+copy of the License at http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+License for the specific language governing permissions and limitations
+under the License.
diff --git a/src/odbc-test/input/meta_queries_test_002.json b/src/odbc-test/input/meta_queries_test_002.json
@@ -0,0 +1,32 @@
+{
+   "_id":{
+      "$oid":"8888f88886afa51ac62bed23"
+   },
+   "fieldDecimal128":{
+      "$numberDecimal":"Infinity"
+   },
+   "fieldDouble":3.1415926E23,
+   "fieldString":"こんにちは",
+   "fieldObjectId":{
+      "$oid":"888888caabcdef8888888888"
+   },
+   "fieldBoolean":false,
+   "fieldDate":{
+      "$date":"2022-03-08T00:00:00Z"
+   },
+   "fieldInt":21345621,
+   "fieldLong":9281883193847263322,
+   "fieldMaxKey":{
+      "$maxKey":1
+   },
+   "fieldMinKey":{
+      "$minKey":1
+   },
+   "fieldNull":null,
+   "fieldBinary":{
+      "$binary":{
+         "base64":"AAEC",
+         "subType":"00"
+      }
+   }
+}
diff --git a/src/odbc-test/scripts/import_test_data.ps1 b/src/odbc-test/scripts/import_test_data.ps1
@@ -20,11 +20,11 @@ Foreach-Object {
 
     # Log what input we're about to import
     Write-Output `
-        "mongoimport --quiet -u=$($env:DOC_DB_USER_NAME) -p=... --authenticationDatabase=admin `
+        "mongoimport -u=$($env:DOC_DB_USER_NAME) -p=... --authenticationDatabase=admin `
             -d=$($DATABASE_NAME) -c=$($COLLECTION_NAME) `
             --file=""$($TEST_INPUT_FOLDER)\$($TEST_FILE_NAME)"""
     # Import the test input
-    mongoimport --quiet -u="$($env:DOC_DB_USER_NAME)" -p="$($env:DOC_DB_PASSWORD)" --authenticationDatabase=admin `
+    mongoimport -u="$($env:DOC_DB_USER_NAME)" -p="$($env:DOC_DB_PASSWORD)" --authenticationDatabase=admin `
         -d="$($DATABASE_NAME)" -c="$($COLLECTION_NAME)" `
         --file="""$($TEST_INPUT_FOLDER)\$($TEST_FILE_NAME)"""
     if (!$?) {

diff --git a/src/odbc-test/scripts/import_test_data.sh b/src/odbc-test/scripts/import_test_data.sh
@@ -21,7 +21,7 @@ do
     COLLECTION_NAME="$(basename -- ${FILENAME%.json})"
     TEST_FILE_NAME="$(basename -- ${FILENAME})"
 
-    mongoimport --quiet -u="${DOC_DB_USER_NAME}" -p="${DOC_DB_PASSWORD}" --authenticationDatabase=admin \
+    mongoimport -u="${DOC_DB_USER_NAME}" -p="${DOC_DB_PASSWORD}" --authenticationDatabase=admin \
         -d="${DATABASE_NAME}" -c="${COLLECTION_NAME}" \
         --file="""${TEST_INPUT_FOLDER}/${TEST_FILE_NAME}"""
     if [ $? -ne 0 ]

diff --git a/src/odbc-test/src/java_test.cpp b/src/odbc-test/src/java_test.cpp
@@ -385,90 +385,90 @@ BOOST_AUTO_TEST_CASE(TestDatabaseMetaDataGetTables) {
 
   int i = 1;
   while (hasNext) {
-    bool wasNull;
-    std::string value;
+    boost::optional<std::string> value;
     // TABLE_CAT (i.e., catalog - always NULL in our case)
-    if (_ctx.Get()->ResultSetGetString(resultSet, 1, value, wasNull, errInfo)
+    if (_ctx.Get()->ResultSetGetString(resultSet, 1, value,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(wasNull);
+    BOOST_CHECK(!value);
 
     // TABLE_CAT (i.e., catalog - always NULL in our case)
-    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_CAT", value, wasNull,
+    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_CAT", value, 
                                        errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(wasNull);
+    BOOST_CHECK(!value);
 
     // TABLE_SCHEM (i.e., database)
-    if (_ctx.Get()->ResultSetGetString(resultSet, 2, value, wasNull, errInfo)
+    if (_ctx.Get()->ResultSetGetString(resultSet, 2, value,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(value == "test");
+    BOOST_REQUIRE(value);
+    BOOST_REQUIRE(*value == "test");
 
     // TABLE_SCHEM (i.e., database)
-    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_SCHEM", value, wasNull,
+    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_SCHEM", value, 
                                        errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(value == "test");
+    BOOST_REQUIRE(value);
+    BOOST_REQUIRE(*value == "test");
 
     // TABLE_NAME
-    if (_ctx.Get()->ResultSetGetString(resultSet, 3, value, wasNull, errInfo)
+    if (_ctx.Get()->ResultSetGetString(resultSet, 3, value,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(value.size() > 0);
+    BOOST_REQUIRE(value);
+    BOOST_REQUIRE(value->size() > 0);
 
     // TABLE_NAME
-    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_NAME", value, wasNull,
+    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_NAME", value, 
                                        errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(value.size() > 0);
+    BOOST_CHECK(value);
+    BOOST_REQUIRE(value->size() > 0);
 
     // TABLE_TYPE
-    if (_ctx.Get()->ResultSetGetString(resultSet, 4, value, wasNull, errInfo)
+    if (_ctx.Get()->ResultSetGetString(resultSet, 4, value,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(value == "TABLE");
+    BOOST_REQUIRE(value);
+    BOOST_REQUIRE(*value == "TABLE");
 
     // TABLE_TYPE
-    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_TYPE", value, wasNull,
+    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_TYPE", value, 
                                        errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(value == "TABLE");
+    BOOST_REQUIRE(value);
+    BOOST_REQUIRE(*value == "TABLE");
 
     // check getRow
-    int val;
-    if (_ctx.Get()->ResultSetGetRow(resultSet, val, wasNull, errInfo)
+    boost::optional<int> val;
+    if (_ctx.Get()->ResultSetGetRow(resultSet, val,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_CHECK_EQUAL(val, i);
+    BOOST_REQUIRE(val);
+    BOOST_CHECK_EQUAL(*val, i);
 
     // Get next
     if (_ctx.Get()->ResultSetNext(resultSet, hasNext, errInfo)
@@ -533,111 +533,111 @@ BOOST_AUTO_TEST_CASE(TestDatabaseMetaDataGetColumns) {
   std::string databaseName = "test";
   int i = 1;
   while (hasNext) {
-    bool wasNull;
-    std::string value;
+    boost::optional<std::string> value;
     // TABLE_CAT (i.e., catalog - always NULL in our case)
-    if (_ctx.Get()->ResultSetGetString(resultSet, 1, value, wasNull, errInfo)
+    if (_ctx.Get()->ResultSetGetString(resultSet, 1, value,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       // grab string from first column
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(wasNull);
+    BOOST_CHECK(!value);
 
     // TABLE_CAT (i.e., catalog - always NULL in our case)
-    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_CAT", value, wasNull,
+    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_CAT", value, 
                                        errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {  // grab string from first
                                                     // column by name
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(wasNull);
+    BOOST_CHECK(!value);
 
     // TABLE_SCHEM (i.e., database)
-    if (_ctx.Get()->ResultSetGetString(resultSet, 2, value, wasNull, errInfo)
+    if (_ctx.Get()->ResultSetGetString(resultSet, 2, value,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_CHECK_EQUAL(databaseName, value);
+    BOOST_CHECK(value);
+    BOOST_CHECK_EQUAL(databaseName, *value);
 
     // TABLE_SCHEM (i.e., database)
-    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_SCHEM", value, wasNull,
+    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_SCHEM", value, 
                                        errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_CHECK_EQUAL(databaseName, value);
+    BOOST_CHECK(value);
+    BOOST_CHECK_EQUAL(databaseName, *value);
 
     // TABLE_NAME
-    if (_ctx.Get()->ResultSetGetString(resultSet, 3, value, wasNull, errInfo)
+    if (_ctx.Get()->ResultSetGetString(resultSet, 3, value,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(value.size() > 0);
+    BOOST_REQUIRE(value);
+    BOOST_REQUIRE(value->size() > 0);
 
     // TABLE_NAME
-    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_NAME", value, wasNull,
+    if (_ctx.Get()->ResultSetGetString(resultSet, "TABLE_NAME", value, 
                                        errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(value.size() > 0);
+    BOOST_REQUIRE(value);
+    BOOST_REQUIRE(value->size() > 0);
 
     // COLUMN_NAME
-    if (_ctx.Get()->ResultSetGetString(resultSet, 4, value, wasNull, errInfo)
+    if (_ctx.Get()->ResultSetGetString(resultSet, 4, value,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(value.size() > 0);
+    BOOST_REQUIRE(value);
+    BOOST_REQUIRE(value->size() > 0);
 
     // COLUMN_NAME
-    if (_ctx.Get()->ResultSetGetString(resultSet, "COLUMN_NAME", value, wasNull,
+    if (_ctx.Get()->ResultSetGetString(resultSet, "COLUMN_NAME", value, 
                                        errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(value.size() > 0);
+    BOOST_REQUIRE(value);
+    BOOST_REQUIRE(value->size() > 0);
 
     // ORDINAL_POSITION
-    int val;
-    if (_ctx.Get()->ResultSetGetInt(resultSet, 17, val, wasNull, errInfo)
+    boost::optional< int > val;
+    if (_ctx.Get()->ResultSetGetInt(resultSet, 17, val,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(val > 0);
+    BOOST_REQUIRE(val);
+    BOOST_REQUIRE(*val > 0);
 
     // ORDINAL_POSITION
-    if (_ctx.Get()->ResultSetGetInt(resultSet, "ORDINAL_POSITION", val, wasNull,
+    if (_ctx.Get()->ResultSetGetInt(resultSet, "ORDINAL_POSITION", val, 
                                     errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_REQUIRE(!wasNull);
-    BOOST_REQUIRE(val > 0);
+    BOOST_REQUIRE(val);
+    BOOST_REQUIRE(*val > 0);
 
     // check getRow
-    if (_ctx.Get()->ResultSetGetRow(resultSet, val, wasNull, errInfo)
+    if (_ctx.Get()->ResultSetGetRow(resultSet, val,  errInfo)
         != JniErrorCode::IGNITE_JNI_ERR_SUCCESS) {
       std::string errMsg = errInfo.errMsg;
       BOOST_FAIL(errMsg);
     }
-    BOOST_CHECK_EQUAL(val, i);
+    BOOST_REQUIRE(val);
+    BOOST_CHECK_EQUAL(*val, i);
 
     // Get next
     if (_ctx.Get()->ResultSetNext(resultSet, hasNext, errInfo)