Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #70 - Add Serverless Support #73

Merged
merged 21 commits into from
Nov 21, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
b401f48
Initial support for OpenSearch Serverless, based simply on parsing URL
currantw Nov 13, 2024
b855e2d
Minor unrelated cleanup
currantw Nov 13, 2024
8491671
Initial updates to add `IsServerless` configuration option.
currantw Nov 14, 2024
c490706
Add `.idea/*` to project `.gitignore`
currantw Nov 14, 2024
0820b35
Fix small errors
currantw Nov 15, 2024
44336bf
Update documentation
currantw Nov 15, 2024
f39036d
Minor cleanuo
currantw Nov 15, 2024
29ac366
Fix format string error
currantw Nov 15, 2024
929088e
Exclude `.pyc` files from test runner, which supports Python in venv
currantw Nov 15, 2024
aa76d77
Add `is_serverless` member variable to avoid repeatedly determining it.
currantw Nov 15, 2024
63ffb0c
Also exclude `.o` and `.d` object files from test runner.
currantw Nov 16, 2024
7fcfa87
Rename configuration option from `IsServerless` to `ServerlessOverrid…
currantw Nov 18, 2024
f0b3abf
Add default `is_servrless` value to connection tests.
currantw Nov 18, 2024
caefd49
Revert unintentional change.
currantw Nov 18, 2024
ec3386a
Add clarifying comments.
currantw Nov 18, 2024
caf9d39
Add constant for OpenSearch distribution.
currantw Nov 19, 2024
4e55ffe
Rename from `is_serverless` to `is_aoss_serverless`.
currantw Nov 19, 2024
8e2f718
Update to first determine if serverless via URL, then default to usin…
currantw Nov 19, 2024
bed6886
Simplify logic, remove tertiary
currantw Nov 20, 2024
c5b48f7
Remove `ServerlessOverride` configuration option.
currantw Nov 21, 2024
effccef
Minor comment cleanup
currantw Nov 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,6 @@ lib64/*
*.tlog
include/*
packages/*
# Visual Studio
.vs/*
# Visual Studio Code
.vscode/*
src/psqlodbc/psqlodbcBuilder/x64_*
src/*/x64
src/*/Win32
Expand Down Expand Up @@ -60,3 +56,8 @@ CTestTestfile.cmake
/src/PowerBIConnector/obj/
/src/PowerBIConnector/.vs/
src/vcpkg_installed/

# IDEs
.vs/*
.vscode/*
.idea/*
currantw marked this conversation as resolved.
Show resolved Hide resolved
4 changes: 2 additions & 2 deletions docs/dev/BUILD_INSTRUCTIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ See [run_tests.md](./run_tests.md)

**BUILD_WITH_TESTS**

(Defaults to ON) If disabled, all tests and and test dependencies will be excluded from build which will optimize the installer package size. This option can set with the command line (using `-D`).
(Defaults to ON) If disabled, all tests and test dependencies will be excluded from build which will optimize the installer package size. This option can set with the command line (using `-D`).

### Working With SSL/TLS

Expand All @@ -119,7 +119,7 @@ If you plan to use OpenSearch Dashboards, as suggested for this project, you mus

### Setting up a DSN

A **D**ata **S**ouce **N**ame is used to store driver information in the system. By storing the information in the system, the information does not need to be specified each time the driver connects.
A **D**ata **S**ource **N**ame is used to store driver information in the system. By storing the information in the system, the information does not need to be specified each time the driver connects.

#### Windows

Expand Down
3 changes: 2 additions & 1 deletion docs/user/configuration_options.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,10 @@
| Option | Description | Type | Acceptable Values | Default |
|------------------------|------------------------------------------------------------------------------------------------------------------------------------------|---------|---------------------------------|-------------|
| `UseSSL` | Whether to establish the connection over SSL/TLS | boolean | `0` or `1` | `0` (false) |
| `HostnameVerification` | Indicate whether certificate hostname verification should be performed for an SSL/TLS connection. | boolean | `0` or `1` | `1` (true) |
| `HostnameVerification` | Whether certificate hostname verification should be performed for an SSL/TLS connection. | boolean | `0` or `1` | `1` (true) |
| `ResponseTimeout` | The maximum time to wait for responses from the `Host`, in seconds. | integer | | `10` |
| `FetchSize` | The page size for all cursor requests. The default value (-1) uses server-defined page size. Set FetchSize to 0 for non-cursor behavior. | integer | `-1`, `0` or any positive value | `-1` |
| `ServerlessOverride` | Whether the connection is to an OpenSearch Serverless cluster. If not specified, it is determined by parsing the server URL. | boolean | `0` or `1` | |

#### Logging Options

Expand Down
5 changes: 3 additions & 2 deletions src/TestRunner/test_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,11 @@
PERFORMANCE_INFO = "performance_info"
PERFORMANCE_RESULTS = "performance_results"
EXCLUDE_EXTENSION_LIST = (
".py", ".c", ".cmake", ".log",
".py", ".pyc", ".c", ".cmake", ".log",
".pdb", ".dll", ".sln", ".vcxproj", ".user",
".tlog", ".lastbuildstate", ".filters",
".obj", ".exp", ".lib", ".h", ".cpp", ".ilk")
".obj", ".o", ".d", ".exp", ".lib", ".h",
".cpp", ".ilk")
total_failures = 0
SYNC_START = "%%__PARSE__SYNC__START__%%"
SYNC_SEP = "%%__SEP__%%"
Expand Down
6 changes: 3 additions & 3 deletions src/UnitTests/UTConn/test_conn.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,14 @@ const std::string invalid_port = "920";
const std::string invalid_user = "amin";
const std::string invalid_pw = "amin";
const std::string invalid_region = "bad-region";
runtime_options valid_opt_val = {{valid_host, valid_port, "1", "0"},
runtime_options valid_opt_val = {{valid_host, valid_port, "1", "0", ""},
{"BASIC", valid_user, valid_pw, valid_region, valid_tunnel_host},
{use_ssl, false, "", "", "", ""}};
runtime_options invalid_opt_val = {
{invalid_host, invalid_port, "1", "0"},
{invalid_host, invalid_port, "1", "0", ""},
{"BASIC", invalid_user, invalid_pw, valid_region, valid_tunnel_host},
{use_ssl, false, "", "", "", ""}};
runtime_options missing_opt_val = {{"", "", "1", "0"},
runtime_options missing_opt_val = {{"", "", "1", "0", ""},
{"BASIC", "", invalid_pw, valid_region, valid_tunnel_host},
{use_ssl, false, "", "", "", ""}};

Expand Down
59 changes: 46 additions & 13 deletions src/sqlodbc/dlg_specific.c
Original file line number Diff line number Diff line change
Expand Up @@ -37,20 +37,43 @@ void makeConnectString(char *connect_string, const ConnInfo *ci, UWORD len) {
encode(ci->password, encoded_item, sizeof(encoded_item));
/* fundamental info */
nlen = MAX_CONNECT_STRING;

const char* connect_format_string =
"%s=%s;"
INI_SERVER "=%s;"
"database=OpenSearch;"
INI_PORT "=%s;"
INI_USERNAME_ABBR "=%s;"
INI_PASSWORD_ABBR "=%s;"
INI_AUTH_MODE "=%s;"
INI_REGION "=%s;"
INI_TUNNEL_HOST "=%s;"
INI_SERVERLESS_OVERRIDE "=%s"
INI_SSL_USE "=%d;"
INI_SSL_HOST_VERIFY "=%d;"
INI_LOG_LEVEL "=%d;"
INI_LOG_OUTPUT "=%s;"
INI_TIMEOUT "=%s;"
INI_FETCH_SIZE "=%s;";

olen = snprintf(
connect_string, nlen,
"%s=%s;" INI_SERVER
"=%s;"
"database=OpenSearch;" INI_PORT "=%s;" INI_USERNAME_ABBR
"=%s;" INI_PASSWORD_ABBR "=%s;" INI_AUTH_MODE "=%s;" INI_REGION
"=%s;" INI_TUNNEL_HOST "=%s;" INI_SSL_USE "=%d;" INI_SSL_HOST_VERIFY
"=%d;" INI_LOG_LEVEL "=%d;" INI_LOG_OUTPUT "=%s;" INI_TIMEOUT "=%s;"
INI_FETCH_SIZE "=%s;",
connect_format_string,
got_dsn ? "DSN" : "DRIVER", got_dsn ? ci->dsn : ci->drivername,
ci->server, ci->port, ci->username, encoded_item, ci->authtype,
ci->region, ci->tunnel_host, (int)ci->use_ssl, (int)ci->verify_server,
(int)ci->drivers.loglevel, ci->drivers.output_dir,
ci->response_timeout, ci->fetch_size);
ci->server,
ci->port,
ci->username,
encoded_item,
ci->authtype,
ci->region,
ci->tunnel_host,
ci->is_serverless,
(int)ci->use_ssl,
(int)ci->verify_server,
(int)ci->drivers.loglevel,
ci->drivers.output_dir,
ci->response_timeout,
ci->fetch_size);
if (olen < 0 || olen >= nlen) {
connect_string[0] = '\0';
return;
Expand Down Expand Up @@ -112,6 +135,8 @@ BOOL copyConnAttributes(ConnInfo *ci, const char *attribute,
STRCPY_FIXED(ci->region, value);
else if (stricmp(attribute, INI_TUNNEL_HOST) == 0)
STRCPY_FIXED(ci->tunnel_host, value);
else if (stricmp(attribute, INI_SERVERLESS_OVERRIDE) == 0)
STRCPY_FIXED(ci->is_serverless, value);
else if (stricmp(attribute, INI_SSL_USE) == 0)
ci->use_ssl = (char)atoi(value);
else if (stricmp(attribute, INI_SSL_HOST_VERIFY) == 0)
Expand Down Expand Up @@ -141,7 +166,7 @@ static void getCiDefaults(ConnInfo *ci) {
strncpy(ci->port, DEFAULT_PORT, SMALL_REGISTRY_LEN);
strncpy(ci->response_timeout, DEFAULT_RESPONSE_TIMEOUT_STR,
SMALL_REGISTRY_LEN);
strncpy(ci->fetch_size, DEFAULT_FETCH_SIZE_STR,
strncpy(ci->fetch_size, DEFAULT_FETCH_SIZE,
SMALL_REGISTRY_LEN);
strncpy(ci->authtype, DEFAULT_AUTHTYPE, MEDIUM_REGISTRY_LEN);
if (ci->password.name != NULL)
Expand All @@ -150,6 +175,7 @@ static void getCiDefaults(ConnInfo *ci) {
strncpy(ci->username, DEFAULT_USERNAME, MEDIUM_REGISTRY_LEN);
strncpy(ci->region, DEFAULT_REGION, MEDIUM_REGISTRY_LEN);
strncpy(ci->tunnel_host, DEFAULT_TUNNEL_HOST, MEDIUM_REGISTRY_LEN);
strncpy(ci->is_serverless, DEFAULT_IS_SERVERLESS, SMALL_REGISTRY_LEN);
ci->use_ssl = DEFAULT_USE_SSL;
ci->verify_server = DEFAULT_VERIFY_SERVER;
strcpy(ci->drivers.output_dir, "C:\\");
Expand Down Expand Up @@ -258,6 +284,10 @@ void getDSNinfo(ConnInfo *ci, const char *configDrvrname) {
sizeof(temp), ODBC_INI)
> 0)
STRCPY_FIXED(ci->tunnel_host, temp);
if (SQLGetPrivateProfileString(DSN, INI_SERVERLESS_OVERRIDE, NULL_STRING, temp,
sizeof(temp), ODBC_INI)
> 0)
STRCPY_FIXED(ci->is_serverless, temp);
if (SQLGetPrivateProfileString(DSN, INI_SSL_USE, NULL_STRING, temp,
sizeof(temp), ODBC_INI)
> 0)
Expand Down Expand Up @@ -313,6 +343,7 @@ void writeDSNinfo(const ConnInfo *ci) {
SQLWritePrivateProfileString(DSN, INI_AUTH_MODE, ci->authtype, ODBC_INI);
SQLWritePrivateProfileString(DSN, INI_REGION, ci->region, ODBC_INI);
SQLWritePrivateProfileString(DSN, INI_TUNNEL_HOST, ci->tunnel_host, ODBC_INI);
SQLWritePrivateProfileString(DSN, INI_SERVERLESS_OVERRIDE, ci->is_serverless, ODBC_INI);
ITOA_FIXED(temp, ci->use_ssl);
SQLWritePrivateProfileString(DSN, INI_SSL_USE, temp, ODBC_INI);
ITOA_FIXED(temp, ci->verify_server);
Expand Down Expand Up @@ -455,7 +486,7 @@ void CC_conninfo_init(ConnInfo *conninfo, UInt4 option) {
strncpy(conninfo->port, DEFAULT_PORT, SMALL_REGISTRY_LEN);
strncpy(conninfo->response_timeout, DEFAULT_RESPONSE_TIMEOUT_STR,
SMALL_REGISTRY_LEN);
strncpy(conninfo->fetch_size, DEFAULT_FETCH_SIZE_STR,
strncpy(conninfo->fetch_size, DEFAULT_FETCH_SIZE,
SMALL_REGISTRY_LEN);
strncpy(conninfo->authtype, DEFAULT_AUTHTYPE, MEDIUM_REGISTRY_LEN);
if (conninfo->password.name != NULL)
Expand All @@ -464,6 +495,7 @@ void CC_conninfo_init(ConnInfo *conninfo, UInt4 option) {
strncpy(conninfo->username, DEFAULT_USERNAME, MEDIUM_REGISTRY_LEN);
strncpy(conninfo->region, DEFAULT_REGION, MEDIUM_REGISTRY_LEN);
strncpy(conninfo->tunnel_host, DEFAULT_TUNNEL_HOST, MEDIUM_REGISTRY_LEN);
strncpy(conninfo->is_serverless, DEFAULT_IS_SERVERLESS, SMALL_REGISTRY_LEN);
conninfo->use_ssl = DEFAULT_USE_SSL;
conninfo->verify_server = DEFAULT_VERIFY_SERVER;

Expand Down Expand Up @@ -505,6 +537,7 @@ void CC_copy_conninfo(ConnInfo *ci, const ConnInfo *sci) {
CORR_STRCPY(authtype);
CORR_STRCPY(region);
CORR_STRCPY(tunnel_host);
CORR_STRCPY(is_serverless);
NAME_TO_NAME(ci->password, sci->password);
CORR_VALCPY(use_ssl);
CORR_VALCPY(verify_server);
Expand Down
5 changes: 3 additions & 2 deletions src/sqlodbc/dlg_specific.h
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,9 @@ extern "C" {
#define INI_LOG_OUTPUT "logOutput"
#define INI_TIMEOUT "responseTimeout"
#define INI_FETCH_SIZE "fetchSize"
#define INI_SERVERLESS_OVERRIDE "ServerlessOverride"

#define DEFAULT_FETCH_SIZE -1
#define DEFAULT_FETCH_SIZE_STR "-1"
currantw marked this conversation as resolved.
Show resolved Hide resolved
#define DEFAULT_FETCH_SIZE "-1"
#define DEFAULT_RESPONSE_TIMEOUT 10 // Seconds
#define DEFAULT_RESPONSE_TIMEOUT_STR "10"
#define DEFAULT_AUTHTYPE "NONE"
Expand All @@ -65,6 +65,7 @@ extern "C" {
#define DEFAULT_DESC ""
#define DEFAULT_DSN ""
#define DEFAULT_VERIFY_SERVER 1
#define DEFAULT_IS_SERVERLESS ""

#define AUTHTYPE_NONE "NONE"
#define AUTHTYPE_BASIC "BASIC"
Expand Down
Loading
Loading