Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storagetest #706

Closed
wants to merge 46 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
dc5f9e6
test keytar library
sfc-gh-ext-simba-jy Oct 10, 2023
5e3cb11
secure storage test
sfc-gh-ext-simba-jy Oct 10, 2023
3bd7fdc
Merge branch 'mfacache' of https://github.com/snowflakedb/snowflake-c…
sfc-gh-ext-simba-jy Oct 10, 2023
d78626e
fix error
sfc-gh-ext-simba-jy Oct 10, 2023
54ac99e
Merge branch 'master' into mfacache
sfc-gh-ext-simba-jy Oct 10, 2023
d3e0c1c
added a command in the docker file to insall linux library
sfc-gh-ext-simba-jy Oct 10, 2023
5c2de01
docker command for linux
sfc-gh-ext-simba-jy Oct 10, 2023
a871fd4
revert
sfc-gh-ext-simba-jy Oct 10, 2023
67e0fff
Merge branch 'master' into mfacache
sfc-gh-ext-simba-jy Oct 11, 2023
8e3b01c
SSO Token Cache
sfc-gh-ext-simba-jy Oct 12, 2023
3c1be02
mfaToken
sfc-gh-ext-simba-jy Oct 12, 2023
319ad44
Fix some errors
sfc-gh-ext-simba-jy Oct 18, 2023
e8f0c86
added session parameters
sfc-gh-ext-simba-jy Nov 7, 2023
40286c3
testing
sfc-gh-ext-simba-jy Nov 9, 2023
c082ca3
Applied idToken Cache
sfc-gh-ext-simba-jy Nov 14, 2023
81348b2
Merge branch 'master' into mfacache
sfc-gh-ext-simba-jy Nov 14, 2023
38324b7
fixed bugs, and remove unnecessary parts
sfc-gh-ext-simba-jy Nov 15, 2023
6e4e6f7
added id_token authentication test
sfc-gh-ext-simba-jy Nov 15, 2023
25cc0be
Removed mfa token implements from this feature
sfc-gh-ext-simba-jy Nov 15, 2023
771afa0
fixed error
sfc-gh-ext-simba-jy Nov 15, 2023
9619abc
fixed lint
sfc-gh-ext-simba-jy Nov 15, 2023
377bd16
added logging option
sfc-gh-ext-simba-jy Nov 15, 2023
4800ca6
added libscret-devel'
sfc-gh-ext-simba-jy Nov 15, 2023
13db3ca
testing
sfc-gh-ext-simba-jy Nov 15, 2023
54c5028
removed downloading libsecret
sfc-gh-ext-simba-jy Nov 16, 2023
0b805f5
libsecret client testing
sfc-gh-ext-simba-jy Nov 16, 2023
9f4f42c
removed unnecessary codes
sfc-gh-ext-simba-jy Nov 16, 2023
5a81ba9
added semicolons, and made lint fixes
sfc-gh-ext-simba-jy Nov 16, 2023
bd672da
Merge branch 'master' into mfacache
sfc-gh-ext-simba-jy Nov 16, 2023
d693f02
added optional dependencies
sfc-gh-ext-simba-jy Nov 17, 2023
e67dc3d
Merge branch 'mfacache' of https://github.com/snowflakedb/snowflake-c…
sfc-gh-ext-simba-jy Nov 17, 2023
5b269a7
dependencies
sfc-gh-ext-simba-jy Nov 17, 2023
43b25ae
dependencies
sfc-gh-ext-simba-jy Nov 17, 2023
fdf53dc
add dependencies
sfc-gh-ext-simba-jy Nov 17, 2023
2b4707a
revert back
sfc-gh-ext-simba-jy Nov 17, 2023
c101a4b
added manual test for idtoken and refactored
sfc-gh-ext-simba-jy Nov 21, 2023
7b69882
removed unnecessary codes
sfc-gh-ext-simba-jy Nov 21, 2023
a2c2097
brought keytar library into the local storage
sfc-gh-ext-simba-jy Nov 22, 2023
bdc3743
testing
sfc-gh-ext-simba-jy Nov 22, 2023
36e1c20
testing
sfc-gh-ext-simba-jy Nov 22, 2023
bdd9a99
testing
sfc-gh-ext-simba-jy Nov 23, 2023
ffcb97e
Merge branch 'master' into storagetest
sfc-gh-ext-simba-jy Nov 23, 2023
eebbbec
dependency testing
sfc-gh-ext-simba-jy Nov 23, 2023
6f7ade0
Merge branch 'storagetest' of https://github.com/snowflakedb/snowflak…
sfc-gh-ext-simba-jy Nov 23, 2023
b8c9c60
fixed Secure storage code
sfc-gh-ext-simba-jy Nov 23, 2023
3eb3c49
fixed error
sfc-gh-ext-simba-jy Nov 23, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/image/Dockerfile.nodejs-centos7-node14-test
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ FROM centos:7
# update OS
RUN yum -y update && \
yum -y install epel-release && \
yum -y install centos-release-scl
yum -y install centos-release-scl

# nvm environment variables
ENV NVM_DIR /usr/local/nvm
Expand Down
2 changes: 1 addition & 1 deletion lib/agent/ocsp_response_cache.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ var maxAgeSec = GlobalConfig.getOcspResponseCacheMaxAge();
Errors.assertInternal(Util.number.isPositiveInteger(sizeLimit));
Errors.assertInternal(Util.number.isPositiveInteger(maxAgeSec));

const cacheDir = GlobalConfig.mkdirCacheDir();
const cacheDir = GlobalConfig.mkdirCacheDir(process.env.SF_OCSP_RESPONSE_CACHE_DIR);
const cacheFileName = path.join(cacheDir, "ocsp_response_cache.json");
// create a cache to store the responses, dynamically changes in size
var cache;
Expand Down
4 changes: 4 additions & 0 deletions lib/authentication/auth_default.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@
{
return;
};

this.reauthenticate = async function (body) {
return;

Check warning on line 35 in lib/authentication/auth_default.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/auth_default.js#L35

Added line #L35 was not covered by tests
}
}

module.exports = auth_default;
49 changes: 49 additions & 0 deletions lib/authentication/auth_idtoken.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
/*
* Copyright (c) 2015-2023 Snowflake Computing Inc. All rights reserved.
*/

const auth_web = require('./auth_web');

/**
* Creates an oauth authenticator.
*
* @param {String} token
*
* @returns {Object}
* @constructor
*/
function auth_idToken(connectionConfig, httpClient) {

this.idToken = connectionConfig.idToken;

/**
* Update JSON body with token.
*
* @param {JSON} body
*
* @returns {null}
*/
this.updateBody = function (body) {
body['data']['TOKEN'] = this.idToken;
body['data']['AUTHENTICATOR'] = 'ID_TOKEN';
};

this.resetSecret = function () {
this.idToken = null;

Check warning on line 32 in lib/authentication/auth_idtoken.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/auth_idtoken.js#L32

Added line #L32 was not covered by tests
};

this.authenticate = async function (authenticator, serviceName, account, username) {
return;
};

this.reauthenticate = async function (body) {
const auth = new auth_web(connectionConfig, httpClient);
await auth.authenticate(connectionConfig.getAuthenticator(),

Check warning on line 41 in lib/authentication/auth_idtoken.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/auth_idtoken.js#L40-L41

Added lines #L40 - L41 were not covered by tests
connectionConfig.getServiceName(),
connectionConfig.account,
connectionConfig.username);
auth.updateBody(body);

Check warning on line 45 in lib/authentication/auth_idtoken.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/auth_idtoken.js#L45

Added line #L45 was not covered by tests
};
}

module.exports = auth_idToken;
4 changes: 4 additions & 0 deletions lib/authentication/auth_keypair.js
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,10 @@
throw jwtErr;
}
}

this.reauthenticate = async function (body) {
return;

Check warning on line 197 in lib/authentication/auth_keypair.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/auth_keypair.js#L197

Added line #L197 was not covered by tests
}
}

module.exports = auth_keypair;
4 changes: 4 additions & 0 deletions lib/authentication/auth_oauth.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@
{
return;
};

this.reauthenticate = async function (body) {
return;

Check warning on line 35 in lib/authentication/auth_oauth.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/auth_oauth.js#L35

Added line #L35 was not covered by tests
}
}

module.exports = auth_oauth;
4 changes: 4 additions & 0 deletions lib/authentication/auth_okta.js
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,10 @@
step5(responseHtml);
};

this.reauthenticate = async function (body) {
return;

Check warning on line 101 in lib/authentication/auth_okta.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/auth_okta.js#L101

Added line #L101 was not covered by tests
}

/**
* Obtain the SSO URL and token URL.
*
Expand Down
5 changes: 5 additions & 0 deletions lib/authentication/auth_web.js
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
*/
this.updateBody = function (body)
{
body['data']['AUTHENTICATOR'] = 'EXTERNALBROWSER'
body['data']['TOKEN'] = token;
body['data']['PROOF_KEY'] = proofKey;
};
Expand Down Expand Up @@ -99,6 +100,10 @@
processGet(tokenData);
};

this.reauthenticate = async function (body) {
return;

Check warning on line 104 in lib/authentication/auth_web.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/auth_web.js#L104

Added line #L104 was not covered by tests
}

/**
* Create server to retrieve SAML token.
*
Expand Down
19 changes: 12 additions & 7 deletions lib/authentication/authentication.js
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
/*
* Copyright (c) 2015-2021 Snowflake Computing Inc. All rights reserved.
* Copyright (c) 2015-2023 Snowflake Computing Inc. All rights reserved.
*/

const auth_default = require('./auth_default');
const auth_web = require('./auth_web');
const auth_keypair = require('./auth_keypair');
const auth_oauth = require('./auth_oauth');
const auth_okta = require('./auth_okta');
const auth_idToken = require('./auth_idtoken');

const authenticationTypes =
{
DEFAULT_AUTHENTICATOR: 'SNOWFLAKE', // default authenticator name
EXTERNAL_BROWSER_AUTHENTICATOR: 'EXTERNALBROWSER',
KEY_PAIR_AUTHENTICATOR: 'SNOWFLAKE_JWT',
OAUTH_AUTHENTICATOR: 'OAUTH',
ID_TOKEN_AUTHENTICATOR: 'ID_TOKEN',
};

exports.authenticationTypes = authenticationTypes;
Expand Down Expand Up @@ -73,9 +75,13 @@
if (auth === authenticationTypes.DEFAULT_AUTHENTICATOR) {
return new auth_default(connectionConfig.password);
} else if (auth === authenticationTypes.EXTERNAL_BROWSER_AUTHENTICATOR) {
return new auth_web(connectionConfig, httpClient);
}
if (auth === authenticationTypes.KEY_PAIR_AUTHENTICATOR) {
if(connectionConfig.getClientStoreTemporaryCredential() && connectionConfig.idToken !== null) {
return new auth_idToken(connectionConfig, httpClient);

Check warning on line 79 in lib/authentication/authentication.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/authentication.js#L78-L79

Added lines #L78 - L79 were not covered by tests
}
else {
return new auth_web(connectionConfig, httpClient);

Check warning on line 82 in lib/authentication/authentication.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/authentication.js#L82

Added line #L82 was not covered by tests
}
} else if (auth === authenticationTypes.KEY_PAIR_AUTHENTICATOR) {

Check warning on line 84 in lib/authentication/authentication.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/authentication.js#L84

Added line #L84 was not covered by tests
return new auth_keypair(connectionConfig.getPrivateKey(),
connectionConfig.getPrivateKeyPath(),
connectionConfig.getPrivateKeyPass());
Expand All @@ -87,13 +93,12 @@
connectionConfig.account,
connectionConfig.getClientType(),
connectionConfig.getClientVersion(),
httpClient
);
httpClient);
} else {
// Authenticator specified does not exist
return new auth_default(connectionConfig.password);
}
};
}

/**
* Returns the boolean describing if the provided authenticator is okta or not.
Expand Down
84 changes: 84 additions & 0 deletions lib/authentication/secureStorage.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
/*
* Copyright (c) 2015-2023 Snowflake Computing Inc. All rights reserved.
*/

const path = require('path');
const GlobalConfig = require('../global_config');
const Logger = require('../logger');
const fs = require('fs/promises');

function createCredentialCacheDir() {
const cacheDirectory = GlobalConfig.mkdirCacheDir(process.env.SF_TEMPORARY_CREDENTIAL_CACHE_DIR);
const credCache = path.join(cacheDirectory, 'temporary_credential.json');
Logger.getInstance().info('Cache directory: ', credCache);
return credCache;
}

/**
*
* @param {String} host
* @param {String} user
* @param {String} credType
* @returns
*/

function buildTemporaryCredentialName(host, user, credType) {
return `{${host.toUpperCase()}}:{${user.toUpperCase()}}:{SF_NODE_JS_DRIVER}:{${credType}}}`;
}

async function writeCredential(host, user, credType, token){
if (!token || token === '') {
Logger.getInstance().debug('Token is not provided');
return;

Check warning on line 32 in lib/authentication/secureStorage.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/secureStorage.js#L31-L32

Added lines #L31 - L32 were not covered by tests
} else {
try {
try {
const keytar = require('keytar');
await keytar.setPassword(host, buildTemporaryCredentialName(host, user, credType), token);
} catch (err) {
const dir = createCredentialCacheDir();
Logger.getInstance().trace('Failed to save the token in the credential manager. The token will be saved on the the local machine at', dir);
await fs.writeFile(dir, token, 'utf8');
}
} catch (err){
Logger.getInstance().error('Failed to save Credential: ', err.message);

Check warning on line 44 in lib/authentication/secureStorage.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/secureStorage.js#L44

Added line #L44 was not covered by tests
}
}
}

async function readCredential(host, user, credType) {
try {
try {
const keytar = require('keytar');
return await keytar.getPassword(host, buildTemporaryCredentialName(host, user, credType));
} catch (err) {
const dir = createCredentialCacheDir();
Logger.getInstance().trace('Failed to read the token from the credential manager. Searching the token at ', dir);
return await fs.readFile(dir, 'utf8');
}
} catch (err){
Logger.getInstance().error('Failed to save Credential: ', err.message);
}
}

async function deleteCredential(host, user, credType) {
try {
try {
const keytar = require('keytar');
return await keytar.deletePassword(host, buildTemporaryCredentialName(host, user, credType));
} catch (err) {
const dir = createCredentialCacheDir();
return await fs.unlink(dir);
}
} catch (err){
Logger.getInstance().error('Failed to save Credential: ', err.message);

Check warning on line 74 in lib/authentication/secureStorage.js

View check run for this annotation

Codecov / codecov/patch

lib/authentication/secureStorage.js#L74

Added line #L74 was not covered by tests
}
}

module.exports = {
createCredentialCacheDir,
buildTemporaryCredentialName,
writeCredential,
readCredential,
deleteCredential
};
19 changes: 12 additions & 7 deletions lib/connection/connection.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
const Url = require('url');
const QueryString = require('querystring');
const GSErrors = require('../constants/gs_errors')
const SecureStorage = require('../authentication/secureStorage');
const QueryStatus = require('../constants/query_status');

var Util = require('../util');
Expand Down Expand Up @@ -302,6 +303,12 @@
// the connection can be passed in when invoking the connection.connect()
// callback
var self = this;
const authType = Authenticator.authenticationTypes;

Check warning on line 306 in lib/connection/connection.js

View check run for this annotation

Codecov / codecov/patch

lib/connection/connection.js#L306

Added line #L306 was not covered by tests

if (connectionConfig.getClientStoreTemporaryCredential()) {
connectionConfig.idToken = await SecureStorage.readCredential(connectionConfig.account,

Check warning on line 309 in lib/connection/connection.js

View check run for this annotation

Codecov / codecov/patch

lib/connection/connection.js#L308-L309

Added lines #L308 - L309 were not covered by tests
connectionConfig.username, authType.ID_TOKEN_AUTHENTICATOR);
}

// Get authenticator to use
const auth = Authenticator.getAuthenticator(connectionConfig, context.getHttpClient());
Expand All @@ -311,32 +318,30 @@
await auth.authenticate(connectionConfig.getAuthenticator(),
connectionConfig.getServiceName(),
connectionConfig.account,
connectionConfig.username)
.then(() =>
{
connectionConfig.username);

// JSON for connection
var body = Authenticator.formAuthJSON(connectionConfig.getAuthenticator(),
connectionConfig.account,
connectionConfig.username,
connectionConfig.getClientType(),
connectionConfig.getClientVersion(),
connectionConfig.getClientEnvironment());

// Update JSON body with the authentication values
auth.updateBody(body);

// Request connection
services.sf.connect({
callback: connectCallback(self, callback),
json: body
});
});
}
catch (authErr)
{
callback(authErr);
}

// return the connection to facilitate chaining
return this;
};
Expand Down
20 changes: 19 additions & 1 deletion lib/connection/connection_config.js
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@
'includeRetryReason',
'disableQueryContextCache',
'retryTimeout',
'clientStoreTemporaryCredential',
];
const Logger = require('../logger');

Expand Down Expand Up @@ -520,6 +521,14 @@
includeRetryReason = options.includeRetryReason;
}

let clientStoreTemporaryCredential = true;
if (Util.exists(options.clientStoreTemporaryCredential)) {
Errors.checkArgumentValid(Util.isBoolean(options.clientStoreTemporaryCredential),
ErrorCodes.ERR_CONN_CREATE_INVALID_CLIENT_STORE_TEMPORARY_CREDENTIAL);

clientStoreTemporaryCredential = options.clientStoreTemporaryCredential;

Check warning on line 529 in lib/connection/connection_config.js

View check run for this annotation

Codecov / codecov/patch

lib/connection/connection_config.js#L529

Added line #L529 was not covered by tests
}

/**
* Returns an object that contains information about the proxy hostname, port,
* etc. for when http requests are made.
Expand Down Expand Up @@ -801,6 +810,15 @@
return disableQueryContextCache;
}

/**
* Returns whether the auth token saves on the local machine or not.
*
* @returns {Boolean}
*/
this.getClientStoreTemporaryCredential = function () {
return clientStoreTemporaryCredential;
}

/**
* Returns the client config file
*
Expand Down Expand Up @@ -992,7 +1010,7 @@
name: PARAM_RETRY_SF_MAX_SLEEP_TIME,
defaultValue: 16,
validate: isNonNegativeNumber
}
},
];
}

Expand Down
1 change: 1 addition & 0 deletions lib/constants/error_messages.js
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ exports[404043] = 'Invalid clientConfigFile value. The specified value must be a
exports[404044] = 'Invalid retryTimeout value. The specified value must be a number.';
exports[404045] = 'Invalid account. The specified value must be a valid subdomain string.';
exports[404046] = 'Invalid region. The specified value must be a valid subdomain string.';
exports[404047] = 'Invalid clientStoreTemporaryCredential. The specified value must be a boolean.';

// 405001
exports[405001] = 'Invalid callback. The specified value must be a function.';
Expand Down
1 change: 1 addition & 0 deletions lib/errors.js
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ codes.ERR_CONN_CREATE_INVALID_CLIENT_CONFIG_FILE = 404043;
codes.ERR_CONN_CREATE_INVALID_RETRY_TIMEOUT = 404044;
codes.ERR_CONN_CREATE_INVALID_ACCOUNT_REGEX = 404045;
codes.ERR_CONN_CREATE_INVALID_REGION_REGEX = 404046;
codes.ERR_CONN_CREATE_INVALID_CLIENT_STORE_TEMPORARY_CREDENTIAL = 404047;

// 405001
codes.ERR_CONN_CONNECT_INVALID_CALLBACK = 405001;
Expand Down
Loading
Loading