feat: Add authentication option for snowflake connector #3039

lsaiken · 2022-08-08T23:37:31Z

Signed-off-by: Lisa Aiken [email protected]

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

felixwang9817

/lgtm

feast-ci-bot · 2022-08-08T23:45:09Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: felixwang9817, lsaiken

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [felixwang9817]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

felixwang9817 · 2022-08-08T23:46:16Z

hey @lsaiken thanks for the PR! everything lgtm except there's a lint error that you'll need to fix

you can find more about how we handle linting here

codecov-commenter · 2022-08-08T23:52:56Z

Codecov Report

Merging #3039 (9289ff0) into master (e78d1d7) will increase coverage by 9.09%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #3039      +/-   ##
==========================================
+ Coverage   67.59%   76.68%   +9.09%     
==========================================
  Files         168      198      +30     
  Lines       14712    16348    +1636     
==========================================
+ Hits         9944    12536    +2592     
+ Misses       4768     3812     -956

Flag	Coverage Δ
integrationtests	`67.44% <100.00%> (-0.15%)`	⬇️
unittests	`58.63% <100.00%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...hon/feast/infra/offline_stores/snowflake_source.py	`93.16% <ø> (+0.85%)`	⬆️
sdk/python/feast/infra/utils/snowflake_utils.py	`77.77% <ø> (ø)`
sdk/python/feast/infra/offline_stores/snowflake.py	`83.05% <100.00%> (+0.14%)`	⬆️
sdk/python/feast/infra/online_stores/snowflake.py	`98.16% <100.00%> (+0.03%)`	⬆️
...offline_stores/contrib/spark_repo_configuration.py	`100.00% <0.00%> (ø)`
...s/contrib/spark_offline_store/tests/data_source.py	`40.42% <0.00%> (ø)`
.../online_stores/contrib/hbase_online_store/hbase.py	`37.25% <0.00%> (ø)`
...ib/trino_offline_store/test_config/manual_tests.py	`33.33% <0.00%> (ø)`
...b/cassandra_online_store/cassandra_online_store.py	`2.63% <0.00%> (ø)`
.../contrib/postgres_offline_store/postgres_source.py	`49.01% <0.00%> (ø)`
... and 98 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

sfc-gh-madkins · 2022-08-09T12:20:04Z

@lsaiken did you run into any issues of having to authenticate an excessive amount?

See https://docs.snowflake.com/en/user-guide/admin-security-fed-auth-use.html#using-connection-caching-to-minimize-the-number-of-prompts-for-authentication-optional

sfc-gh-madkins · 2022-08-09T12:21:55Z

sdk/python/feast/infra/offline_stores/snowflake.py

@@ -91,6 +91,9 @@ class SnowflakeOfflineStoreConfig(FeastConfigBaseModel):
    schema_: Optional[str] = Field(None, alias="schema")
    """ Snowflake schema name """

+    authenticator_: Optional[str] = Field("", alias="authenticator")


Was "authenticator" a reserved word?

No, it's not a reserved keyword. I'll make the adjustment. Thanks for catching that!

To answer your question about excessive authentication, I didn't run into any issues having to authenticate an excessive amount. During testing, it only authenticates automatically through Okta when the snowflake.connector.connect commands are called.

sfc-gh-madkins · 2022-08-09T18:08:56Z

sdk/python/feast/infra/offline_stores/snowflake.py

@@ -91,6 +91,9 @@ class SnowflakeOfflineStoreConfig(FeastConfigBaseModel):
    schema_: Optional[str] = Field(None, alias="schema")
    """ Snowflake schema name """

+    authenticator: Optional[str] = ""


Got it on excessive authentication.

See following error in integration test:

"E snowflake.connector.errors.DatabaseError: 390139 (08001): None: Failed to connect to DB: ***.snowflakecomputing.com:443, The specified authenticator is not accepted by your Snowflake account configuration. Please contact your local system administrator to get the correct URL to use."

My guess is your code is always passing the authenticator kwarg, when it should only be passed if specified

authenticator: Optional[str] = ""
vs
authenticator: Optional[str] = None

would be my first guess

Signed-off-by: Lisa Aiken <[email protected]>

felixwang9817 · 2022-08-10T16:53:07Z

/lgtm

felixwang9817 · 2022-08-10T16:53:20Z

thanks @sfc-gh-madkins for the help here!

lsaiken · 2022-08-10T18:31:07Z

thanks @sfc-gh-madkins and @felixwang9817!

Signed-off-by: Lisa Aiken <[email protected]> Signed-off-by: Lisa Aiken <[email protected]> Signed-off-by: Francisco Javier Arceo <[email protected]>

# [0.24.0](v0.23.0...v0.24.0) (2022-08-25) ### Bug Fixes * Check if on_demand_feature_views is an empty list rather than None for snowflake provider ([#3046](#3046)) ([9b05e65](9b05e65)) * FeatureStore.apply applies BatchFeatureView correctly ([#3098](#3098)) ([41be511](41be511)) * Fix Feast Java inconsistency with int64 serialization vs python ([#3031](#3031)) ([4bba787](4bba787)) * Fix feature service inference logic ([#3089](#3089)) ([4310ed7](4310ed7)) * Fix field mapping logic during feature inference ([#3067](#3067)) ([cdfa761](cdfa761)) * Fix incorrect on demand feature view diffing and improve Java tests ([#3074](#3074)) ([0702310](0702310)) * Fix Java helm charts to work with refactored logic. Fix FTS image ([#3105](#3105)) ([2b493e0](2b493e0)) * Fix on demand feature view output in feast plan + Web UI crash ([#3057](#3057)) ([bfae6ac](bfae6ac)) * Fix release workflow to release 0.24.0 ([#3138](#3138)) ([a69aaae](a69aaae)) * Fix Spark offline store type conversion to arrow ([#3071](#3071)) ([b26566d](b26566d)) * Fixing Web UI, which fails for the SQL registry ([#3028](#3028)) ([64603b6](64603b6)) * Force Snowflake Session to Timezone UTC ([#3083](#3083)) ([9f221e6](9f221e6)) * Make infer dummy entity join key idempotent ([#3115](#3115)) ([1f5b1e0](1f5b1e0)) * More explicit error messages ([#2708](#2708)) ([e4d7afd](e4d7afd)) * Parse inline data sources ([#3036](#3036)) ([c7ba370](c7ba370)) * Prevent overwriting existing file during `persist` ([#3088](#3088)) ([69af21f](69af21f)) * Register BatchFeatureView in feature repos correctly ([#3092](#3092)) ([b8e39ea](b8e39ea)) * Return an empty infra object from sql registry when it doesn't exist ([#3022](#3022)) ([8ba87d1](8ba87d1)) * Teardown tables for Snowflake Materialization testing ([#3106](#3106)) ([0a0c974](0a0c974)) * UI error when saved dataset is present in registry. ([#3124](#3124)) ([83cf753](83cf753)) * Update sql.py ([#3096](#3096)) ([2646a86](2646a86)) * Updated snowflake template ([#3130](#3130)) ([f0594e1](f0594e1)) ### Features * Add authentication option for snowflake connector ([#3039](#3039)) ([74c75f1](74c75f1)) * Add Cassandra/AstraDB online store contribution ([#2873](#2873)) ([feb6cb8](feb6cb8)) * Add Snowflake materialization engine ([#2948](#2948)) ([f3b522b](f3b522b)) * Adding saved dataset capabilities for Postgres ([#3070](#3070)) ([d3253c3](d3253c3)) * Allow passing repo config path via flag ([#3077](#3077)) ([0d2d951](0d2d951)) * Contrib azure provider with synapse/mssql offline store and Azure registry store ([#3072](#3072)) ([9f7e557](9f7e557)) * Custom Docker image for Bytewax batch materialization ([#3099](#3099)) ([cdd1b07](cdd1b07)) * Feast AWS Athena offline store (again) ([#3044](#3044)) ([989ce08](989ce08)) * Implement spark offline store `offline_write_batch` method ([#3076](#3076)) ([5b0cc87](5b0cc87)) * Initial Bytewax materialization engine ([#2974](#2974)) ([55c61f9](55c61f9)) * Refactor feature server helm charts to allow passing feature_store.yaml in environment variables ([#3113](#3113)) ([85ee789](85ee789))

feast-ci-bot added the size/S label Aug 8, 2022

felixwang9817 approved these changes Aug 8, 2022

View reviewed changes

feast-ci-bot assigned felixwang9817 Aug 8, 2022

feast-ci-bot added lgtm approved labels Aug 8, 2022

lsaiken changed the title ~~Add authentication option for snowflake connector~~ feat: Add authentication option for snowflake connector Aug 8, 2022

sfc-gh-madkins reviewed Aug 9, 2022

View reviewed changes

feast-ci-bot removed the lgtm label Aug 9, 2022

sfc-gh-madkins reviewed Aug 9, 2022

View reviewed changes

feat: Add authentication option for snowflake connector

9289ff0

Signed-off-by: Lisa Aiken <[email protected]>

sfc-gh-madkins added the ok-to-test label Aug 10, 2022

feast-ci-bot added the lgtm label Aug 10, 2022

feast-ci-bot merged commit 74c75f1 into feast-dev:master Aug 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add authentication option for snowflake connector #3039

feat: Add authentication option for snowflake connector #3039

lsaiken commented Aug 8, 2022

felixwang9817 left a comment

feast-ci-bot commented Aug 8, 2022

felixwang9817 commented Aug 8, 2022

codecov-commenter commented Aug 8, 2022 •

edited

Loading

sfc-gh-madkins commented Aug 9, 2022

sfc-gh-madkins Aug 9, 2022

lsaiken Aug 9, 2022 •

edited

Loading

sfc-gh-madkins Aug 9, 2022

felixwang9817 commented Aug 10, 2022

felixwang9817 commented Aug 10, 2022

lsaiken commented Aug 10, 2022

feat: Add authentication option for snowflake connector #3039

feat: Add authentication option for snowflake connector #3039

Conversation

lsaiken commented Aug 8, 2022

felixwang9817 left a comment

Choose a reason for hiding this comment

feast-ci-bot commented Aug 8, 2022

felixwang9817 commented Aug 8, 2022

codecov-commenter commented Aug 8, 2022 • edited Loading

Codecov Report

sfc-gh-madkins commented Aug 9, 2022

sfc-gh-madkins Aug 9, 2022

Choose a reason for hiding this comment

lsaiken Aug 9, 2022 • edited Loading

Choose a reason for hiding this comment

sfc-gh-madkins Aug 9, 2022

Choose a reason for hiding this comment

felixwang9817 commented Aug 10, 2022

felixwang9817 commented Aug 10, 2022

lsaiken commented Aug 10, 2022

codecov-commenter commented Aug 8, 2022 •

edited

Loading

lsaiken Aug 9, 2022 •

edited

Loading