Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Feast Security Model (aka RBAC) #4380

Merged
merged 170 commits into from
Aug 21, 2024
Merged
Changes from 1 commit
Commits
Show all changes
170 commits
Select commit Hold shift + click to select a range
4390a41
initial commit
dmartinol Jun 21, 2024
a4859b7
fixed linting issues (but 1)
dmartinol Jun 21, 2024
1f6d6f3
deleted AuthzedResource and moved types to the Permission class
dmartinol Jun 21, 2024
a2fa5de
using pytest.mark.parametrize tests
dmartinol Jun 24, 2024
d906554
moved decorator to decorator module
dmartinol Jun 24, 2024
21d29a8
parametrized decision tests
dmartinol Jun 24, 2024
f477add
Added matcher and action modules. Added global assert_permissions fun…
dmartinol Jun 24, 2024
30ee844
fixed linting error
dmartinol Jun 24, 2024
183d0c4
Managing with_subclasses flag and overriding it in case it's an abstr…
dmartinol Jun 24, 2024
e4a0f9f
Permission includes a single Policy
dmartinol Jun 25, 2024
b5c5af1
completed docstrings for permissions package
dmartinol Jun 26, 2024
dd15dd9
fixed inter issues
dmartinol Jun 26, 2024
7d7a787
Changed roles matching rule from "all" to "any"
dmartinol Jun 26, 2024
bb857e2
Introducing permission framework and authorization manager in user gu…
dmartinol Jun 27, 2024
abf384f
removed test code
dmartinol Jun 27, 2024
8fb5227
hiding sensitive data (false positive, anyway)
dmartinol Jun 27, 2024
ae01740
Added filter_only flag to assert_permissions and returning a list of …
dmartinol Jul 1, 2024
7b02d26
added the option to return the single resource, or None
dmartinol Jul 1, 2024
ad6765b
separate validating functions: assert_permission and filtered_resources
dmartinol Jul 1, 2024
b1fbdb6
Store and Manage permissions in the Registry
tmihalac Jun 27, 2024
a9d7a92
Applied review comments
tmihalac Jun 28, 2024
1534a8d
Store and Manage permissions in the Registry
tmihalac Jun 28, 2024
a21845d
Store and Manage permissions in the Registry
tmihalac Jun 28, 2024
e06a631
Store and Manage permissions in the Registry
tmihalac Jul 1, 2024
4fadb7b
Store and Manage permissions in the Registry
tmihalac Jun 27, 2024
7d17c78
Store and Manage permissions in the Registry
tmihalac Jun 28, 2024
ec079a6
Store and Manage permissions in the Registry
tmihalac Jun 28, 2024
aef0747
Store and Manage permissions in the Registry
tmihalac Jul 1, 2024
14ea4c2
replaced aggregated actions with aliases for QUERY and WRITE and ALL
dmartinol Jul 2, 2024
98eed44
Updated user guide
dmartinol Jul 2, 2024
78decaa
Updated enum in proto
dmartinol Jul 2, 2024
a02602d
Store and Manage permissions in the Registry
tmihalac Jul 2, 2024
6fba046
Store and Manage permissions in the Registry
tmihalac Jul 3, 2024
4408be5
Added permission assert check for registry server, offline server, on…
redhatHameed Jun 25, 2024
34c151c
Fix linter after rebase
redhatHameed Jul 5, 2024
aa0758c
CLI command "feast permissions list"
tmihalac Jul 8, 2024
fbe1bd5
CLI command "feast permissions list"
tmihalac Jul 8, 2024
87710cb
CLI command "feast permissions list"
tmihalac Jul 8, 2024
e9ff6e6
added the documents reference for permissions for online, offline, re…
redhatHameed Jul 5, 2024
679579e
Incorporating code review comments to parse the auth block from the f…
lokeshrangineni Jul 9, 2024
1085b99
definition and integration of auth manager in feast offline and onlin…
dmartinol Jul 8, 2024
7bfc945
typo
dmartinol Jul 9, 2024
d988846
duplicated if
dmartinol Jul 9, 2024
07211da
renamed functions with long name
dmartinol Jul 9, 2024
5386419
using User class instead of RoleManager (completely removed)
dmartinol Jul 9, 2024
0548aea
Feed SecurityManager with Registry instance to fetch the actual permi…
dmartinol Jul 9, 2024
a921f07
fixed linter
dmartinol Jul 9, 2024
f0b95c8
review comments
dmartinol Jul 9, 2024
f8a7140
fixed broken IT
dmartinol Jul 10, 2024
95eca10
Adding registry server (UT to be completed)
dmartinol Jul 10, 2024
8d42fcf
fix linter
dmartinol Jul 10, 2024
ef6d21f
passing auth manager type from config
dmartinol Jul 10, 2024
86a2e6b
used auth config to set auth manager type
redhatHameed Jul 9, 2024
9fc6db7
inject the user details
redhatHameed Jul 9, 2024
ea97997
created decorator function and applied to arrow function for injectin…
redhatHameed Jul 9, 2024
711374a
code review fixes including the unit test and integration test as sug…
redhatHameed Jul 10, 2024
6899f41
Implementation of oidc client authentication. (#40)
lokeshrangineni Jul 10, 2024
7b6561e
Client module-grpc
tmihalac Jul 10, 2024
bc086ef
Client module-grpc
tmihalac Jul 11, 2024
fb48f1a
Client module-grpc
tmihalac Jul 11, 2024
0158cfd
Client module-grpc
tmihalac Jul 11, 2024
2dbf6b1
Client module-grpc
tmihalac Jul 11, 2024
6ac6c01
Client module-grpc
tmihalac Jul 12, 2024
dc99e59
Client module-grpc
tmihalac Jul 12, 2024
aaeb7b7
Client module-grpc
tmihalac Jul 12, 2024
23ac8a6
added auth configuration for arrow flight client
redhatHameed Jul 11, 2024
c7e92a0
Client module-grpc
tmihalac Jul 12, 2024
77abd12
fix linter
dmartinol Jul 12, 2024
c6e9638
Propagating auth config to token parser in server init
dmartinol Jul 12, 2024
53d7d10
adding headers and client_secret to token request
dmartinol Jul 12, 2024
fcd7419
working E2E test of authenticated registy server
dmartinol Jul 12, 2024
f3e36d2
renamed test
dmartinol Jul 12, 2024
811dc83
fixed broken test
dmartinol Jul 12, 2024
7dca956
fix rebase issues
dmartinol Jul 12, 2024
d2a9f6c
fix rebase issues
dmartinol Jul 12, 2024
92198de
Adding the auth client documentations and unit testing for auth clien…
lokeshrangineni Jul 12, 2024
2cf1f00
Adding the auth client documentations and unit testing for auth clien…
lokeshrangineni Jul 12, 2024
74fe957
Incorporating code review comments.
lokeshrangineni Jul 15, 2024
3b43c04
Incorporating code review comments.
lokeshrangineni Jul 15, 2024
a83c5a6
Introducing permission framework and authorization manager in user gu…
dmartinol Jun 27, 2024
ce6413b
CLI command "feast permissions list"
tmihalac Jul 15, 2024
a400f8f
Client module-grpc
tmihalac Jul 15, 2024
dcec0cb
Fix auth tests with permissions
tmihalac Jul 22, 2024
db1cfd6
Fix auth tests with permissions
tmihalac Jul 24, 2024
93e6f4d
Fix auth tests with permissions
tmihalac Jul 24, 2024
a0a68ef
Fix auth tests with permissions
tmihalac Jul 24, 2024
b4e0e08
Moved the common fixtures to the root conftest.py or auth_permissions…
lokeshrangineni Jul 25, 2024
137fed1
added check and list-roles subcommands
dmartinol Jul 25, 2024
716e099
typo
dmartinol Jul 25, 2024
bacd4a1
added comment in cli_utils to remind the original function from which…
dmartinol Jul 26, 2024
d76e3c4
1) Updating the existing integration test with auth permissions confi…
lokeshrangineni Jul 26, 2024
5fff1ff
Moved the common fixtures to the root conftest.py or auth_permissions…
lokeshrangineni Jul 25, 2024
23d9a31
Adding missed dependency and regenerated the requirements files.
lokeshrangineni Jul 22, 2024
49fd90d
1) Updating the existing integration test with auth permissions confi…
lokeshrangineni Jul 26, 2024
6a96e70
1) Fixing an issue with the way getting markers after changing the fi…
lokeshrangineni Jul 28, 2024
9308731
Fixed bug in GetPermission API
dmartinol Jul 30, 2024
6538db6
Permission CRUD test
dmartinol Jul 30, 2024
c9bdbbe
Added feast-rbac example
redhatHameed Jul 17, 2024
3ab7087
Added support to read the token from enviroment variable to run from …
redhatHameed Jul 17, 2024
b260756
Fix the header for arrow fligth
redhatHameed Jul 17, 2024
269cc33
fix the header issue
redhatHameed Jul 17, 2024
741010d
added permissions apply file
redhatHameed Jul 17, 2024
8c1993b
set the user in the grpc server
redhatHameed Jul 18, 2024
6558760
added roles and updated permission with all roles
redhatHameed Jul 19, 2024
cb4add7
updated chart to include the service account
redhatHameed Jul 19, 2024
bf4ac90
created client example with roles and updated installation/cleanup sc…
redhatHameed Jul 22, 2024
d7defc1
rebased with master
redhatHameed Jul 31, 2024
190e609
Moved the common fixtures to the root conftest.py or auth_permissions…
lokeshrangineni Jul 25, 2024
542f389
Fixed DecisionStrategy not persisted
tmihalac Jul 27, 2024
bdd4fd6
Fixed DecisionStrategy not persisted
tmihalac Jul 31, 2024
571d9c6
Fixed DecisionStrategy not persisted
tmihalac Aug 1, 2024
de34a11
Revert "Fix decision strategy not saved"
dmartinol Aug 2, 2024
900bc86
Dropped global decision strategy
dmartinol Aug 2, 2024
065c99c
updated rbac demo example
redhatHameed Aug 1, 2024
444ae71
Adding permissions directly instead of from the common place for the …
lokeshrangineni Aug 2, 2024
86ec133
Initial Draft version to the tests with remote offline server with OI…
lokeshrangineni Aug 2, 2024
1fcb89d
Abstracting the specific code for Offline Permissions by creating new…
lokeshrangineni Aug 2, 2024
787105b
Formatting the python files using make format-python.
lokeshrangineni Aug 2, 2024
2f169df
Separated the permissions for online, offline and registry servers. m…
lokeshrangineni Aug 2, 2024
af35739
Separated the permissions for online, offline and registry servers. m…
lokeshrangineni Aug 2, 2024
0333c56
Created the grpc client auth header interceptor and removed the manua…
lokeshrangineni Aug 2, 2024
9c42cfb
Created the grpc client auth header interceptor and removed the manua…
lokeshrangineni Aug 2, 2024
b51e58c
fix: java to proto failing
tmihalac Aug 6, 2024
bc1f30c
CLI command "feast permissions list"
tmihalac Jul 8, 2024
63fa1e8
Moved the common fixtures to the root conftest.py or auth_permissions…
lokeshrangineni Jul 25, 2024
8c17cf8
fix: java to proto failing
tmihalac Aug 6, 2024
035f197
Adding the extra writer permission to fix the integration test issue …
lokeshrangineni Aug 6, 2024
f98b8d1
Try to fix java integration test - ModuleNotFoundError: No module nam…
tmihalac Aug 6, 2024
7c7ec37
fix java integration test - ModuleNotFoundError: No module named 'jwt'
tmihalac Aug 6, 2024
414cd48
fix java integration test - ModuleNotFoundError: No module named 'kub…
tmihalac Aug 6, 2024
7631099
Adding missing permissions for offline store test cases - classes Fil…
lokeshrangineni Aug 6, 2024
54a8392
Updating the offline integration test permissions.
lokeshrangineni Aug 7, 2024
366e288
updated test.py file for rbac-example
redhatHameed Aug 7, 2024
cdb19fa
fix the DeleteFeatureView function to handle stream feature view type
redhatHameed Aug 7, 2024
3b06316
Updating permissions of the integration test cases to address code re…
lokeshrangineni Aug 8, 2024
61f5ef1
Incorporating the code review comments from Francisco on upstream PR.
lokeshrangineni Aug 8, 2024
5b67a6c
Update docs/getting-started/concepts/permission.md
dmartinol Aug 8, 2024
ab454be
Update docs/getting-started/concepts/permission.md
dmartinol Aug 8, 2024
8cfe18f
Update docs/getting-started/concepts/permission.md
dmartinol Aug 8, 2024
7e8bfaa
Small fixes (#71)
dmartinol Aug 8, 2024
6418549
commented/removed oidc tests to verify integration test
redhatHameed Aug 8, 2024
f79b6ec
Enabling the keycloak related integration tests and also initializing…
lokeshrangineni Aug 8, 2024
bebd292
Making number of workers back to 8 and enabled the test_remote_online…
lokeshrangineni Aug 8, 2024
e6b5e4c
Making number of workers to 4.
lokeshrangineni Aug 8, 2024
33bb445
Incorporating the code review comments from Tornike to use @pytest.ma…
lokeshrangineni Aug 9, 2024
12895c0
Reverting number of workers from 8 to 4.
lokeshrangineni Aug 9, 2024
9951489
Reverting number of workers from 8 to 4. Reverting the marker @pytest…
lokeshrangineni Aug 9, 2024
71e4044
Reverting number of workers from 8 to 4 for make target test-python-i…
lokeshrangineni Aug 9, 2024
fd1243b
Added the arrow flight interceptor to inject the auth header. (#68)
lokeshrangineni Aug 11, 2024
807c0f5
removed with_subclasses option (it's the default and unique behavior)
dmartinol Aug 12, 2024
a8abd21
a full, minimal, reproducible example of the RBAC feature
dmartinol Aug 9, 2024
1183fbd
Add missing required_tags to permission object and cli info
tmihalac Aug 13, 2024
063a87f
Fixed the registry apply function assertation
redhatHameed Aug 12, 2024
4922b8b
removed the examples
redhatHameed Aug 14, 2024
7c9389d
Integrated comment
dmartinol Aug 14, 2024
3ff811e
removed the firebase depdency and fix the doc conflicts
redhatHameed Aug 14, 2024
0498d3f
Introducing permission framework and authorization manager in user gu…
dmartinol Jun 27, 2024
23c829f
Permission resources miss the created_timestamp and last_updated_time…
tmihalac Aug 14, 2024
6c5e2c7
remove error incase if user has no roles assinged incase unthorized user
redhatHameed Aug 16, 2024
8409014
renamed READ action to DESCRIBE
dmartinol Aug 19, 2024
fdf331f
Specified authorization manager and authorization configuration
dmartinol Aug 19, 2024
3a4f122
fix the linter and remove subclass from doc
redhatHameed Aug 19, 2024
acf2190
addressed the pr reivew comments
redhatHameed Aug 20, 2024
136719e
Incorporating code review comment and this file is not needed.
lokeshrangineni Aug 19, 2024
9e3efe2
Addressed the review comments on the PR
redhatHameed Aug 20, 2024
5534044
Reducing the markers from 8 to 4 to see if it fixes the issues with m…
lokeshrangineni Aug 20, 2024
09893af
addresses feedback on rbac doc
redhatHameed Aug 20, 2024
2581335
rename action name from QUERY to READ
redhatHameed Aug 20, 2024
f04150a
Merge pull request #84 from redhatHameed/query-to-read
redhatHameed Aug 20, 2024
3f1cd9c
fix the doc to replace query with read
redhatHameed Aug 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
used auth config to set auth manager type
Signed-off-by: Abdul Hameed <ahameed@redhat.com>
redhatHameed committed Aug 19, 2024
commit 86a2e6b32c33ae2b07256d5d426d221dbf7b12d5
3 changes: 1 addition & 2 deletions sdk/python/feast/feature_server.py
Original file line number Diff line number Diff line change
@@ -25,7 +25,6 @@
from feast.permissions.server.rest import inject_user_details
from feast.permissions.server.utils import (
ServerType,
auth_manager_type_from_env,
init_auth_manager,
init_security_manager,
)
@@ -346,7 +345,7 @@ def start_server(
monitoring_thread.start()

# TODO RBAC remove and use the auth section of the feature store config instead
auth_manager_type = auth_manager_type_from_env()
auth_manager_type = store.config.auth_config.type
init_security_manager(auth_manager_type=auth_manager_type, fs=store)
init_auth_manager(
auth_manager_type=auth_manager_type,
8 changes: 4 additions & 4 deletions sdk/python/feast/offline_server.py
Original file line number Diff line number Diff line change
@@ -20,7 +20,6 @@
)
from feast.permissions.server.utils import (
ServerType,
auth_manager_type_from_env,
init_auth_manager,
init_security_manager,
)
@@ -33,7 +32,9 @@
class OfflineServer(fl.FlightServerBase):
def __init__(self, store: FeatureStore, location: str, **kwargs):
super(OfflineServer, self).__init__(
location, middleware=arrowflight_middleware(), **kwargs
location,
middleware=arrowflight_middleware(store.config.auth_config.type),
**kwargs,
)
self._location = location
# A dictionary of configured flights, e.g. API calls received and not yet served
@@ -448,8 +449,7 @@ def start_server(
host: str,
port: int,
):
# TODO RBAC remove and use the auth section of the feature store config instead
auth_manager_type = auth_manager_type_from_env()
auth_manager_type = store.config.auth_config.type
init_security_manager(auth_manager_type=auth_manager_type, fs=store)
init_auth_manager(
auth_manager_type=auth_manager_type,
8 changes: 4 additions & 4 deletions sdk/python/feast/permissions/server/arrow.py
Original file line number Diff line number Diff line change
@@ -15,24 +15,24 @@
from feast.permissions.security_manager import get_security_manager
from feast.permissions.server.utils import (
AuthManagerType,
auth_manager_type_from_env,
)
from feast.permissions.user import User

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)


def arrowflight_middleware() -> Optional[dict[str, fl.ServerMiddlewareFactory]]:
def arrowflight_middleware(
auth_manager_type: str,
) -> Optional[dict[str, fl.ServerMiddlewareFactory]]:
"""
A dictionary with the configured middlewares to support extracting the user details when the authorization manager is defined.
The authorization middleware key is `auth`.

Returns:
dict[str, fl.ServerMiddlewareFactory]: Optional dictionary of middlewares. If the authorization type is set to `NONE`, it returns `None`.
"""
# TODO RBAC remove and use the auth section of the feature store config instead
auth_manager_type = auth_manager_type_from_env()

if auth_manager_type == AuthManagerType.NONE:
return None