Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filter: add conditions to access control filter #7716

Merged
merged 43 commits into from
Aug 19, 2019
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
3d4f7aa
Initial ABAC filter
kyessenov Jul 24, 2019
beb197c
typos
kyessenov Jul 25, 2019
e914f46
Merge remote-tracking branch 'upstream/master' into abac_filter_needs…
kyessenov Jul 26, 2019
ec74571
review
kyessenov Jul 26, 2019
97efd6d
spelling
kyessenov Jul 26, 2019
f3668b2
undo watermark
kyessenov Jul 26, 2019
dca0933
Merge remote-tracking branch 'upstream/master' into abac_filter_needs…
kyessenov Jul 29, 2019
8e80999
review feedback
kyessenov Jul 30, 2019
970b361
review feedback
kyessenov Jul 30, 2019
ebf4c4a
review feedback
kyessenov Jul 30, 2019
d1fd462
add code owners
kyessenov Jul 30, 2019
1235c7d
Merge remote-tracking branch 'upstream/master' into abac_filter_needs…
kyessenov Jul 30, 2019
41220da
update cel-cpp
kyessenov Jul 30, 2019
982e3fd
combine engines
kyessenov Jul 30, 2019
c31b4a7
make arena explicit
kyessenov Jul 30, 2019
e309dd8
more attributes
kyessenov Jul 30, 2019
6cdbe8e
build fix
kyessenov Jul 31, 2019
f934dda
refactor
kyessenov Jul 31, 2019
bf99900
fix unit tests
kyessenov Jul 31, 2019
6861b98
Merge remote-tracking branch 'upstream/master' into abac_filter_needs…
kyessenov Jul 31, 2019
b27790a
unit tests
kyessenov Jul 31, 2019
9393ca9
fix api
kyessenov Jul 31, 2019
5d44fea
add metadata test
kyessenov Jul 31, 2019
784a970
release note
kyessenov Aug 1, 2019
c081695
typo
kyessenov Aug 1, 2019
9f812b5
Merge remote-tracking branch 'upstream/master' into abac_filter_needs…
kyessenov Aug 1, 2019
184fe6b
add constant folding; use unique_ptr to avoid copying the engine
kyessenov Aug 2, 2019
811fba6
merge fix
kyessenov Aug 2, 2019
64c9201
merge fix
kyessenov Aug 5, 2019
684d473
apply a patch for gcc
kyessenov Aug 6, 2019
08fe702
more specific patch
kyessenov Aug 6, 2019
580a79f
fix the macro specializer
kyessenov Aug 7, 2019
8f72a50
oops, reverse the patch
kyessenov Aug 8, 2019
a2a9a7f
Merge remote-tracking branch 'upstream/master' into abac_filter_needs…
kyessenov Aug 8, 2019
f86eadf
update re2 import
kyessenov Aug 8, 2019
b9c755c
Merge remote-tracking branch 'upstream/master' into abac_filter_needs…
kyessenov Aug 9, 2019
532ed67
align with ext_authz by using source and destination
kyessenov Aug 9, 2019
c1f1890
bump up coverage
kyessenov Aug 9, 2019
aed67b6
Merge remote-tracking branch 'upstream/master' into abac_filter_needs…
kyessenov Aug 15, 2019
76b6788
update cel-cpp
kyessenov Aug 15, 2019
a0daefa
Merge remote-tracking branch 'upstream/master' into abac_filter_needs…
kyessenov Aug 15, 2019
753b352
merge fix
kyessenov Aug 15, 2019
8df3414
bump up coverage
kyessenov Aug 15, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,5 @@ extensions/filters/common/original_src @snowp @klarose
/*/extensions/filters/http/adaptive_concurrency @tonya11en @mattklein123
# http inspector
/*/extensions/filters/listener/http_inspector @crazyxy @PiotrSikora @lizan
# attribute context
/*/extensions/filters/common/expr @kyessenov @yangminzhu
7 changes: 4 additions & 3 deletions api/bazel/api_build_system.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,13 @@ def _LibrarySuffix(library_name, suffix):
# TODO(htuch): Convert this to native py_proto_library once
# https://github.com/bazelbuild/bazel/issues/3935 and/or
# https://github.com/bazelbuild/bazel/issues/2626 are resolved.
def api_py_proto_library(name, srcs = [], deps = [], has_services = 0):
def api_py_proto_library(name, srcs = [], deps = [], external_py_proto_deps = [], has_services = 0):
_py_proto_library(
name = _Suffix(name, _PY_SUFFIX),
srcs = srcs,
default_runtime = "@com_google_protobuf//:protobuf_python",
protoc = "@com_google_protobuf//:protoc",
deps = [_LibrarySuffix(d, _PY_SUFFIX) for d in deps] + [
deps = [_LibrarySuffix(d, _PY_SUFFIX) for d in deps] + external_py_proto_deps + [
"@com_envoyproxy_protoc_gen_validate//validate:validate_py",
"@com_google_googleapis//google/rpc:status_py_proto",
"@com_google_googleapis//google/api:annotations_py_proto",
Expand Down Expand Up @@ -116,6 +116,7 @@ def api_proto_library(
deps = [],
external_proto_deps = [],
external_cc_proto_deps = [],
external_py_proto_deps = [],
has_services = 0,
linkstatic = None,
require_py = 1):
Expand Down Expand Up @@ -152,7 +153,7 @@ def api_proto_library(
)
py_export_suffixes = []
if (require_py == 1):
api_py_proto_library(name, srcs, deps, has_services)
api_py_proto_library(name, srcs, deps, external_py_proto_deps, has_services)
py_export_suffixes = ["_py", "_py_genproto"]

# Allow unlimited visibility for consumers
Expand Down
10 changes: 10 additions & 0 deletions api/envoy/config/rbac/v2/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ load("@envoy_api//bazel:api_build_system.bzl", "api_go_proto_library", "api_prot
api_proto_library_internal(
name = "rbac",
srcs = ["rbac.proto"],
external_cc_proto_deps = [
"@com_google_googleapis//google/api/expr/v1alpha1:syntax_cc_proto",
],
external_proto_deps = [
"@com_google_googleapis//google/api/expr/v1alpha1:syntax_proto",
],
external_py_proto_deps = [
"@com_google_googleapis//google/api/expr/v1alpha1:syntax_py_proto",
],
visibility = ["//visibility:public"],
deps = [
"//envoy/api/v2/core:address",
Expand All @@ -22,5 +31,6 @@ api_go_proto_library(
"//envoy/api/v2/route:route_go_proto",
"//envoy/type/matcher:metadata_go_proto",
"//envoy/type/matcher:string_go_proto",
"@com_google_googleapis//google/api/expr/v1alpha1:cel_go_proto",
],
)
8 changes: 7 additions & 1 deletion api/envoy/config/rbac/v2/rbac.proto
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ import "envoy/api/v2/route/route.proto";
import "envoy/type/matcher/metadata.proto";
import "envoy/type/matcher/string.proto";

import "google/api/expr/v1alpha1/syntax.proto";

package envoy.config.rbac.v2;

option java_outer_classname = "RbacProto";
Expand Down Expand Up @@ -81,7 +83,7 @@ message RBAC {

// Policy specifies a role and the principals that are assigned/denied the role. A policy matches if
// and only if at least one of its permissions match the action taking place AND at least one of its
// principals match the downstream.
// principals match the downstream AND the condition is true if specified.
message Policy {
// Required. The set of permissions that define a role. Each permission is matched with OR
// semantics. To match all actions for this policy, a single Permission with the `any` field set
Expand All @@ -92,6 +94,10 @@ message Policy {
// principal is matched with OR semantics. To match all downstreams for this policy, a single
// Principal with the `any` field set to true should be used.
repeated Principal principals = 2 [(validate.rules).repeated .min_items = 1];

// An optional symbolic expression specifying an access control condition.
// The condition is combined with AND semantics.
google.api.expr.v1alpha1.Expr condition = 3;
}

// Permission defines an action (or actions) that a principal can take.
Expand Down
67 changes: 67 additions & 0 deletions bazel/com_google_cel_cpp.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
diff --git a/eval/public/cel_function_adapter.h b/eval/public/cel_function_adapter.h
index d99239c..4dc8cae 100644
--- a/eval/public/cel_function_adapter.h
+++ b/eval/public/cel_function_adapter.h
@@ -118,6 +118,34 @@ class FunctionAdapter : public CelFunction {
return registry->Register(std::move(status.ValueOrDie()));
}

+#if !defined(__clang_major_version__) || (defined(__clang_major_version__) && __clang_major_version__ < 8) || defined(__APPLE__)
+ inline cel_base::Status RunWrap(std::function<ReturnType()> func,
+ const absl::Span<const CelValue> argset,
+ ::google::protobuf::Arena* arena, CelValue* result,
+ int arg_index) const {
+ return CreateReturnValue(func(), arena, result);
+ }
+
+ template <typename Arg, typename... Args>
+ inline cel_base::Status RunWrap(std::function<ReturnType(Arg, Args...)> func,
+ const absl::Span<const CelValue> argset,
+ ::google::protobuf::Arena* arena, CelValue* result,
+ int arg_index) const {
+ Arg argument;
+ if (!ConvertFromValue(argset[arg_index], &argument)) {
+ return cel_base::Status(cel_base::StatusCode::kInvalidArgument,
+ "Type conversion failed");
+ }
+
+ std::function<ReturnType(Args...)> wrapped_func =
+ [func, argument](Args... args) -> ReturnType {
+ return func(argument, args...);
+ };
+
+ return RunWrap(std::move(wrapped_func), argset, arena, result,
+ arg_index + 1);
+ }
+#else
template <int arg_index>
inline cel_base::Status RunWrap(absl::Span<const CelValue> arguments,
std::tuple<::google::protobuf::Arena*, Arguments...> input,
@@ -137,6 +165,7 @@ class FunctionAdapter : public CelFunction {
::google::protobuf::Arena* arena) const {
return CreateReturnValue(absl::apply(handler_, input), arena, result);
}
+#endif

::cel_base::Status Evaluate(absl::Span<const CelValue> arguments,
CelValue* result,
@@ -146,9 +175,19 @@ class FunctionAdapter : public CelFunction {
"Argument number mismatch");
}

+
+#if !defined(__clang_major_version__) || (defined(__clang_major_version__) && __clang_major_version__ < 8) || defined(__APPLE__)
+ const auto* handler = &handler_;
+ std::function<ReturnType(Arguments...)> wrapped_handler =
+ [handler, arena](Arguments... args) -> ReturnType {
+ return (*handler)(arena, args...);
+ };
+ return RunWrap(std::move(wrapped_handler), arguments, arena, result, 0);
+#else
std::tuple<::google::protobuf::Arena*, Arguments...> input;
std::get<0>(input) = arena;
return RunWrap<0>(arguments, input, result, arena);
+#endif
}

private:
12 changes: 12 additions & 0 deletions bazel/repositories.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,8 @@ def envoy_dependencies(skip_targets = []):
_com_lightstep_tracer_cpp()
_io_opentracing_cpp()
_net_zlib()
_repository_impl("com_googlesource_code_re2")
_com_google_cel_cpp()
_repository_impl("bazel_toolchains")

_python_deps()
Expand Down Expand Up @@ -315,6 +317,16 @@ def _net_zlib():
actual = "@envoy//bazel/foreign_cc:zlib",
)

def _com_google_cel_cpp():
location = REPOSITORY_LOCATIONS["com_google_cel_cpp"]
http_archive(
name = "com_google_cel_cpp",
# TODO(kyessenov): requires C++17 partial template specialization available in clang-8 but not gcc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not keen to keep this in Envoy repo, as this seems a general open-source issue of cel-cpp. I understand that this is not an issue in google3 but as a open-sourced project it is general enough to upstreamed. I'm fine if other (possibly Google) maintainer is up for this. @htuch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's be ideal to use absl::bind_front but absl keeps delaying the release of important libraries. I'm happy to save this workaround in google3 once bind_front comes out.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, can we upstream this patch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not happy about the patch since it's almost 2x slower than the fast path. Can we postpone it till we get bind_front in absl (any day now TM)? I'm going to continue to maintain the integration, and it should be easier to reconcile the differences.

patch_args = ["-p1"],
patches = ["@envoy//bazel:com_google_cel_cpp.patch"],
**location
)

def _com_github_nghttp2_nghttp2():
location = REPOSITORY_LOCATIONS["com_github_nghttp2_nghttp2"]
http_archive(
Expand Down
10 changes: 10 additions & 0 deletions bazel/repository_locations.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -248,4 +248,14 @@ REPOSITORY_LOCATIONS = dict(
sha256 = "fcdebf54c89d839ffa7eefae166c8e4b551c765559db13ff15bff98047f344fb",
urls = ["https://storage.googleapis.com/quiche-envoy-integration/2a930469533c3b541443488a629fe25cd8ff53d0.tar.gz"],
),
com_google_cel_cpp = dict(
sha256 = "82186be314a2a9c6b9eb2477f15c4f3704b5ac9b4b26bf65694e231a48f4c1f1",
strip_prefix = "cel-cpp-71fb0562a59c05239f92025d3e7beb63169c3923",
urls = ["https://github.com/google/cel-cpp/archive/71fb0562a59c05239f92025d3e7beb63169c3923.tar.gz"],
),
com_googlesource_code_re2 = dict(
sha256 = "f31db9cd224d018a7e4fe88ef84aaa874b0b3ed91d4d98ee5a1531101d3fdc64",
strip_prefix = "re2-87e2ad45e7b18738e1551474f7ee5886ff572059",
urls = ["https://github.com/google/re2/archive/87e2ad45e7b18738e1551474f7ee5886ff572059.tar.gz"],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kyessenov qq: does cel-cpp rely on specific commit of re2? Asking because it might conflict with #7878, or latest release (2019-08-01) is fine? cc @mattklein123

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be no difference between which version is used. I think I chose the latest version which I started this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK that's fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to ask the same question. I'll switch this back a release version of re2 on a subsequent dependency PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will just fix this when I merge master.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Happy to help if necessary. Google3 doesn't really have versions for its repositories, and the upstream cel-cpp is continuously tested against head.

),
)
1 change: 1 addition & 0 deletions docs/root/intro/version_history.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ Version history
* tls: added verification of IP address SAN fields in certificates against configured SANs in the
certificate validation context.
* upstream: added network filter chains to upstream connections, see :ref:`filters<envoy_api_field_Cluster.filters>`.
* rbac: added conditions to the policy, see :ref:`condition <envoy_api_field_config.rbac.v2.Policy.condition>`.

1.11.0 (July 11, 2019)
======================
Expand Down
34 changes: 34 additions & 0 deletions source/extensions/filters/common/expr/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
licenses(["notice"]) # Apache 2

load(
"//bazel:envoy_build_system.bzl",
"envoy_cc_library",
"envoy_package",
)

envoy_package()

envoy_cc_library(
name = "evaluator_lib",
srcs = ["evaluator.cc"],
hdrs = ["evaluator.h"],
deps = [
":context_lib",
"//source/common/http:utility_lib",
"//source/common/protobuf",
"@com_google_cel_cpp//eval/public:builtin_func_registrar",
"@com_google_cel_cpp//eval/public:cel_expr_builder_factory",
"@com_google_cel_cpp//eval/public:cel_expression",
"@com_google_cel_cpp//eval/public:cel_value",
],
)

envoy_cc_library(
name = "context_lib",
srcs = ["context.cc"],
hdrs = ["context.h"],
deps = [
"//source/common/http:utility_lib",
"@com_google_cel_cpp//eval/public:cel_value",
],
)
Loading