-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Kibana Core] Allow configuration to specify an array of "disabled routes" for server AND UI #157266
Comments
Pinging @elastic/kibana-core (Team:Core) |
Could you to elaborate a bit on why you think it would be beneficial to have this configuration driven? Ihmo, it would introduce a split between where the routes are registered (plugin code) and where routes can be disabled (core's http service / configuration).
Can you describe a normal development cycle where this would have any value? Do you foresee scenarios where it would make sense to be able to 'disable' such routes without touching the code.
Could you elaborate how this would be an issue? Configuration-driven behavior only makes sense if we're foreseeing the need to be able to change said behavior without delivering a new version. Are you planning / do you need to be able to disable some routes outside of a serverless delivery cycle / version? If the issue is about the verbosity of wrapping each route definition with some kind of E.g router.post(
{
path: '/someApi',
options: {
hiddenOnServerless: true, // <-- this
},
validate: false
},
myHandler,
); I'm fine having a way to automate this more easily, but I still strongly think that this the information should be held by the route definition, and not by some arbitrary, decoupled, cross-plugin configuration property. |
cc @rudolf @jloleysens curious to see your thoughts on that too |
For serverless, we have apps inside the infra plugin: some of the routes should continue to work, but others should not (mostly UI side). We'll remove those from the navigation but a customer could type the URL in and the page would load, unless we write code to read from config to disable those discrete parts of the plugin. I expect writing that kind of code inside the plugin will be fragile, especially as the scope of what needs to be hidden in that environment will be changing over time. Similarly in serverless, for ML we want to keep a few of the APIs and UI pages available, but certain API endpoints should not be available. Here, it's more important, because a user who interacts with those APIs could start expensive jobs that could be problematic in that environment. I do understand your concern about how loosely related the two things would be. I think the idea of tagging routes, and then allowing some config-driven disabling by tag could also be an option? My understanding is that we want to avoid coupling the idea of "serverless" to the code itself, so |
Note that it could also be performed via http hooks E.g kibana/packages/core/http/core-http-server/src/http_contract.ts Lines 260 to 270 in 885e80a
it could be a good middleground: definition of the list of exclusions is not based on the route definitions, but still managed by plugin code. |
I wonder if we could use feature capabilities. By tagging an API endpoint the security plugin would automatically restrict access to it. I think this would require explicitly giving users access to these API endpoints in non-serverless. |
I was hoping we'd be able to use capabilities for this, in ML we have the access of every route (146 of them) controlled by capability tags. |
I agree that using capabilities for this could be a a good way to reuse the mechanisms already in place.
This is the part that bothers me. Ideally we would have a mechanism to 'add' those capabilities to everyone when not on serverless, or at least a system that doesn't imply adding specific permissions to our users on non-serverless (or more like requiring our customers to add specific permissions to their users) |
cc @elastic/kibana-security given capabilities are kinda also owned by then. Would be happy to have your opinion on that one. |
I still struggle to see how this is an authorization problem. We're not saying that certain users should/shouldn't call these APIs. Rather, we're saying that nobody, regardless of their privileges, should be able to call them. That's not an authorization problem, IMO. We're shipping code that we don't want anyone to invoke. This is a byproduct of our current architecture, and ideally not something we'd do given the option.
Superusers, by definition, can do anything. Even if we tag an API endpoint to restrict access, this would not forbid a superuser from calling these endpoints. I suspect that's not desired behavior. I'm not opposed to leveraging capabilities to solve this problem, but I don't believe that the |
Fully agree with @legrego here - disabling certain routes for serverless is a product decision and not an authorisation concern.
@jasonrhodes Can you elaborate a bit on this point? What's the concern with making code changes for serverless? |
Ideally, plugins shouldn't be aware of the environment Kibana's running on. However, core introduced a Core doesn't know anything about route handling in plugins and plugins will need to handle their own logic for disabling routes depending on the env. Both config and a route decorator are ways to solve this. However, disabling/enabling routes has a higher chance of changing, hence we lean towards a route decorator as a way plugins can handle conditional logic. |
Larry's right, this isn't an authorization issue. If we have a subset of routes that we know we don't want to be available anywhere in serverless, it feels wrong to keep them in place and then rely on privileges prevent access to them (plus, there's the superuser concern already mentioned). I don't feel strongly about which approach we go with here -- a route option, tags, or configuration -- all have their pros and cons. But I don't think we should let our guiding principle of "avoid introducing Concretely, there are three specific use cases we need to think about:
Use case (2) could be solved by registering such routes from a Serverless-only plugin. Use cases (1) and (3) are the scenarios I think we should be trying to solve for here. I've had multiple teams bring this up already, and I expect we'll continue to be asked these questions as folks begin prepping & versioning their APIs. |
++ to Luke's and Larry's comments. My main concern is blocking UI routes: Core owns the navigation to the app. However, navigating to sub-paths inside the app is out of core's control. I wonder how we're going to block those UI navigations. I'd like to focus on that use case for a minute because if we find that the only option is for the plugin to check either a Following that same thought: we may find situations where we link those paths inside our UI. So wrapping our code in TL;DR, I think plugins having config flags to enable/disable specific features is more future-proof. We can set the defaults for serverless inside the |
I think it would be helpful to have more examples of APIs plugins want to disable. |
This is a matter of perspective, I think. We're saying "some users should not have access to these APIs" and what we mean by "some users" are "users who are created in serverless context", for example. We've been instructed (rightly, I think) to avoid tight coupling between the idea of serverless/not-serverless as the actual driver of product changes, and rather to define semantic differences that describe real changes in the product and then specify those semantic changes in the serverless config. I don't have strong opinions on whether this is auth or not, so I think it's just a matter of whether a parallel set of configurable access tags makes more sense?
Some examples:
There are a lot of ways we can do this kind of change, I think?
I think [4] is the one we should avoid, based on my own understanding of how coupling these kinds of things can lead to bad situations as well as what I've heard others say about the serverless initiative in general. [3] works, but involves some amount of code thrashing for what's essentially per-environment configuration changes. That's why I was hoping for something more like [1] or [2], but if those have significant challenges, [3] will likely work okay. |
++ to defining them as semantic options vs. per- To be pedantically correct, environment configs tend to differentiate If we look at this as a new offering/product, options [1] and [2] may externally change the product we build, test, and ship. These alternatives may lead to unexpected errors that we didn't cover in our CI builds like API You may argue that option [3] could cause a similar outcome if the
I'm not entirely sure [1] and [2] would avoid that code trashing: while it's true it saves a bunch of "ifs" when registering the REST APIs, and we could look at ways of defining a common UI router to block access to app's internal routes (AFAIK, it doesn't exist today) to avoid those IFs in the UI, you'd still need to check the users' role/access to show/hide the buttons that link to the not-accessible UI. And there are other components that would still be registered unless the app checks if the user has enough permissions (like flyouts, top-bar items, and modals). To be clear, I'm not opposing to either, just giving my 2 cents on each one of those 😇 |
All fair points, @afharo. The broken buttons and links would just fail with 404s in the most basic case, which is at least better than arriving at a UI screen that has been "removed from the navigation" without being otherwise blocked in any way, and getting who knows what kind of experience.
Ultimately, if I'm being very honest, I think we're seeing the cracks here in trying to use a single product (Kibana and its on-prem plugins) to also serve an entirely new offering/product. I imagine every solution to these problems is going to feel "a bit off" because of this. |
@rudolf We have several API's that we'd like to disable in serverless. Here's the list as of today (spaces to be re-evaluated):
We also have some API's that are currently designated as public but we'd like to designate as internal for serverless:
|
Just dropping another thought here (trying to look at this from a different POV): if we think of Serverless' Project Types as product offerings with different features and pricing, you could argue that we are limiting access to some features based on their level of access (even when it's applied to all the users in that specific Project Type) 😇 WDYT? |
This, to me, sounds most similar to the |
Thanks for all of the conversation, everyone. Unfortunately, it doesn't seem like we are any closer to a solution, here. Which likely means each plugin will develop their own, bespoke way for controlling route access in a serverless (and other, similar context) world. I'm feeling the urge to go "wrap the router" and make it respond to custom config to turn off routes, and that always feels like the wrong urge. @jeramysoucy do you have plans for how you are going to control the access changes you mention in your latest comment? |
@jasonrhodes We don't have a definitive plan or design yet. We were hoping for either a unified mechanism or some guidance from core. We have a sync with them tomorrow and this will be part of the discussion. |
My 2 cps: Security strongly pushed against I agree that Remain I personally would prefer
Now, to implement So, if we want to go with E.g, would Because in that case, we could imagine something that would looks like the example from my initial contribution to this issue: router.post(
{
path: '/someApi',
options: {
condition: { // only enabled on serverless for solution-1 or solution-2 type of projects.
env: "serverless",
projects: ["project-1", "project-2"]
}
},
validate: false
},
myHandler,
);
router.post(
{
path: '/someOtherApi',
options: {
condition: { // not enabled on serverless (on any solution type)
env: "not-serverless", // idk, we need to name this consistently at some point
}
},
validate: false
},
myHandler,
); (naive API example, it's just to illustrate) Otherwise, I think we would unfortunately have to go with |
This is sounding very much to me like route-level feature flags. Sticking with (2) I can see two variations:
serverless.yml
serverless.oblt.yml
|
I just want to get back to @afharo's comment in #157266 (comment):
This is absolutely right. Core has no control on application sub paths ("UI routes") in any way (we're not even the same react app, technically). Meaning that option So depending on how much consistency we want between the two sides of the problem ( blocking UI routes and disabling server endpoints), we may prefer to just use Note though, I personally believe these two sides of the problem are actually two distinct problems that should be addressed differently. Blocking subparts of an application on the UI seems to be just like one of the dozen of kinds of "UI customizations" we want to be able to do to differentiate the two "products", so I would be fine with less consistency, and more specialization in the way we'll address them, and think that doing with |
@rudolf @pgayvallet There are also cases where API access would differ between current offerings and serverless (public -> internal). On/off would not account for environment dependant access differences. Would it make sense to specify access as 'public' | 'internal' | 'disabled' per environment, or consider access separately from enabled/disabled? Not sure if other teams are also considering access differences in their API's for serverless, but we have a few mentioned here. |
Define But see, this confirms my concerns regarding whether we can design something generic here. From my understanding, your team only need to configure routes depending on server/not-serverless. Some IIRC other teams already expressed the need to configure depending on the project type (but without the need to switch the APIs to |
By I don't know how cleanly this logic would apply to the feature flags approach Rudolf suggested, but I do think it would work with Pierre's suggestion (#157266 (comment)) of having env-based conditions in the route options. In that case two routes would be registered: one I suppose there's also the router.post(
{
path: '/someApi',
options: {
// all conditions must be true for route to be registered
condition: {
env: "serverless",
projects: ["project-1", "project-2"],
featureFlags: ["foo"]
}
},
validate: false
},
myHandler,
); featureFlags.foo.enabled: true # default for the feature flag can be set wherever we register them If we want to go down the feature flags route, my one suggestion would be to give the design some thought -- feature flags have been brought up frequently in the past, so if we decide to provide an official platform-level mechanism for doing this, we should be certain we are aligned on functional requirements, etc. IMO the easier short-term path here is env-based configuration since it seems it would address the main concerns as a starting point, and we could still layer in feature flags later if it turned out to be necessary.
Agree with this, I see the UI use case as distinct here and something that could be solved with lower level configs. For example - say I have plugin # foo is enabled, but skips registering bar app, so none of the UI routes are enabled
foo.bar.ui.enabled: false
# foo registers bar app, but disables feature A and any associated UI routes
foo.bar.ui.a.enabled: false Because it's not just about disabling the routes in the browser, ideally we are excluding any unnecessary JS from our initial and async browser bundles so we aren't loading code for features that will never be available anyway. |
I love how this conversation is evolving. To make sure I understand where we are getting at, I'll try to summarize the current suggestion:
Please, feel free to correct me if I misunderstood or missed anything. Related side conversations:
|
Thanks @lukeelmers, that's correct (see also #156935).
@afharo This was Luke's suggestion as well, and I think this would work. Internal callers would need to know which path to call as well. |
TBH, that's my main concern with treating both (HTTP APIs and UI) separately: the UI needs to know if the route is available. Otherwise, it will degrade the UX by showing hundreds of expected errors. We'll need the plugin-owned settings used to register the UI fixtures to be perfectly in sync with the feature flags/conditions that enable/disable HTTP APIs. IMO, that's a ticking bomb waiting to explode. IMO, the easiest mental model for the developers is: Feature A is enabled, hence, related HTTP APIs and UI components are enabled. IMO, this translates to one option, We can def modify the HTTP registrar to avoid devs using router.post(
{
path: '/getLogs',
options: {
featureFlags: [
{name: "myPlugin.featureA.enabled", default: "enabled"}
]
},
validate: false
},
myHandler,
); But that'd involve the HTTP Core service having access to all plugins' config, which goes against our design patterns, so the registrar will require the plugin to provide the config: router.post(
{
path: '/getLogs',
options: {
featureFlags: {
pluginConfig: config,
rules: [
{name: "featureA.enabled", default: "enabled"}
]
},
validate: false
},
myHandler,
); At that point, is there such a big difference to the code below? if (config.featureA.enabled) {
router.post(
{
path: '/getLogs',
validate: false
},
myHandler,
);
} Another option could be to create a new
Also: are we sure that feature flags are |
@pgayvallet and @afharo I'm not sure I follow this. If the router knows how to interpret some kind of tag/flag on the route definition in the UI, aren't all paths and sub-paths in a Kibana plugin's UI defined via this router's route definitions? Is there a way plugins are defining ad-hoc sub-path routes that aren't defined by Or do you just mean links to other parts of a plugin sometimes failing because that path happens to be blocked? |
Looks like APM is already moving forward with a custom way of handling this, so I'll back myself out of this conversation. If you all end on a recommended approach, please ping me, but for now we will likely just handle it all inside the plug-in code. |
An update on what ML has done for this.
I've added three new ML capabilities ( I've updated our ML capabilities switcher to look at which ML features are disabled and to switch off all related capabilities. ML has always had very granular capabilities which are checked for both server and client side routes, so being able to switch off, for example, all anomaly detection endpoints and client side pages for all users with a single flag is again very convenient. I've updated our deeplink registration based on enabled features, so the serverless nav menus and app search will only contain links to pages which are enabled in the project. |
Kibana plugins can specify routes on the server and on the client/UI side. The plugin author may not want all of that plugin's routes to be available in every scenario, but adding forking code within the route handler can be cumbersome. It might be very useful to introduce a configuration value available in kibana.yml, e.g.
disabledRoutes
, that would allow plugin authors to "404" a subset of routes in a certain situation without needing to make code changes inside of the plugin's app structure.Something like this?
or
My thought was that the router could check this config only on start-up, and compare it to each route as it's being registered. If the disabled parameters match a registered route, that route handler would be replaced with a default 404 handler instead of the handler being registered (along with possible auto-telemetry instrumentation to log the number of requests that have been 404-ed?)
This is just one possible solution to the issue, but the primary use case for this right now is that we want to disable certain UI and API paths in the serverless world, and we'd love it if that didn't require the registration logic to check config inside of each app that needs to do this disabling.
The text was updated successfully, but these errors were encountered: