[RFC] Introduce Application Configuration Service to support hot reloading #5796

tianleh · 2024-02-03T02:08:30Z

Problem Statement #

OpenSearch Dashboards (OSD) currently provides two ways to make configuration changes. The first one is to use OSD YML file opensearch_dashboards.yml (one example is to configure the OSD index name). The second is to use advanced settings page (one example is to configure dark mode). Recently there is an emerging requirement to have an application level configuration mechanism. These configurations may need to take effect without system restart and are not user level configuration. Besides supporting use cases from OSD core, we also plan to support OSD plugins use cases. The last factor to consider is the storage database. While current OSD assumes a default OpenSearch as database, we plan to have an extensible solution where other databases can be used.

We will use this issue to explain why the existing features won't meet the requirements and proposal a new solution which will abstract an interface and web APIs in order to decouple with storage, while we need a implementation of this interface for opensearch. The solution is not a direct extension of existing configurations in YML. Instead each use case developer will decide whether they want to pick YML configurations or dynamic configuration (or have both) and implement logic to use these configurations.

Desired State #

The configuration updates shall be "hot reload" which means that they will take effect without waiting for OSD to restart.
Both OSD core and plugins could use it to read and write their application level configurations.
It shall be feasible to use a different storage for the configurations other than the default OpenSearch.
When FGAC is enabled, there shall be permission control over updates of the configurations.

Out of Scope #

We do not plan to provide new UX for reading and writing the configurations
We do not plan to move all existing configurations in YML or advanced settings to the new application configuration

Existing Limitations #

We will take a close look at the current limitations.

OSD YML #

Only system administrators who have access to OSD YML could update the values. The changes need a server restart to take effect. It will cause the risk of service availability interruption. Use a cluster with 10 hosts running OSD as an example. The system administrator will need to connect to each of the host, update the YML with new value and then restart OSD. Note that regular OSD customers only need to access OSD through endpoint without even knowing the OSD's running environment under the hood.

Also OSD YML controls the static application level configurations and doesn't support user specific configurations.

Advanced Settings #

Advanced Settings have a page on OSD to configure user specific settings. The changes will take effect immediately without a server start. The values are stored in the OSD system index (.kibana). When FGAC is enabled, the values will be stored into tenant specific system index. (e.g .kibana_{tenant name}) where each tenant could have different configuration values. This doesn't meet the requirement of application level configuration service where the values shall not be user specific.

Proposed Approach #

We propose to introduce a new index (.opensearch_dashboards_config) to store application configurations. Each entity could be put in a document inside the index. (E.g Content security policy rule configurations can be put in a document whose doc ID is csp.rules. ) We will make abstractions to a interface and web APIs in order to decouple with storage, while we will provide a implementation of this interface for OpenSearch.

The index won’t exist by default. When the OSD process reads it empty, there is no dynamic configuration. We will implement APIs at OSD to allow customers to read and write the content of the index. Note that the OSD process won’t create this index and thus there is no concern for race condition. When FGAC is enabled, similar to how regular indices are access controlled, only customers with the write permission could perform the update. OSD admins could delegate the work to others by assigning them a role which has write permission to the new index.

Below is how a document looks like in the index (use csp.rules as an example).

      {
        "_index": ".opensearch_dashboards_config",
        "_id": "csp.rules",
        "_score": 1,
        "_source": {
          "value": "frame-ancestors 'self' file://* filesystem:"
        }
      },

APIs #

There will be two types of APIs mentioned in this issue. One type is about the configuration level APIs which will be implemented as part of this issue. The other type is about the document level (or use case specific) APIs which will be designed and implemented by whoever onboards their use cases on to this configuration service.

Below is the list of index level APIs customers can call.

Get the configuration. The return value is the content of the index.
curl -X GET 'http://localhost:5601/api/appconfig'

Below is the list of document level APIs customers can call.

Get the field. The return value is the value of the field in the index.
curl -X GET 'http://localhost:5601/api/appconfig/{entity}'

Update the field. It will update the field with a new value specified in the request body.
curl -X POST 'http://localhost:5601/api/appconfig/{entity}'

Delete the field. 
curl -X DELETE 'http://localhost:5601/api/appconfig/{entity}'

Note that we do not provide update or delete API at the index level to avoid accidental mistakes. Also no create API is provided at index level as document level update has been provided. See this section Onboard More Use Cases for the onboarding instruction.

New Plugin #

We will introduce a new plugin ApplicationConfiguration to implement the APIs above. The plugin will only have a server side. It could be extended to have a public side when the team decides to add UX to manage the application configurations. Inside the plugin, we will add a service ApplicationConfigurationService which enables other plugins to use the configurations.

Multiple Storage Types #

Multiple data sources and metadata storage have decoupled from OpenSearch as database. Our proposal will be extensible for different types of configuration storage (e.g DynamoDB, Postgres) instead of hard coding the dependence on OpenSearch. Note that a different storage to replace OpenSearch is for the whole OSD core and plugins and is not for each plugin to have their own storage. Thus at most one plugin to provide different storage is allowed.

Thus an interface abstracting operations for this configuration would be introduced. Below is the proposed interface for configuration level APIs.

export interface ConfigurationClient {
    getConfig(): Promise<Map>;

    getFeildConfig(entity);

    updateFeildConfig(entity, newValue);

    deleteFeildConfig(entity);
}

A default implementation to this interface will be based on OpenSearch. The ApplicationConfiguration plugin would expose a set function to allow other plugins (external to OSD core) to provide different implementations to this interface. Specifically, the ApplicationConfiguration has a member variable configurationClient which is of the type ConfigurationClient mentioned above. It also has a member method setConfigurationClient to set the value of the configurationClient. This setConfigurationClient function would be exposed as return value of ApplicationConfiguration’s setup function. The ApplicationConfiguration has another member method getConfigurationClient to return the client to use. See below class skeleton for references.

// ApplicationConfigurationPlugin
export class ApplicationConfigurationPlugin{
    private configurationClient: ConfigurationClient;

    private getConfigurationClient(inputOpenSearchClient: IScopedClusterClient) {
        if (this.configurationClient) {
            // use whatever client if already set
            return this.configurationClient;
        }
        
        // Default OpenSearchConfigurationClient implements the interface ConfigurationClient which is 
        // based on OpenSearch.
        return new OpenSearchConfigurationClient(inputOpenSearchClient);
    }
    
    private setConfigurationClient(inputConfigurationClient: ConfigurationClient) {
        this.configurationClient = inputConfigurationClient;
    }
    
    public async setup(core: CoreSetup) {
  
        ...
        return {
            // use bind to preserve this
            setConfigurationClient: this.setConfigurationClient.bind(this),
        };
  
    }

  }

Now an external plugin, e.g MyConfigurationPlugin could inject its version of ConfigurationClient into the class ApplicationConfigurationPlugin in the following way.

Define AppPluginSetupDependencies as follows.

export interface AppPluginSetupDependencies {
  applicationConfig: ApplicationConfigPluginSetup;
}

It will import the type ApplicationConfigurationPluginSetup from the directory of the class ApplicationConfigurationPlugin.

Next add AppPluginSetupDependencies to its own setup input.

# MyConfigurationPlugin   
  public setup(core: CoreSetup, { applicationConfig }: AppPluginSetupDependencies) {
     ...
    # The function createClient provides an instance of ConfigurationClient which
    # could have a underlying DynamoDB or Postgres implementation.
    const myClient: ConfigurationClient = this.createClient();

    applicationConfig.setConfigurationClient(myClient);
     ...  
    return {};
  }

(Note that the second variable in setup function must be exactly applicationConfig which is the ID of the configuration plugin.)

Onboard More Use Cases #

There is no need to add more APIs or interfaces for new use cases since path parameters could be leveraged. Use CSP rules as an example. Customers can use the APIs as follows.

Get the CSP rules
curl -X GET 'http://localhost:5601/api/appconfig/{csp.rules}'

Update the CSP rules
curl -X POST 'http://localhost:5601/api/appconfig/{csp.rules}'

Delete the CSP rules
curl -X 'http://localhost:5601/api/appconfig/{csp.rules}'

Alternative Approach #

The proposed approach provides a generic way for all use cases. One constraint is that all storage systems will have to use the same name for the document and filed. Use CSP rules as an example, they will have to use the same string csp.rules. (One workaround would be maintaining a mapping from the input csp.rules to the actual schema name. This would be a tedious to maintain.)

A alternative approach has the use case name in the APIs and interfaces. Below is the list of APIs.

Get the CSP rules
curl -X GET 'http://localhost:5601/api/appconfig/csp/rules'

Update the CSP rules
curl -X POST 'http://localhost:5601/api/appconfig/csp/rules'

Delete the CSP rules
curl -X 'http://localhost:5601/api/appconfig/csp/rules'

Below is the interface.

export interface ConfigurationClient {
    getCspRules(): Promise<string>;

    updateCspRules(cspRules: string): Promise<string>;

    deleteCspRules(): Promise<string>;
}

This means that any future use case will need to make changes in OSD core package and add more APIs and interfaces.
Use the number of parameters in OSD YML file (~70) to estimate the potential number of use cases to onboard the application configuration. Each of them would have three functions which totally results in ~210 functions into the interface and the APIs. This still doesn't include the use cases from OSD plugins.

One advantage of this alternative approach is that it can avoid accidental updates. Since each use case has its own API path, there is no chance one use case would accidentally update another use case. As a comparison, in the generic approach, since each use case just uses a string, there is a chance for one use case to accidentally modify the configurations of another use case.

In a summary, the gain of the flexibility of naming the table schema is trivial compared to the cost of increasing APIs and interfaces to support. Thus this alternative approach is not preferred.

Comparison Chart #

Description	code changes for new use case	flexibility of naming
Generic APIs/interfaces	No	No
Specific APIs/interfaces	Yes	Yes

Appendix #

Clickjacking use case to onboard #

This is one use case to store CSP rules into the application configurations. #5639

POC of plugin registration #

This plugin csp_storage_provider is an example where it can create its own configuration client and register to the configuration plugin xframeOptions. The POC code is pushed to https://github.com/tianleh/OpenSearch-Dashboards/tree/cj-dev/plugins/csp_storage_provider

Collect input from all plugin teams #

Row ID	Plugin	Tracking Issue	Status
1	dashboards-observability	opensearch-project/dashboards-observability#1465	Issue Created
2	dashboards-visualizations	opensearch-project/dashboards-visualizations#346	Issue Created
3	dashboards-assistant	opensearch-project/dashboards-assistant#153	Issue Created
4	security-analytics-dashboards-plugin	opensearch-project/security-analytics-dashboards-plugin#897	Issue Created
5	dashboards-reporting	opensearch-project/dashboards-reporting#311	Issue Created
6	dashboards-query-workbench	opensearch-project/dashboards-query-workbench#271	Issue Created
7	dashboards-notifications	opensearch-project/dashboards-notifications#163	Issue Created
8	anomaly-detection-dashboards-plugin	opensearch-project/anomaly-detection-dashboards-plugin#692	Issue Created
9	alerting-dashboards-plugin	opensearch-project/alerting-dashboards-plugin#887	Issue Created
10	dashboards-maps	opensearch-project/dashboards-maps#586	Issue Created
11	ml-commons-dashboards	opensearch-project/ml-commons-dashboards#305	Issue Created
12	index-management-dashboards-plugin	opensearch-project/index-management-dashboards-plugin#998	Issue Created
13	dashboards-search-relevance	opensearch-project/dashboards-search-relevance#372	Issue Created
14	security-dashboards-plugin	opensearch-project/security-dashboards-plugin#1791	Issue Created

The text was updated successfully, but these errors were encountered:

seraphjiang · 2024-02-06T19:49:13Z

The application config service provide a way to load dynamic application config after OSD process started compare to the way current osd.yaml support. That make OSD application config support multi-tenancy ready at application config tier.

To help the design, we could brainstorming user cases that could leverage the application config service

Content Security Policy @tianleh
CORS policy @tianleh
Application config for AI Assistant @lezzago
Customer branding config @abbyhu2000

CC: @dagneyb @kgcreative

seraphjiang · 2024-02-06T20:39:24Z

@tianleh, while we are collecting more user cases, would you please come up with the initial proposal soon?

tianleh · 2024-02-06T21:53:25Z

@tianleh, while we are collecting more user cases, would you please come up with the initial proposal soon?

Sure. This is my priority.

seraphjiang · 2024-02-11T17:53:44Z

We propose to introduce a new index (.opensearch_dashboards_config) to store application configurations. Each category of configurations could be organized in a document inside the index. (E.g Content security policy related configurations can be put in a document whose doc ID is csp. )

We need to abstract to a interface and webapi in order decouple with storage, while we need a implementation of this interface for opensearch.

seraphjiang · 2024-02-11T17:58:15Z

Update the CSP rules
curl 'http://localhost:5601/api/config/csp/rules/update'
Delete the CSP rules
curl 'http://localhost:5601/api/config/csp/rules/delete'

any reason we put verb (GET, UPDATE, DELETE) part of endpoint?

tianleh · 2024-02-12T21:37:11Z

Update the CSP rules
curl 'http://localhost:5601/api/config/csp/rules/update'
Delete the CSP rules
curl 'http://localhost:5601/api/config/csp/rules/delete'

any reason we put verb (GET, UPDATE, DELETE) part of endpoint?

I will switch to use route.get/delete/post and avoid making it part of endpoint.

tianleh · 2024-02-13T21:22:25Z

We propose to introduce a new index (.opensearch_dashboards_config) to store application configurations. Each category of configurations could be organized in a document inside the index. (E.g Content security policy related configurations can be put in a document whose doc ID is csp. )

We need to abstract to a interface and webapi in order decouple with storage, while we need a implementation of this interface for opensearch.

Explicitly called this out in the problem statement.

tianleh · 2024-02-13T21:51:31Z

Removed draft from the title and make it ready for review.

tianleh · 2024-02-16T02:34:16Z

Talked with @seraphjiang and @bandinib-amzn about generic and specific APIs/interfaces. I have updated the RFC to prefer generic style and moved specific style to alternative approach.

Will collect more input from @lezzago and @abbyhu2000

zengyan-amazon · 2024-02-20T21:03:47Z

are we going to save the configurations as saved objects? does customer need some permission to manage the configs?

kavilla · 2024-02-20T21:04:20Z

The configuration updates shall be "hot reload" which means that they will take effect without waiting for OSD to restart.

Do we think we need to break out the config or the APIs into "on setup" or run time. Some configurations might require at least on browser reload depending if it's trying to intercept all requests.

kavilla · 2024-02-20T21:10:10Z

#5796 (comment)

curl -X POST 'http://localhost:5601/api/appconfig/

to create the index with the mappings as expected so a privileged user can just do a POST call. Or have a "hidden" public part of the plugin that allows navigating to and has a button to create the plugin? Sort of like this hidden route https://playground.opensearch.org/app/status

kavilla · 2024-02-20T21:15:12Z

I like the generic APIs were we get the value instead of targetting like CSPs specfic.

But perhaps theres an inbetween layer like stated before like "before load" or on load that we define. or need to subset them in a different way like "httpConfig" "appConfig" etc. Then I can just getHttpConfig

bandinib-amzn · 2024-02-20T21:16:01Z

While providing flexibility how are we making sure that documentName is unique? Also can we use something more generic for documentName parameter name? document only makes sense in case of index.

tianleh · 2024-02-20T21:30:02Z

Comments from Yan: check whether we could extend the existing config plugin

tianleh · 2024-02-20T21:54:57Z

There are some confusions around whether each plugin will have their own storage or all components will use a single storage. I will update the doc to address this confusion.

tianleh · 2024-02-21T01:35:14Z

are we going to save the configurations as saved objects? does customer need some permission to manage the configs?

By default, these configurations will be stored into a new OpenSearch index. Each entity will become a document in the index instead of saved objects. Use csp.rules as an example. This is how it looks like in the index.

      {
        "_index": ".opensearch_dashboards_config",
        "_id": "csp.rules",
        "_score": 1,
        "_source": {
          "value": "frame-ancestors 'self' file://* filesystem:"
        }
      },

For permission control, the customers will require write permission to this index to call the update API. This is achieved through using scoped asCurrentUser client.

If there is an external configuration client, e.g based on DynamoDB, then it will be up to the client to perform permission check.

@zengyan-amazon

tianleh · 2024-02-21T22:47:37Z

As follow up of @bandinib-amzn 's request, shared my previous POC work in the issue #5796 (comment)

tianleh · 2024-02-22T00:05:22Z

There are some confusions around whether each plugin will have their own storage or all components will use a single storage. I will update the doc to address this confusion.

Added contents to emphasize that a different storage is for both OSD core and plugins to use and not for each plugin to use their own storage. #5796 (comment)

seraphjiang · 2024-02-22T00:38:54Z

Added contents to emphasize that a different storage is for both OSD core and plugins to use and not for each plugin to use their own storage. #5796 (comment)

Plugin should not own their own storage for meta shared cross OSD.

@tianleh would you help to create issue in each plugin repo remind plugin maintainer to review the proposal and raise concern if they had different ideas.

tianleh · 2024-02-22T02:07:58Z

Added contents to emphasize that a different storage is for both OSD core and plugins to use and not for each plugin to use their own storage. #5796 (comment)

Plugin should not own their own storage for meta shared cross OSD.

@tianleh would you help to create issue in each plugin repo remind plugin maintainer to review the proposal and raise concern if they had different ideas.

Tracking issues are created. #5796 (comment)
@seraphjiang

bandinib-amzn · 2024-02-22T02:14:48Z

Hi @tianleh

Thank you for creating issue. I think Dynamic Configuration will greatly help customers.

I have few questions regarding design.

Do you consider extending current static config service to achieve Dynamic Configuration? If not what are your plans to validate the configuration?
Also why extensions (component or plugin which implement ConfigurationClient) should specify entity? Are we expecting plugins to handle request according to Dynamic config? For example, here you build new plugin to handle CSP rules in header. I think approach could be vulnerable to security issue and error, causing customer frustration. Ideally, OpenSearch Dashboard should be in control of configuration keys, which are basically. Dynamic Configuration should check if for config ABC value is coming from storage (in your case config index) then it should overwrite static value from YML file. The above design does not talk about how you will apply dynamic config to application.
As a user I want to dynamically set server.maxPayloadBytes to greater value 10485760000 (in YML, it is 1048576). I'm still not clear what needs to done? If I set, server.maxPayloadBytes using curl -X POST 'http://localhost:5601/api/appconfig/{entity}' Will it be enough?

tianleh · 2024-02-22T03:27:50Z

Hi @tianleh

Thank you for creating issue. I think Dynamic Configuration will greatly help customers.

I have few questions regarding design.

1. Do you consider extending current static [config service](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/packages/osd-config/src/config_service.ts) to achieve Dynamic Configuration? If not what are your plans to validate the configuration?

2. Also why extensions (component or plugin which implement `ConfigurationClient`) should specify entity? Are we expecting plugins to handle request according to Dynamic config? For example, [here](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5641/files#diff-a323d0cc51966fc49953717005fd133b4aa6311a043df924bc6ea0fb86c8a9aa) you build new plugin to handle CSP rules in header. I think approach could be vulnerable to security issue and error, causing customer frustration. Ideally, OpenSearch Dashboard should be in control of configuration keys, which are basically. Dynamic Configuration should check if for config `ABC` value is coming from storage (in your case config index) then it should overwrite static value from YML file. The above design does not talk about how you will apply dynamic config to application.

3. As a user I want to dynamically set `server.maxPayloadBytes` to greater value `10485760000` (in YML, it is `1048576`). I'm still not clear what needs to done? If I set, server.maxPayloadBytes using `curl -X POST 'http://localhost:5601/api/appconfig/{entity}'` Will it be enough?

For # 3, the application configuration plugin solves the problem of storage of the configurations and doesn't solve how the stored configurations will be used. With this said, below is the experience of your example of server.maxPayloadBytes. Customers will call the POST API to set the value. This solves the problem of how new values are stored. Then code changes are required at the place where server.maxPayloadBytes is used to use the value from application configuration plugin.

The more I read the comments, the more I understand where the confusion comes from. The way I write the RFC might give readers an impression that my proposal is an enhancement/extension of the existing YML configurations. Then it will take care of the aggregation of the value from YML and index. Actually it is more providing a runtime configuration which is irrelevant whether the configuration is part of YML or not. There could be a use case where they have no such thing in YML at all. Each use case will implement their code logic about how to use their configurations. @bandinib-amzn

cc @seraphjiang

tianleh · 2024-02-22T03:37:04Z

While providing flexibility how are we making sure that documentName is unique? Also can we use something more generic for documentName parameter name? document only makes sense in case of index.

Switched to call it an {entity}. There is no unique constraint applied. We will need to add documentations about using unique entity names when making the configuration changes. One potential way to reduce the risk is that we add a namespace in the path to make it look /{namespace}/{entity}. I think we can start with a simple solution and then evolve to have more protections. (e.g make namespace part of the API instead of path parameter) @bandinib-amzn

cc @seraphjiang

tianleh · 2024-02-22T03:51:54Z

Since we plan to start parallel work to develop an external plugin for providing a DynamoDB based configuration client, let me know whether the API and interfaces part are concerning from your perspective. @bandinib-amzn

cc @seraphjiang

tianleh · 2024-02-22T04:56:19Z

Hi @tianleh

Thank you for creating issue. I think Dynamic Configuration will greatly help customers.

I have few questions regarding design.

1. Do you consider extending current static [config service](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/packages/osd-config/src/config_service.ts) to achieve Dynamic Configuration? If not what are your plans to validate the configuration?

2. Also why extensions (component or plugin which implement `ConfigurationClient`) should specify entity? Are we expecting plugins to handle request according to Dynamic config? For example, [here](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5641/files#diff-a323d0cc51966fc49953717005fd133b4aa6311a043df924bc6ea0fb86c8a9aa) you build new plugin to handle CSP rules in header. I think approach could be vulnerable to security issue and error, causing customer frustration. Ideally, OpenSearch Dashboard should be in control of configuration keys, which are basically. Dynamic Configuration should check if for config `ABC` value is coming from storage (in your case config index) then it should overwrite static value from YML file. The above design does not talk about how you will apply dynamic config to application.

3. As a user I want to dynamically set `server.maxPayloadBytes` to greater value `10485760000` (in YML, it is `1048576`). I'm still not clear what needs to done? If I set, server.maxPayloadBytes using `curl -X POST 'http://localhost:5601/api/appconfig/{entity}'` Will it be enough?

For point # 2, when there is no default OpenSearch, an external storage plugin (say based on DynamoDB) will provide a configuration client and register to the application configuration plugin. Similar to query a document from an index in OpenSearch, a customer shall be able to query an entity from a DynamoDB table. The request to the configuration storage will go through the application configuration plugin. The example link you referred was based on old implementation where both configurations and CSP handler are mixed together. (This was also the reason why we separate the scope of application configuration out of the original RFC. ) Once the current application configuration plugin is implemented, the example link will be updated to depend on application configuration plugin. @bandinib-amzn

tianleh · 2024-02-22T06:32:33Z

Hi @tianleh

Thank you for creating issue. I think Dynamic Configuration will greatly help customers.

I have few questions regarding design.

1. Do you consider extending current static [config service](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/packages/osd-config/src/config_service.ts) to achieve Dynamic Configuration? If not what are your plans to validate the configuration?

2. Also why extensions (component or plugin which implement `ConfigurationClient`) should specify entity? Are we expecting plugins to handle request according to Dynamic config? For example, [here](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5641/files#diff-a323d0cc51966fc49953717005fd133b4aa6311a043df924bc6ea0fb86c8a9aa) you build new plugin to handle CSP rules in header. I think approach could be vulnerable to security issue and error, causing customer frustration. Ideally, OpenSearch Dashboard should be in control of configuration keys, which are basically. Dynamic Configuration should check if for config `ABC` value is coming from storage (in your case config index) then it should overwrite static value from YML file. The above design does not talk about how you will apply dynamic config to application.

3. As a user I want to dynamically set `server.maxPayloadBytes` to greater value `10485760000` (in YML, it is `1048576`). I'm still not clear what needs to done? If I set, server.maxPayloadBytes using `curl -X POST 'http://localhost:5601/api/appconfig/{entity}'` Will it be enough?

For point # 1, @zengyan-amazon has a similar comment as well #5796 (comment)

There are two reasons not to extend it.

The application configuration is not intended to mimic YML and could embrace more use cases than the YML.
Forcing extending it would cause some dependency issue.

Below is the code snippet from the server.ts
https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/core/server/server.ts#L112


    this.configService = new ConfigService(rawConfigProvider, env, this.logger);

    const core = { coreId, configService: this.configService, env, logger: this.logger };
    this.context = new ContextService(core);
    this.http = new HttpService(core);

Currently configService only reads from file system and doesn't have other dependencies. However, if we force extending it, it will need to have a core.http to define the routes (for API use case). Then the dependency chain looks like configService -> http -> core -> configService.

This is why we create a new plugin to host the dynamic configuration logic.

@bandinib-amzn @zengyan-amazon

bandinib-amzn · 2024-02-23T20:30:23Z

Each use case will implement their code logic about how to use their configuration

I understand your perspective. I'm curious, however, about the added value of this feature beyond offering flexibility in storing configurations, especially considering that each use case would need to implement their own code logic for utilizing their configurations.

tianleh · 2024-02-24T01:14:36Z

I understand your perspective. I'm curious, however, about the added value of this feature beyond offering flexibility in storing configurations, especially considering that each use case would need to implement their own code logic for utilizing their configurations.

Part of the flexibility is that use case owners now have the option to only use dynamic configurations.

cwperks · 2024-03-04T17:00:20Z

FYI The security-dashboards-plugin already has its own mechanism for supporting dynamic configuration via the security index. This section of the security config powers some (but not all) of the dashboards security configuration.

The security plugin has a /_plugins/_security/api/dashboardsinfo endpoint that the security-dashboards-plugin calls on to get the most recent config values and it uses the internal user kibanaserver when making the call to that endpoint.

tianleh · 2024-03-06T18:08:36Z

FYI The security-dashboards-plugin already has its own mechanism for supporting dynamic configuration via the security index. This section of the security config powers some (but not all) of the dashboards security configuration.

The security plugin has a /_plugins/_security/api/dashboardsinfo endpoint that the security-dashboards-plugin calls on to get the most recent config values and it uses the internal user kibanaserver when making the call to that endpoint.

Thanks @cwperks for sharing this! Since your configurations are stored in the security index, does security plugin have a plan to support a different backend storage, say if there is no default opensearch?

cc @seraphjiang

github-actions bot added the untriaged label Feb 3, 2024

tianleh mentioned this issue Feb 3, 2024

Support dynamic CSP rules to mitigate clickjacking #5641

Merged

7 tasks

seraphjiang added the v2.13.0 label Feb 6, 2024

seraphjiang assigned tianleh Feb 6, 2024

seraphjiang removed the untriaged label Feb 6, 2024

xinruiba mentioned this issue Feb 8, 2024

[MD] Able to Hide "Local Cluster" option from datasource DropDown and Index Pattern creation page #5754

Closed

tianleh changed the title ~~[RFC][Draft] Introduce Application Configuration Service~~ [RFC][Draft] Introduce Application Configuration Service to support hot reloading Feb 8, 2024

This was referenced Feb 9, 2024

Add support for dynamic application configurations #5855

Merged

[RFC] Clickjacking Mitigation #5639

Closed

tianleh changed the title ~~[RFC][Draft] Introduce Application Configuration Service to support hot reloading~~ [RFC] Introduce Application Configuration Service to support hot reloading Feb 13, 2024

seraphjiang mentioned this issue Feb 25, 2024

[Meta][2.13] Security, bug fix and enhancemanet for dashboards anywhere #5752

Open

28 tasks

ZilongX closed this as completed in #5855 Mar 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC] Introduce Application Configuration Service to support hot reloading #5796

[RFC] Introduce Application Configuration Service to support hot reloading #5796

tianleh commented Feb 3, 2024 •

edited

Loading

seraphjiang commented Feb 6, 2024

seraphjiang commented Feb 6, 2024

tianleh commented Feb 6, 2024

seraphjiang commented Feb 11, 2024

seraphjiang commented Feb 11, 2024

tianleh commented Feb 12, 2024

tianleh commented Feb 13, 2024

tianleh commented Feb 13, 2024

tianleh commented Feb 16, 2024

zengyan-amazon commented Feb 20, 2024

kavilla commented Feb 20, 2024

kavilla commented Feb 20, 2024 •

edited

Loading

kavilla commented Feb 20, 2024 •

edited

Loading

bandinib-amzn commented Feb 20, 2024 •

edited

Loading

tianleh commented Feb 20, 2024

tianleh commented Feb 20, 2024

tianleh commented Feb 21, 2024 •

edited

Loading

tianleh commented Feb 21, 2024

tianleh commented Feb 22, 2024

seraphjiang commented Feb 22, 2024

tianleh commented Feb 22, 2024

bandinib-amzn commented Feb 22, 2024

tianleh commented Feb 22, 2024

tianleh commented Feb 22, 2024

tianleh commented Feb 22, 2024

tianleh commented Feb 22, 2024

tianleh commented Feb 22, 2024

bandinib-amzn commented Feb 23, 2024

tianleh commented Feb 24, 2024

cwperks commented Mar 4, 2024

tianleh commented Mar 6, 2024

[RFC] Introduce Application Configuration Service to support hot reloading #5796

[RFC] Introduce Application Configuration Service to support hot reloading #5796

Comments

tianleh commented Feb 3, 2024 • edited Loading

Problem Statement #

Desired State #

Out of Scope #

Existing Limitations #

OSD YML #

Advanced Settings #

Proposed Approach #

APIs #

New Plugin #

Multiple Storage Types #

Onboard More Use Cases #

Alternative Approach #

Comparison Chart #

Appendix #

Clickjacking use case to onboard #

POC of plugin registration #

Collect input from all plugin teams #

seraphjiang commented Feb 6, 2024

seraphjiang commented Feb 6, 2024

tianleh commented Feb 6, 2024

seraphjiang commented Feb 11, 2024

seraphjiang commented Feb 11, 2024

tianleh commented Feb 12, 2024

tianleh commented Feb 13, 2024

tianleh commented Feb 13, 2024

tianleh commented Feb 16, 2024

zengyan-amazon commented Feb 20, 2024

kavilla commented Feb 20, 2024

kavilla commented Feb 20, 2024 • edited Loading

kavilla commented Feb 20, 2024 • edited Loading

bandinib-amzn commented Feb 20, 2024 • edited Loading

tianleh commented Feb 20, 2024

tianleh commented Feb 20, 2024

tianleh commented Feb 21, 2024 • edited Loading

tianleh commented Feb 21, 2024

tianleh commented Feb 22, 2024

seraphjiang commented Feb 22, 2024

tianleh commented Feb 22, 2024

bandinib-amzn commented Feb 22, 2024

tianleh commented Feb 22, 2024

tianleh commented Feb 22, 2024

tianleh commented Feb 22, 2024

tianleh commented Feb 22, 2024

tianleh commented Feb 22, 2024

bandinib-amzn commented Feb 23, 2024

tianleh commented Feb 24, 2024

cwperks commented Mar 4, 2024

tianleh commented Mar 6, 2024

tianleh commented Feb 3, 2024 •

edited

Loading

kavilla commented Feb 20, 2024 •

edited

Loading

kavilla commented Feb 20, 2024 •

edited

Loading

bandinib-amzn commented Feb 20, 2024 •

edited

Loading

tianleh commented Feb 21, 2024 •

edited

Loading