[RFC] OpenSearch Dashboards: Extension Installation

[manasvinibs]

Background

The OpenSearch project is introducing the OpenSearch and OpenSearch Dashboards SDKs. These SDKs provide a number of tools to make it easy to build, test, manage, install, and share feature enhancements. Additionally, the new framework results in an easier experience for administrators to add on to their OpenSearch and OpenSearch Dashboards deployments and maintain reliability.

In this document we are proposing different approaches to solve Extension installation into OpenSearch Dashboards.
Problem

Extension installation

Today, cluster admins have to completely stop OpenSearch Dashboards, download plugins and its dependencies that are compatible with their version of the application, and then restart the application when adding extra functionality with plugins. Customers need a way to easily find and install plugins and dependencies, that reduces the manual effort of ensuring OpenSearch and OpenSearch Dashboards has the required dependencies, and minimal interruption of services.

With this the first step is to decide on where extensions js bundle will live, how will the installation of new js bundle works with existing OpenSearch Dashboard core application. This decision will impact the way security authentication is done on extensions and how we can solve granting granular level access to extensions and if it is even required, which will be discussed as follow-up.

Scope:

In this doc, we are focused on Extensions being able to register itself within OpenSearch Dashboards without manual steps and render in the browser with minimum/no server downtime.

Out of scope for this doc:

Managing the dependencies
Extensions can install with configurations
Can extensions be configured with API?
How to manage extension dependencies and any other backend OpenSearch extension dependencies during installation process.
Working Backwards:

Who are the actors in the community:
a. Extension developer
b. End user and cluster admin

What would the customer like to see/use:
a. Not worry about updating Extension for every patch version of OpenSearch Dashboards.
b. Extension is not broken when OpenSearch Dashboards minor version is upgraded.
c. Install/Update/Remove an extension with/without restarting OpenSearch Dashboards ?
d. Not worry about manually restarting the OpenSearch Dashboards server to register the extension when it is installed.
e .Not worry about OpenSearch Dashboards going down due to an extension misbehaving.
f. Run a 3rd party extension and not worry about a 3rd party accessing data and configurations on the cluster to which it should not have permissions.
g. Ability to support granular access control of cluster resources for an extension, e.g. CPU, Memory, etc.
h. Ability to write an extension in any language of choice.
i. Want to be able to call the APIs provided by OpenSearch if available.

Possible solutions

• Run extensions as part of OpenSearch Dashboard’s process which is similar to current plugin architecture which involves extension discovery, setup dependencies, establish routes for all the bundles, bootstrapping, loading it to the browser and rendering it. (Recommended)

  • This will be enhancement to the current state where installing extension will require auto-restart of the OpenSearch Dashboard server.
  • In this approach extensions are hosted on the same server where OpenSearch dashboard is running.
  • There will be a server downtime and can impact user experience in the browser .
  • If extension JS bundle is hosted alongside OpenSearch Dashboards, then we will automate extensions installation on multi nodes within cluster as well. Installation script will take care of iterating through all the nodes in the domain and installing extensions on each one of them. The list of nodes will need to be discovered and provided to the API/CLI script as input param.
    

• Micro Frontend approach - In this approach, OpenSearch Dashboards core will be host application (shell) managing interfaces, contracts with backend and other performance improvement architecture work and extensions will be micro frontends focusing on feature extension, independent deployment, decoupled versions. (Recommended)

  • Extension JS bundle is hosted in remote node, fetched through http request and integrates bundle with OpenSearch Dashboards core using main thread execution during runtime of the application.
  • More isolated and can also be scaled between different machines/nodes
  • Multiple javascript extension bundles to render the final application instead of one monolithic javascript bundle.
  • Extensions can deploy independently from OSD.
  • Core OSD decouple from extension project, benefits version decoupling.
  • Gives extension developers flexibility to develop in their desirable technology.
  • Extension can be lazy loaded as required.
  • Pros:
    • Technology agnostic - Extensions need not follow same technology stack as OSD core does.
    • More isolated than current plugin architecture.
    • Better manageability of deployments and versioning.
  • Cons:
    • Risk of taking down the OSD application when extension micro-frontend has some issues and can cause run-time errors.
    • Complex architecture compared to monolithic repo model.

With this approach we are not running Extensions on a separate child thread within same container/process, but only using the event loop thread to discover, setup dependencies and establish routes to the extension bundle and call back bootstrap to give updated bundle chunks and re-render UI on the DOM mounted by main thread.

• Independent process/sandboxing of extension - Spinning separate thread to run extensions in the different node process than OSD process is running
  • Anti-pattern?
  • Locking issues, race conditions, memory/resource sharing issues basically not a thread safe
  • Scalability issue when multiple extensions spawns multiple child processes.

• No changes - Extension is same as plugins today.
  • Pros
    • No developmental effort required.
    • Continue to extend OpenSearch dashboards functionality with plugin’s current architecture.
    • Access to cluster internal APIs
  • Cons
    • Extensions continue to tightly couple with OSD core.
    • Slow developmental timeline due to cohesive framework.
    • Lack of ability to install/update/uninstall plugins without cluster restart.
    • No dynamic integration of features to dashboards.
    • Lack of dependency management and constant maintenance effort to be in sync with OpenSearch and OpenSearch dashboards versions.
    • No independent communication between extensions and between extension and OpenSearch.
    • No granular control access for extensions.

I want to propose installing extensions with first two approaches here because with the first approach we are going to quickly improve the current state of art for plugin installation by automating all the manual steps involved though it will not solve release problems, versioning for extensions due to its tight coupling with OpenSearch Dashboard. Here server restarts when registering extension which impacts user experience. This option will allow extension developers to not worry about hosting their extensions outside of OpenSearch Dashboards if they don't want to and maintains current plugin registration behavior except no manual steps involved.
With the second micro-frontend approach, we are making the architectural improvements to the bootstrapping technique to provide more scalable solution for extensions along with providing a layer of isolation for extensions as they are hosted in a remote node and integrated during runtime by avoiding server restart.

[seanneumann]

Related Hosting #40

[AMoo-Miki]

How about "Run extensions as part of OpenSearch Dashboard’s process" but without the downtime during install and uninstall? Instead of plugins calling PluginsService.setup, OSD will call PluginsService.setup for them. The extensions will have a config file that contains all the things they need to be set up correctly and OSD will read that config and do all that is needed for them.

If we find that sharing memory among OSD and extensions is a problem or we are worried that an extension crashing OSD, we could go with discreet micro-applications which are executed as child processes of OSD and communication happens over IPC. This differs from the "Micro-frontend" proposal in 2 ways: (1) the extensions are executed as children of OSD and OSD can respawn them if they crash (2) "Extensions can deploy independently from OSD" cannot happen.

The drawback of micro-applications is that encoding and decoding messages will impact performance and that impact might limit what all we would like to let extensions do.

[saratvemulapalli]

Run extensions as part of OpenSearch Dashboard’s process which is similar to current plugin architecture which involves extension discovery, setup dependencies, establish routes for all the bundles, bootstrapping, loading it to the browser and rendering it. (Recommended)

Trying to understand, for this approach would it mean when the extension goes down it takes down OSD with it, since its running on the same process?
This wouldn't really help if we would like to run tons of extensions without impacting the core functionality.

Second question, did we think about security concerns with installing extensions which could run malicious code with any of these approaches?

[kavilla]

• Track latency
• Optimization on global shared dep (use major)