[Discuss] Ideal domain hierarchy and folder structure

[stacey-gammon]

Our Kibana Developer Principles mention "Organize by domain":

Organize by domain
The Kibana codebase should be organized roughly into software domains, which is to say that modules are defined based on their role within the product rather than by the technical type of code that comprise them. In practice the two highest technical levels we use to express domains are that of core and plugins, but even within these modules we seek to organize the code by more specific domains.

Within core itself we express domains as services such as HTTP, config, elasticsearch, etc. These things are essential elements to constructing most useful features of Kibana, which is why they belong to the "core" domain.

Each plugin is one top-level domain of the Kibana project, and will generally be constructed by many sub-domains within. In this way, plugins are modeled in the same spirit as core itself.

Be cautious when adding new domains as doing so inherently expands the scope of the parent domain, which in the case of core or plugin boundaries means the scope of Kibana itself.

Code should be organized by domain up until you've reached a level of granularity in a module where it is no longer practical.

This does mention that developers should be cautious when adding new domains, especially at the top level (in other words, be cautious when adding new plugins), but I am frequently coming across situations where developers are naturally driven to create smaller, more isolated plugins, rather than nested services under a larger parent domain (myself included!).

I'd like to discuss some of these examples and come up with some specific recommendations.

Before doing so, it is important to consider the goal of this principle:

Make it as easy as possible for developers to find the code they are looking for.

To that aim, we'd like to:

• avoid too many top level folders that make it cumbersome to scan through quickly.
• use obvious, unambiguous folder names (I know, naming is hard!).

Top level domains have some special attributes in that they can be disabled, but don't create a new plugin just because it is possible for it to work on its own. Don't split one plugin into two for this reason unless there is a known use case for users wanting some services on, and the other services off. Also, don't forget about feature controls. This is the preferred way to turn on and off services/apps inside a plugin, rather than the more drastic measure of disabling an entire plugin.

Related

This issue is quite related to #47065 but that issue is less focused and harder to follow for someone just looking for a recommendation on where they should put their functionality and code.

Specific examples

Actions, triggers, alerts

• src
  • ui_actions
• xpack
  • advanced_ui_actions
  • actions
  • alerting
  • triggers_actions_ui (maybe eventually trigger_actions_management)
  • event_log

Security, SIEM, endpoint

• xpack
  • security
  • spaces
  • features
  • siem
  • licensing
  • encypted_saved_objects
  • endpoint

Elasticsearch management stuff

• src
  • dev_tools
  • console (aren't dev tools and console the same thing??)
• xpack
  • remote_cluster
  • rollup
  • searchprofiler
  • es_ui_shared
  • console_extensions

Visualizations

• src
  • vis_type_vega
  • vis_type_timeseries
  • visualizations
  • inspector
  • lens

Miscellaneous ones:

• src
  • share (module for registering share context menus.). Could probably be replaced by an action.
  • newsfeed
  • navigation
  • advanced_settings
  • testbed

telemetry vs usage_collection

bfetch -> data

dashboard_embeddable_container -> dashboard

• data_enchanced?
  • search
    • sql?
    • eql?

Some thoughts

• We're never going to find the perfect folder structure. If we do, it will degrade to "not perfect" in no time at all. Folder structure/organization is about as hard as naming, if not more so. naming + location/path.

• Ideally we'd be able to make changes periodically without too much risk/effort.

• Things we can consider when coming up with more top level folders:

  • 3 + 1 messaging
  • Kibana left nav (e.g. "Analyze" group)

[pgayvallet]

Just FYI, #68811 has been merged. It is now (at least technically) possible to implement 'nested' or 'grouped' hierarchy for plugins.

[stacey-gammon]

Closing this in favor of @joshdover's #71566