This guide applies to all development within the OpenSearch Dashboards project and is recommended for the development of all OpenSearch Dashboards plugins.
This guide is for any developer who wants a running local development environment where you can make, see, and test changes. It's opinionated to get you running as quickly and easily as possible, but it's not the only way to set up a development environment.
If you're only interested in installing and running this project, please see the Installing OpenSearch Dashboards instead.
If you're planning to contribute code (features or fixes) to this repository, great! Make sure to also read the contributing guide.
OpenSearch Dashboards is primarily a Node.js web application built using React. To effectively contribute you should be familiar with HTML (usage guide), SASS styling (usage guide), TypeScript and JavaScript (usage guide), and React (usage guide).
To develop on OpenSearch Dashboards, you'll need:
- A GitHub account
gitfor version controlNode.js,npm, andYarnfor building and running the project- A code editor of your choice, configured for JavaScript/TypeScript. If you don't have a favorite editor, we suggest Visual Studio Code
If you already have these installed or have your own preferences for installing them, skip ahead to the Fork and clone OpenSearch Dashboards section.
If you don't already have it installed (check with git --version) we recommend following the the git installation guide for your OS.
We recommend using Node Version Manager (nvm) to install and manage different node versions, which may differ between release branches.
- Install nvm (as specified by the
nvmREADME):curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash - Install the version of the Node.js runtime defined in
.nvmrc:nvm install
If it's the only version of node installed, it will automatically be set to the default alias. Otherwise, use nvm list to see all installed node versions, and nvm use to select the node version required by OpenSearch Dashboards.
All local development should be done in a forked repository. Fork OpenSearch Dashboards by clicking the "Fork" button at the top of the GitHub repository.
Clone your forked version of OpenSearch Dashboards to your local machine (replace opensearch-project in the command below with your GitHub username):
$ git clone [email protected]:opensearch-project/OpenSearch-Dashboards.gitOpenSearch Dashboards is set up using yarn, which can be installed through corepack. To install yarn, run:
$ # Update corepack to the latest version
$ npm i -g corepack
$ # Install the correct version of yarn
$ corepack install(See the corepack documentation for more information.)
If you haven't already, change directories to your cloned repository directory:
$ cd OpenSearch-DashboardsThe osd bootstrap command will install the project's dependencies and build all internal packages and plugins. Bootstrapping is necessary any time you need to update packages, plugins, or dependencies, and it's recommended to run it anytime you sync with the latest upstream changes.
$ yarn osd bootstrapNote: If you experience a network timeout while bootstrapping:
| There appears to be trouble with your network connection. Retrying...
You can run command with —network-timeout flag:
$ yarn osd bootstrap —network-timeout 1000000
Or use the timeout by configuring it in the .yarnrc. For example:
network-timeout 1000000
If you've previously bootstrapped the project and need to start fresh, first run:
$ yarn osd cleanOpenSearch Dashboards requires a running version of OpenSearch to connect to. In a separate terminal you can run the latest snapshot built using:
(Linux, Windows, Darwin (MacOS) only - for others, you'll need to set up using Docker or run OpenSearch from a tarball instead)
$ yarn opensearch snapshotWarning: Starting the OpenSearch Dashboards instance before the OpenSearch server is fully initialized can cause Dashboards to misbehave. Ensure that the OpenSearch server instance is up and running first. You can validate by running curl localhost:9200 in another console tab or window (see OpenSearch developer guide).
Start the OpenSearch Dashboards development server:
$ yarn startWhen the server is up and ready (the console messages will look something like this),
[info][listening] Server running at http://localhost:5603/pgt
[info][server][OpenSearchDashboards][http] http server running at http://localhost:5603/pgt
click on the link displayed in your terminal to access it.
Note - it may take a couple minutes to generate all the necessary bundles. If the Dashboards link is not yet accessible, wait for a log message like
[success][@osd/optimizer] 28 bundles compiled successfully after 145.9 sec, watching for changes
Note: If you run a docker image, an error may occur:
Error: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
This error is because there is not enough memory so more memory must be allowed to be used:
$ sudo sysctl -w vm.max_map_count=262144
For windows:
$ wsl -d docker-desktop
$ sysctl -w vm.max_map_count=262144
Now that you have a development environment to play with, there are a number of different paths you may take next.
- Introduction to OpenSearch Dashboards Plugins blog post
- OpenSearch Dashboards plugin user documentation (install, update, and remove)
- Much of the technical architectural information about the plugin system is located in
/src/core
- The Quickstart guide for OpenSearch Dashboards will show you how to explore, inspect, and visualize sample data
- Running queries in the Dev Tools Console provides helpful guidance on how to interact with OpenSearch
The easiest way to understand some of the essential plugins and APIs is to run OpenSearch Dashboards with developer examples turned on:
$ yarn start --run-examplesSee the communication guidefor information on how to join our slack workspace, forum, or developer office hours.
Although the getting started guide covers the recommended development environment setup, there are several alternatives worth being aware of.
By default, the snapshot command will run a minimal distribution of OpenSearch, with no plugins installed.
If you would like to run OpenSearch with a particular plugin installed on the cluster snapshot, pass the --P flag after yarn opensearch snapshot. You can use the flag multiple times to install multiple plugins. The argument value can be a URL to the plugin's zip file, maven coordinates of the plugin, or a local zip file path (use file:// followed by the absolute or relative path, in that case). For example:
(Linux, Windows, Darwin (MacOS) only - for others, you'll need to run OpenSearch from a tarball instead)
$ yarn opensearch snapshot --P https://repo1.maven.org/maven2/org/opensearch/plugin/opensearch-test-plugin/2.4.0.0/opensearch-test-plugin-2.4.0.0.zipNote - if you add the security plugin, you'll also need to configure OpenSearch Dashboards for security.
The osd-plugin-generator tool makes it easier to create a plugin for OpenSearch Dashboards. It sets up the basic structure of the project and provides scripts to build it. Refer to osd-plugin-generator for more details.
Additional options can be passed after yarn opensearch snapshot to further configure the cluster snapshot.
Options:
--license Run with a 'oss', 'basic', or 'trial' license [default: oss]
--version Version of OpenSearch to download [default: 3.0.0}]
--base-path Path containing cache/installations [default: /home/ubuntu/OpenSearch-Dashboards/.opensearch]
--install-path Installation path, defaults to 'source' within base-path
--data-archive Path to zip or tarball containing an OpenSearch data directory to seed the cluster with.
--password Sets password for opensearch user [default: changeme]
-E Additional key=value settings to pass to OpenSearch
--download-only Download the snapshot but don't actually start it
--ssl Sets up SSL on OpenSearch
--security Installs and sets up OpenSearch Security plugin on the cluster
--P OpenSearch plugin artifact URL to install it on the cluster.
$ yarn opensearch snapshot --version 2.2.0 -E cluster.name=test -E path.data=/tmp/opensearch-data --P org.opensearch.plugin:test-plugin:2.2.0.0 --P file:/home/user/opensearch-test-plugin-2.2.0.0.zip --securityThis feature will only work if you have the security plugin installed on your OpenSearch cluster with https/authentication enabled.
Please follow the design described in the docs
If you would like to run OpenSearch from the tarball, you'll need to download the minimal distribution, install it, and then run the executable. (You'll also need Java installed and the JAVA_HOME environmental variable set - see OpenSearch developer guide for details).
- Download the latest minimal distribution of OpenSearch from the downloads page. Note the version and replace in commands below.
- Unzip the
tar.gzfile:tar -xvf opensearch-<OpenSearch-version>-linux-x64.tar.gz - Change directory:
cd opensearch-<OpenSearch-version> - Run the cluster:
./bin/opensearch
Note - OpenSearch and OpenSearch Dashboards must have matching version strings. Because the tarball is the latest released version of OpenSearch, it's likely behind the version on the main branch of OpenSearch Dashboards, which is generally set to the next upcoming major release. So to work from main, update the OpenSearch Dashboards version in package.json to match the OpenSearch version running.
This method can also be used to develop against the full distribution of OpenSearch instead. In that case, you'll also need to configure OpenSearch Dashboards for security.
This step is only needed if you want your dev environment to also start with security. To do so both the OpenSearch node and OpenSearch Dashboards cluster need to have the security plugin installed. Follow the steps below to get setup correctly.
To startup the OpenSearch snapshot with security
OpenSearch has strong password requirements and will fail to bootstrap if the password requirements are not met. e.g. myStrongPassword123!
- Run
export OPENSEARCH_INITIAL_ADMIN_PASSWORD=<initial admin password>since it's needed by the configuration script - Run
yarn opensearch snapshot --security - Wait a few seconds while the plugin is installed, configured, and OpenSearch starts up.
Then within another window you can start OpenSearch Dashboards:
First make sure to clone the https://github.com/opensearch-project/security-dashboards-plugin repo into the plugins folder and build it (Using yarn build). You can follow the instructions here https://github.com/opensearch-project/security-dashboards-plugin/blob/main/DEVELOPER_GUIDE.md#install-opensearch-dashboards-with-security-dashboards-plugin.
You do not have to edit the
config/opensearch-dashboards.ymlfile since theyarn start:securitycommand sets up the default overrides automatically
Then do the following:
- Run
yarn start:security - Navigate to OpenSearch Dashboards and login with the username
adminand password<initial admin password>.
For more detailed documentation, see Configure TLS for OpenSearch Dashboards.
To build the artifacts for all supported platforms, run the following:
yarn build --skip-os-packages
If you want to build a specific platform, pass the platform flag after yarn build-platform. For example, to build darwin x64, run the following:
yarn build-platform --darwin
You could pass one or multiple flags. If you don't pass any flag, yarn build-platform will build an artifact based on your local environment.
Currently, the supported flags for this script are:
darwin(builds Darwin x64)linux(builds Linux x64)linux-arm(builds Linux ARM64).windows(builds Windows x64)
If you would like to build only a DEB x64 artifact, run the following:
yarn build --deb --skip-archives
If you would like to build only a DEB ARM64 artifact, run the following:
yarn build --deb-arm --skip-archives
If you would like to build only a RPM x64 artifact, run the following:
yarn build --rpm --skip-archives
If you would like to build only a RPM ARM64 artifact, run the following:
yarn build --rpm-arm --skip-archives
To build the Docker image, run the following:
yarn osd bootstrap
yarn build --docker
All filenames should use snake_case.
Right: src/opensearch-dashboards/index_patterns/index_pattern.js
Wrong: src/opensearch-dashboards/IndexPatterns/IndexPattern.js
We use a version management system. If a line of code is no longer needed, remove it, don't simply comment it out.
We are gradually moving the OpenSearch Dashboards code base over to Prettier. All TypeScript code
and some JavaScript code (check .eslintrc.js) is using Prettier to format code. You
can run node script/eslint --fix to fix linting issues and apply Prettier formatting.
We recommend you to enable running ESLint via your IDE.
Whenever possible we are trying to use Prettier and linting over written developer guide rules. Consider every linting rule and every Prettier rule to be also part of our developer guide and disable them only in exceptional cases and ideally leave a comment why they are disabled at that specific place.
This part contains developer guide rules around general (framework agnostic) HTML usage.
Use camel case for the values of attributes such as id and data-test-subj selectors.
<button id="veryImportantButton" data-test-subj="clickMeButton">Click me</button>The only exception is in cases where you're dynamically creating the value, and you need to use hyphens as delimiters:
buttons.map(btn => (
<button
id={`veryImportantButton-${btn.id}`}
data-test-subj={`clickMeButton-${btn.id}`}
>
{btn.label}
</button>
)It's important that when you write CSS/SASS selectors using classes, IDs, and attributes (keeping in mind that we should never use IDs and attributes in our selectors), that the capitalization in the CSS matches that used in the HTML. HTML and CSS follow different case sensitivity rules, and we can avoid subtle gotchas by ensuring we use the same capitalization in both of them.
When labeling elements (and for some other accessibility tasks) you will often need ids. Ids must be unique within the page i.e. no duplicate ids in the rendered DOM at any time.
Since we have some components that are used multiple times on the page, you must
make sure every instance of that component has a unique id. To make the generation
of those ids easier, you can use the htmlIdGenerator service in the @elastic/eui.
A React component could use it as follows:
import { htmlIdGenerator } from '@elastic/eui';
render() {
// Create a new generator that will create ids deterministic
const htmlId = htmlIdGenerator();
return (<div>
<label htmlFor={htmlId('agg')}>Aggregation</label>
<input id={htmlId('agg')}/>
</div>);
}Each id generator you create by calling htmlIdGenerator() will generate unique but
deterministic ids. As you can see in the above example, that single generator
created the same id in the label's htmlFor as well as the input's id.
A single generator instance will create the same id when passed the same argument to the function multiple times. But two different generators will produce two different ids for the same argument to the function, as you can see in the following example:
const generatorOne = htmlIdGenerator();
const generatorTwo = htmlIdGenerator();
// Those statements are always true:
// Same generator
generatorOne('foo') === generatorOne('foo');
generatorOne('foo') !== generatorOne('bar');
// Different generator
generatorOne('foo') !== generatorTwo('foo');This allows multiple instances of a single React component to now have different ids. If you include the above React component multiple times in the same page, each component instance will have a unique id, because each render method will use a different id generator.
You can also use this service outside of React.
When writing a new component, create a sibling SASS file of the same name and import directly into the top of the JS/TS component file. Doing so ensures the styles are never separated or lost on import and allows for better modularization (smaller individual plugin asset footprint).
All SASS (.scss) files will automatically build with the OUI & OpenSearch Dashboards invisibles (SASS variables, mixins, functions) from the globals_[theme].scss file.
While the styles for this component will only be loaded if the component exists on the page, the styles will be global and so it is recommended to use a three letter prefix on your classes to ensure proper scope.
Example:
// component.tsx
import './component.scss';
// All other imports below the SASS import
export const Component = () => {
return <div className="plgComponent" />;
};// component.scss
.plgComponent { ... }Do not use the underscore _ SASS file naming pattern when importing directly into a javascript file.
The following developer guide rules apply for working with TypeScript/JavaScript files.
Whenever possible, write code in TypeScript instead of JavaScript, especially if it's new code. Check out TYPESCRIPT.md for help with this process.
You should prefer modern language features in a lot of cases, e.g.:
- Prefer
classoverprototypeinheritance - Prefer arrow function over function expressions
- Prefer arrow function over storing
this(noconst self = this;) - Prefer template strings over string concatenation
- Prefer the spread operator for copying arrays (
[...arr]) overarr.slice() - Use optional chaining (
?.) and nullish Coalescing (??) overlodash.get(and similar utilities)
Wherever possible, do not rely on mutable state. This means you should not reassign variables, modify object properties, or push values to arrays. Instead, create new variables, and shallow copies of objects and arrays:
// good
function addBar(foos, foo) {
const newFoo = { ...foo, name: 'bar' };
return [...foos, newFoo];
}
// bad
function addBar(foos, foo) {
foo.name = 'bar';
foos.push(foo);
}Since TypeScript 3.0 and the introduction of the
unknown type there are rarely any
reasons to use any as a type. Nearly all places of former any usage can be replace by either a
generic or unknown (in cases the type is really not known).
You should always prefer using those mechanisms over using any, since they are stricter typed and
less likely to introduce bugs in the future due to insufficient types.
If you’re not having any in your plugin or are starting a new plugin, you should enable the
@typescript-eslint/no-explicit-any
linting rule for your plugin via the .eslintrc.js config.
You should try avoiding non-null assertions (!.) wherever possible. By using them you tell
TypeScript, that something is not null even though by it’s type it could be. Usage of non-null
assertions is most often a side-effect of you actually checked that the variable is not null
but TypeScript doesn’t correctly carry on that information till the usage of the variable.
In most cases it’s possible to replace the non-null assertion by structuring your code/checks slightly different or using user defined type guards to properly tell TypeScript what type a variable has.
Using non-null assertion increases the risk for future bugs. In case the condition under which we assumed that the variable can’t be null has changed (potentially even due to changes in completely different files), the non-null assertion would now wrongly disable proper type checking for us.
If you’re not using non-null assertions in your plugin or are starting a new plugin, consider enabling the
@typescript-eslint/no-non-null-assertion
linting rule for you plugin in the .eslintrc.js config.
To avoid deep nesting of if-statements, always return a function's value as early as possible. And where possible, do any assertions first:
// good
function doStuff(val) {
if (val > 100) {
throw new Error('Too big');
}
if (val < 0) {
return false;
}
// ... stuff
}
// bad
function doStuff(val) {
if (val >= 0) {
if (val < 100) {
// ... stuff
} else {
throw new Error('Too big');
}
} else {
return false;
}
}This helps avoid temporary references and helps prevent typo-related bugs.
// best
function fullName({ first, last }) {
return `${first} ${last}`;
}
// good
function fullName(user) {
const { first, last } = user;
return `${first} ${last}`;
}
// bad
function fullName(user) {
const first = user.first;
const last = user.last;
return `${first} ${last}`;
}Directly accessing array values via index should be avoided, but if it is necessary, use array destructuring:
const arr = [1, 2, 3];
// good
const [first, second] = arr;
// bad
const first = arr[0];
const second = arr[1];These are numbers (or other values) simply used in line in your code. Do not use these, give them a variable name so they can be understood and changed easily.
// good
const minWidth = 300;
if (width < minWidth) {
...
}
// bad
if (width < 300) {
...
}Module dependencies should be written using native ES2015 syntax wherever possible (which is almost everywhere):
// good
import { mapValues } from 'lodash';
export mapValues;
// bad
const _ = require('lodash');
module.exports = _.mapValues;
// worse
define(['lodash'], function (_) {
...
});In those extremely rare cases where you're writing server-side JavaScript in a file that does not pass run through webpack, then use CommonJS modules.
In those even rarer cases where you're writing client-side code that does not run through webpack, then do not use a module loader at all.
The files inside a module are implementation details of that module. They should never be imported directly. Instead, you must only import the top-level API that's exported by the module itself.
Without a clear mechanism in place in JS to encapsulate protected code, we make a broad assumption that anything beyond the root of a module is an implementation detail of that module.
On the other hand, a module should be able to import parent and sibling modules.
// good
import foo from 'foo';
import child from './child';
import parent from '../';
import ancestor from '../../../';
import sibling from '../foo';
// bad
import inFoo from 'foo/child';
import inSibling from '../foo/child';Don't do this. Everything should be wrapped in a module that can be depended on by other modules. Even things as simple as a single value should be a module.
And never use multiple ternaries together, because they make it more difficult to reason about how different values flow through the conditions involved. Instead, structure the logic for maximum readability.
// good, a situation where only 1 ternary is needed
const foo = a === b ? 1 : 2;
// bad
const foo = a === b ? 1 : a === c ? 2 : 3;Any non-trivial conditions should be converted to functions or assigned to descriptively named variables. By breaking up logic into smaller, self-contained blocks, it becomes easier to reason about the higher-level logic. Additionally, these blocks become good candidates for extraction into their own modules, with unit-tests.
// best
function isShape(thing) {
return thing instanceof Shape;
}
function notSquare(thing) {
return !(thing instanceof Square);
}
if (isShape(thing) && notSquare(thing)) {
...
}
// good
const isShape = thing instanceof Shape;
const notSquare = !(thing instanceof Square);
if (isShape && notSquare) {
...
}
// bad
if (thing instanceof Shape && !(thing instanceof Square)) {
...
}// good
const validPassword = /^(?=.*\d).{4,}$/;
if (password.length >= 4 && validPassword.test(password)) {
console.log('password is valid');
}
// bad
if (password.length >= 4 && /^(?=.*\d).{4,}$/.test(password)) {
console.log('losing');
}Keep your functions short. A good function fits on a slide that the people in the last row of a big room can comfortably read. So don't count on them having perfect vision and limit yourself to ~15 lines of code per function.
For expressiveness sake, and so you can be mix dynamic and explicit arguments.
// good
function something(foo, ...args) {
...
}
// bad
function something(foo) {
const args = Array.from(arguments).slice(1);
...
}Always use the default argument syntax for optional arguments.
// good
function foo(options = {}) {
...
}
// bad
function foo(options) {
if (typeof options === 'undefined') {
options = {};
}
...
}And put your optional arguments at the end.
// good
function foo(bar, options = {}) {
...
}
// bad
function foo(options = {}, bar) {
...
}For trivial examples (like the one that follows), thunks will seem like overkill, but they encourage isolating the implementation details of a closure from the business logic of the calling code.
// good
function connectHandler(client, callback) {
return () => client.connect(callback);
}
setTimeout(connectHandler(client, afterConnect), 1000);
// not as good
setTimeout(() => {
client.connect(afterConnect);
}, 1000);
// bad
setTimeout(() => {
client.connect(() => {
...
});
}, 1000);Use slashes for both single line and multi line comments. Try to write comments that explain higher level mechanisms or clarify difficult segments of your code. Don't use comments to restate trivial things.
Exception: Comment blocks describing a function and its arguments
(docblock) should start with /**, contain a single * at the beginning of
each line, and end with */.
// good
// 'ID_SOMETHING=VALUE' -> ['ID_SOMETHING=VALUE', 'SOMETHING', 'VALUE']
const matches = item.match(/ID_([^\n]+)=([^\n]+)/));
/**
* Fetches a user from...
* @param {string} id - id of the user
* @return {Promise}
*/
function loadUser(id) {
// This function has a nasty side effect where a failure to increment a
// redis counter used for statistics will cause an exception. This needs
// to be fixed in a later iteration.
...
}
const isSessionValid = (session.expires < Date.now());
if (isSessionValid) {
...
}
// bad
// Execute a regex
const matches = item.match(/ID_([^\n]+)=([^\n]+)/));
// Usage: loadUser(5, function() { ... })
function loadUser(id, cb) {
// ...
}
// Check if the session is valid
const isSessionValid = (session.expires < Date.now());
// If the session is valid
if (isSessionValid) {
...
}Feel free to use getters that are free from side effects, like providing a length property for a collection class.
Do not use setters, they cause more problems than they can solve.
Parts of the JavaScript developer guide were initially forked from the node style guide created by Felix Geisendörfer which is licensed under the CC BY-SA 3.0 license.
The following developer guide rules are specific for working with the React framework.
Name action functions in the form of a strong verb and passed properties in the form of on. E.g:
<sort-button onClick={action.sort}/>
<pagerButton onPageNext={action.turnToNextPage} />The following developer guide rules are targeting development of server side API endpoints.
API routes must start with the /api/ path segment, and should be followed by the plugin id if applicable:
Right: /api/marvel/nodes
Wrong: /marvel/api/nodes
OpenSearch Dashboards uses snake_case for the entire API, just like OpenSearch. All urls, paths, query string parameters, values, and bodies should be snake_case formatted.
Right:
POST /api/opensearch-dashboards/index_patterns
{
"id": "...",
"time_field_name": "...",
"fields": [
...
]
}