Skip to content

Commit

Permalink
[Chore] --security for snapshot and OSD server (opensearch-project#…
Browse files Browse the repository at this point in the history
…5451)

Add the ability to run:
```
yarn opensearch snapshot --security
```
which will start the OpenSearch cluster with the security plugin.

And add the ability to run:
```
yarn start:security
```
which will start the OpenSearch Dashboards server if the security
plugin is available.

This is only intended for demo and local purposes.

Issue:
n/a

Signed-off-by: Kawika Avilla <[email protected]>
  • Loading branch information
kavilla authored Nov 30, 2023
1 parent dadfefa commit 2e7a2ba
Show file tree
Hide file tree
Showing 7 changed files with 118 additions and 3 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Add an achievement badger to the PR ([#3721](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3721))
- [CI] Enable inputs for manually triggered Cypress test jobs ([#5134](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5134))
- [CI] Replace usage of deprecated `set-output` in workflows ([#5340](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5340))
- [Chore] Add `--security` for `opensearch snapshot` and `opensearch_dashboards` to configure local setup with the security plugin ([#5451](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/5451))

### 📝 Documentation

Expand Down
15 changes: 14 additions & 1 deletion DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,10 +252,11 @@ Options:
-E Additional key=value settings to pass to OpenSearch
--download-only Download the snapshot but don't actually start it
--ssl Sets up SSL on OpenSearch
--security Installs and sets up OpenSearch Security plugin on the cluster
--P OpenSearch plugin artifact URL to install it on the cluster.

```bash
$ yarn opensearch snapshot --version 2.2.0 -E cluster.name=test -E path.data=/tmp/opensearch-data --P org.opensearch.plugin:test-plugin:2.2.0.0 --P file:/home/user/opensearch-test-plugin-2.2.0.0.zip
$ yarn opensearch snapshot --version 2.2.0 -E cluster.name=test -E path.data=/tmp/opensearch-data --P org.opensearch.plugin:test-plugin:2.2.0.0 --P file:/home/user/opensearch-test-plugin-2.2.0.0.zip --security
```

#### Read Only capabilities
Expand All @@ -281,6 +282,18 @@ This method can also be used to develop against the [full distribution of OpenSe

_This step is only mandatory if you have the [`security` plugin](https://github.com/opensearch-project/security) installed on your OpenSearch cluster with https/authentication enabled._

> 1. Run `export initialAdminPassword=<initial admin password>` since it's needed by the configuration script
> 2. Run `yarn opensearch snapshot --security`
> 3. Wait a few seconds while the plugin is installed, configured, and OpenSearch starts up.
Then within another window. You can start:

> 1. Run `export OPENSEARCH_USERNAME=admin`
> 2. Run `export OPENSEARCH_PASSWORD=<initial admin password>`
> 3. Optional: Run `export OPENSEARCH_SECURITY_READONLY_ROLE=<read only role>`
> 4. Run `yarn start:security`
> 5. Navigate to OpenSearch Dashboards and login with the above username and password.
Once the bootstrap of OpenSearch Dashboards is finished, you need to apply some
changes to the default [`opensearch_dashboards.yml`](https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/config/opensearch_dashboards.yml#L25-L72) in order to connect to OpenSearch.

Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
"build": "scripts/use_node scripts/build --all-platforms",
"start": "scripts/use_node scripts/opensearch_dashboards --dev",
"start:docker": "scripts/use_node scripts/opensearch_dashboards --dev --opensearch.hosts=$OPENSEARCH_HOSTS --opensearch.ignoreVersionMismatch=true --server.host=$SERVER_HOST",
"start:security": "scripts/use_node scripts/opensearch_dashboards --dev --security",
"debug": "scripts/use_node --nolazy --inspect scripts/opensearch_dashboards --dev",
"debug-break": "scripts/use_node --nolazy --inspect-brk scripts/opensearch_dashboards --dev",
"lint": "yarn run lint:es && yarn run lint:style",
Expand Down
7 changes: 7 additions & 0 deletions packages/osd-opensearch/src/cli_commands/snapshot.js
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ exports.help = (defaults = {}) => {
-E Additional key=value settings to pass to OpenSearch
--download-only Download the snapshot but don't actually start it
--ssl Sets up SSL on OpenSearch
--security Installs and sets up the OpenSearch Security plugin on the cluster
--P OpenSearch plugin artifact URL to install it on the cluster. We can use the flag multiple times
to install multiple plugins on the cluster snapshot. The argument value can be url to zip file, maven coordinates of the plugin
or for local zip files, use file:<followed by the absolute or relative path to the plugin zip file>.
Expand All @@ -74,6 +75,8 @@ exports.run = async (defaults = {}) => {

boolean: ['download-only'],

boolean: ['security'],

default: defaults,
});

Expand All @@ -91,6 +94,10 @@ exports.run = async (defaults = {}) => {
await cluster.installOpenSearchPlugins(installPath, options.opensearchPlugins);
}

if (options.security) {
await cluster.setupSecurity(installPath, options.version ?? defaults.version);
}

options.bundledJDK = true;

await cluster.run(installPath, options);
Expand Down
35 changes: 33 additions & 2 deletions packages/osd-opensearch/src/cluster.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,27 @@ const execa = require('execa');
const chalk = require('chalk');
const path = require('path');
const { downloadSnapshot, installSnapshot, installSource, installArchive } = require('./install');
const { OPENSEARCH_BIN, OPENSEARCH_PLUGIN } = require('./paths');
const { OPENSEARCH_BIN, OPENSEARCH_PLUGIN, OPENSEARCH_SECURITY_INSTALL } = require('./paths');
const { log: defaultLog, parseOpenSearchLog, extractConfigFiles, decompress } = require('./utils');
const { createCliError } = require('./errors');
const { promisify } = require('util');
const treeKillAsync = promisify(require('tree-kill'));
const { parseSettings, SettingsFilter } = require('./settings');
const { CA_CERT_PATH, OPENSEARCH_P12_PATH, OPENSEARCH_P12_PASSWORD } = require('@osd/dev-utils');
const readFile = util.promisify(fs.readFile);
const chmodAsync = util.promisify(fs.chmod);

const LATEST_ENGINE_PLUGIN_BASE_URL =
'https://ci.opensearch.org/ci/dbc/distribution-build-opensearch';

function generateEnginePluginUrl(version, plugin) {
const legacyVersion = `${version}.0`;
const [platform, type] =
process.platform === 'win32' ? ['windows', 'zip'] : [process.platform, 'tar'];
const arch = process.arch === 'arm64' ? 'arm64' : 'x64';

return `${LATEST_ENGINE_PLUGIN_BASE_URL}/${version}/latest/${platform}/${arch}/${type}/builds/opensearch/plugins/${plugin}-${legacyVersion}.zip`;
}

// listen to data on stream until map returns anything but undefined
const first = (stream, map) =>
Expand All @@ -57,9 +70,10 @@ const first = (stream, map) =>
});

exports.Cluster = class Cluster {
constructor({ log = defaultLog, ssl = false } = {}) {
constructor({ log = defaultLog, ssl = false, security = false } = {}) {
this._log = log;
this._ssl = ssl;
this._security = security;
this._caCertPromise = ssl ? readFile(CA_CERT_PATH) : undefined;
}

Expand Down Expand Up @@ -193,6 +207,23 @@ exports.Cluster = class Cluster {
}
}

/**
* Setups cluster with security demo configuration
*
* @param {string} installPath
* @property {String} version - version of OpenSearch
*/
async setupSecurity(installPath, version) {
const pluginUrl = generateEnginePluginUrl(version, 'opensearch-security');
await this.installOpenSearchPlugins(installPath, pluginUrl);
this._log.info('Setting up security');
const pluginPath = path.resolve(installPath, OPENSEARCH_SECURITY_INSTALL);
if (pluginPath) {
await chmodAsync(pluginPath, '755');
await execa(OPENSEARCH_SECURITY_INSTALL, ['-y', '-i', '-s'], { cwd: installPath });
}
}

/**
* Starts OpenSearch and returns resolved promise once started
*
Expand Down
7 changes: 7 additions & 0 deletions packages/osd-opensearch/src/paths.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ function maybeUseBat(bin) {
return os.platform().startsWith('win') ? `${bin}.bat` : bin;
}

function maybeUseBatOrShell(bin) {
return os.platform().startsWith('win') ? `${bin}.bat` : `${bin}.sh`;
}

const tempDir = os.tmpdir();

exports.BASE_PATH = path.resolve(tempDir, 'osd-opensearch');
Expand All @@ -45,3 +49,6 @@ exports.OPENSEARCH_CONFIG = 'config/opensearch.yml';

exports.OPENSEARCH_KEYSTORE_BIN = maybeUseBat('./bin/opensearch-keystore');
exports.OPENSEARCH_PLUGIN = maybeUseBat('./bin/opensearch-plugin');
exports.OPENSEARCH_SECURITY_INSTALL = maybeUseBatOrShell(
'./plugins/opensearch-security/tools/install_demo_configuration'
);
55 changes: 55 additions & 0 deletions src/cli/serve/serve.js
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,60 @@ function applyConfigOverrides(rawConfig, opts, extraCliOptions) {
set('opensearch.hosts', opensearchHosts);
set('opensearch.ssl.certificateAuthorities', CA_CERT_PATH);
}

if (opts.security) {
const customOpenSearchHosts = opts.opensearch
? opts.opensearch.split(',')
: [].concat(get('opensearch.hosts') || []);

const opensearchHosts = (
(customOpenSearchHosts.length > 0 && customOpenSearchHosts) || ['https://localhost:9200']
).map((hostUrl) => {
const parsedUrl = new URL('', hostUrl);
return `https://localhost:${parsedUrl.port}`;
});

if (!get('opensearch.hosts')) {
set('opensearch.hosts', opensearchHosts);
}

if (!get('opensearch.ssl.verificationMode')) {
set('opensearch.ssl.verificationMode', 'none');
}

if (get('opensearch.username') === 'opensearch_dashboards_system') {
set('opensearch.username', process.env.OPENSEARCH_USERNAME);
}

if (get('opensearch.password') === 'changeme') {
set('opensearch.password', process.env.OPENSEARCH_PASSWORD);
}

if (!get('opensearch.requestHeadersWhitelist')) {
set('opensearch.requestHeadersWhitelist', ['authorization', 'securitytenant']);
}

if (!get('opensearch_security.multitenancy.enabled')) {
set('opensearch_security.multitenancy.enabled', true);
}

if (!get('opensearch_security.multitenancy.tenants.preferred')) {
set('opensearch_security.multitenancy.tenants.preferred', ['Private', 'Global']);
}

if (
!get('opensearch_security.readonly_mode.roles') &&
process.env.OPENSEARCH_SECURITY_READONLY_ROLE
) {
set('opensearch_security.readonly_mode.roles', [
process.env.OPENSEARCH_SECURITY_READONLY_ROLE,
]);
}

if (!get('opensearch_security.cookie.secure')) {
set('opensearch_security.cookie.secure', false);
}
}
}

if (opts.opensearch) set('opensearch.hosts', opts.opensearch.split(','));
Expand Down Expand Up @@ -195,6 +249,7 @@ export default function (program) {
command
.option('--dev', 'Run the server with development mode defaults')
.option('--ssl', 'Run the dev server using HTTPS')
.option('--security', 'Run the dev server using security defaults')
.option('--dist', 'Use production assets from osd/optimizer')
.option(
'--no-base-path',
Expand Down

0 comments on commit 2e7a2ba

Please sign in to comment.