Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Rename] kbn-pm in packages directory to osd-pm #65

Merged
merged 3 commits into from
Mar 12, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
File renamed without changes.
130 changes: 65 additions & 65 deletions packages/kbn-pm/README.md → packages/osd-pm/README.md
Original file line number Diff line number Diff line change
@@ -1,146 +1,146 @@
# `@kbn/pm` — The Kibana project management tool
# `@osd/pm` — The OpenSearch Dashboards project management tool

`@kbn/pm` is a project management tool inspired by Lerna, which enables sharing
code between Kibana and Kibana plugins.
`@osd/pm` is a project management tool inspired by Lerna, which enables sharing
code between OpenSearch Dashboards and OpenSearch Dashboards plugins.

To run `@kbn/pm`, go to Kibana root and run `yarn kbn`.
To run `@osd/pm`, go to OpenSearch Dashboards root and run `yarn osd`.

## Why `@kbn/pm`?
## Why `@osd/pm`?

Long-term we want to get rid of Webpack from production (basically, it's causing
a lot of problems, using a lot of memory and adding a lot of complexity).
Ideally we want each plugin to build its own separate production bundles for
both server and UI. To get there all Kibana plugins (including x-pack) need to
be able to build their production bundles separately from Kibana, which means
they need to be able to depend on code from Kibana without `import`-ing random
files directly from the Kibana source code.
both server and UI. To get there all OpenSearch Dashboards plugins (including x-pack) need to
be able to build their production bundles separately from OpenSearch Dashboards , which means
they need to be able to depend on code from OpenSearch Dashboards without `import`-ing random
files directly from the OpenSearch Dashboards source code.

From a plugin perspective there are two different types of Kibana dependencies:
From a plugin perspective there are two different types of OpenSearch Dashboards dependencies:
runtime and static dependencies. Runtime dependencies are things that are
instantiated at runtime and that are injected into the plugin, for example
config and elasticsearch clients. Static dependencies are those dependencies
config and opensearch clients. Static dependencies are those dependencies
that we want to `import`. `elastic-eslint-config-kibana` is one example of this, and
it's actually needed because eslint requires it to be a separate package. But we
also have dependencies like `datemath`, `flot`, `eui` and others that we
control, but where we want to `import` them in plugins instead of injecting them
(because injecting them would be painful to work with). (Btw, these examples
aren't necessarily a part of the Kibana repo today, they are just meant as
aren't necessarily a part of the OpenSearch Dashboards repo today, they are just meant as
examples of code that we might at some point want to include in the repo while
having them be `import`able in Kibana plugins like any other npm package)
having them be `import`able in OpenSearch Dashboards plugins like any other npm package)

Another reason we need static dependencies is that we're starting to introduce
TypeScript into Kibana, and to work nicely with TypeScript across plugins we
TypeScript into OpenSearch Dashboards , and to work nicely with TypeScript across plugins we
need to be able to statically import dependencies. We have for example built an
observable library for Kibana in TypeScript and we need to expose both the
observable library for OpenSearch Dashboards in TypeScript and we need to expose both the
functionality and the TypeScript types to plugins (so other plugins built with
TypeScript can depend on the types for the lib).

However, even though we have multiple packages we don't necessarily want to
`npm publish` them. The ideal solution for us is being able to work on code
locally in the Kibana repo and have a nice workflow that doesn't require
locally in the OpenSearch Dashboards repo and have a nice workflow that doesn't require
publishing, but where we still get the value of having "packages" that are
available to plugins, without these plugins having to import files directly from
the Kibana folder.
the OpenSearch Dashboards folder.

Basically, we just want to be able to share "static code" (aka being able to
`import`) between Kibana and Kibana plugins. To get there we need tooling.
`import`) between OpenSearch Dashboards and OpenSearch Dashboards plugins. To get there we need tooling.

`@kbn/pm` is a tool that helps us manage these static dependencies, and it
enables us to share these packages between Kibana and Kibana plugins. It also
`@osd/pm` is a tool that helps us manage these static dependencies, and it
enables us to share these packages between OpenSearch Dashboards and OpenSearch Dashboards plugins. It also
enables these packages to have their own dependencies and their own build
scripts, while still having a nice developer experience.

## How it works

### Internal usage

For packages that are referenced within the Kibana repo itself (for example,
using the `@kbn/i18n` package from an `x-pack` plugin), we are leveraging
For packages that are referenced within the OpenSearch Dashboards repo itself (for example,
using the `@osd/i18n` package from an `x-pack` plugin), we are leveraging
Yarn's workspaces feature. This allows yarn to optimize node_modules within
the entire repo to avoid duplicate modules by hoisting common packages as high
in the dependency tree as possible.

To reference a package from within the Kibana repo, simply use the current
version number from that package's package.json file. Then, running `yarn kbn
To reference a package from within the OpenSearch Dashboards repo, simply use the current
version number from that package's package.json file. Then, running `yarn osd
bootstrap` will symlink that package into your dependency tree. That means
you can make changes to `@kbn/i18n` and immediately have them available
in Kibana itself. No `npm publish` needed anymore — Kibana will always rely
you can make changes to `@osd/i18n` and immediately have them available
in OpenSearch Dashboards itself. No `npm publish` needed anymore — OpenSearch Dashboards will always rely
directly on the code that's in the local packages.

### External Plugins

For external plugins, referencing packages in Kibana relies on
For external plugins, referencing packages in OpenSearch Dashboards relies on
`link:` style dependencies in Yarn. With `link:` dependencies you specify the
relative location to a package instead of a version when adding it to
`package.json`. For example:

```
"@kbn/i18n": "link:packages/kbn-i18n"
"@osd/i18n": "link:packages/osd-i18n"
```

Now when you run `yarn` it will set up a symlink to this folder instead of
downloading code from the npm registry. This allows external plugins to always
use the versions of the package that is bundled with the Kibana version they
use the versions of the package that is bundled with the OpenSearch Dashboards version they
are running inside of.

```
"@kbn/i18n": "link:../../kibana/packages/kbn-date-math"
"@osd/i18n": "link:../../opensearch-dashboards/packages/osd-date-math"
```

This works because we moved to a strict location of Kibana plugins,
`./plugins/{pluginName}` inside of Kibana, or `../kibana-extra/{pluginName}`
relative to Kibana. This is one of the reasons we wanted to move towards a setup
This works because we moved to a strict location of OpenSearch Dashboards plugins,
`./plugins/{pluginName}` inside of OpenSearch Dashboards , or `../opensearch-dashboards-extra/{pluginName}`
relative to OpenSearch Dashboards . This is one of the reasons we wanted to move towards a setup
that looks like this:

```
elastic
└── kibana
opensearch
└── opensearch-dashboards
└── plugins
├── kibana-canvas
└── x-pack-kibana
├── opensearch-dashboards-canvas
└── x-pack-opensearch-dashboards
```

Relying on `link:` style dependencies means we no longer need to `npm publish`
our Kibana specific packages. It also means that plugin authors no longer need
to worry about the versions of the Kibana packages, as they will always use the
packages from their local Kibana.
our OpenSearch Dashboards specific packages. It also means that plugin authors no longer need
to worry about the versions of the OpenSearch Dashboards packages, as they will always use the
packages from their local OpenSearch Dashboards .

## The `kbn` use-cases
## The `osd` use-cases

### Bootstrapping

Now, instead of installing all the dependencies with just running `yarn` you use
the `@kbn/pm` tool, which can install dependencies (and set up symlinks) in
the `@osd/pm` tool, which can install dependencies (and set up symlinks) in
all the packages using one command (aka "bootstrap" the setup).

To bootstrap Kibana:
To bootstrap OpenSearch Dashboards :

```
yarn kbn bootstrap
yarn osd bootstrap
```

By default, `@kbn/pm` will bootstrap all packages within Kibana, plus all
Kibana plugins located in `./plugins` or `../kibana-extra`. There are several
options for skipping parts of this, e.g. to skip bootstrapping of Kibana
By default, `@osd/pm` will bootstrap all packages within OpenSearch Dashboards , plus all
OpenSearch Dashboards plugins located in `./plugins` or `../opensearch-dashboards-extra`. There are several
options for skipping parts of this, e.g. to skip bootstrapping of OpenSearch Dashboards
plugins:

```
yarn kbn bootstrap --skip-kibana-plugins
yarn osd bootstrap --skip-opensearch-dashboards-plugins
```

Or just skip few selected packages:

```
yarn kbn bootstrap --exclude @kbn/pm --exclude @kbn/i18n
yarn osd bootstrap --exclude @osd/pm --exclude @osd/i18n
```

For more details, run:

```
yarn kbn
yarn osd
```

Bootstrapping also calls the `kbn:bootstrap` script for every included project.
Bootstrapping also calls the `osd:bootstrap` script for every included project.
This is intended for packages that need to be built/transpiled to be usable.

### Running scripts
Expand All @@ -150,52 +150,52 @@ e.g. `build` or `test`. Instead of jumping into each package and running
`yarn build` you can run:

```
yarn kbn run build
yarn osd run build
```

And if needed, you can skip packages in the same way as for bootstrapping, e.g.
with `--exclude` and `--skip-kibana-plugins`:
with `--exclude` and `--skip-opensearch-dashboards-plugins`:

```
yarn kbn run build --exclude kibana
yarn osd run build --exclude opensearch-dashboards
```

### Watching

During development you can also use `kbn` to watch for changes. For this to work
package should define `kbn:watch` script in the `package.json`:
During development you can also use `osd` to watch for changes. For this to work
package should define `osd:watch` script in the `package.json`:

```
yarn kbn watch
yarn osd watch
```

By default `kbn watch` will sort all packages within Kibana into batches based on
By default `osd watch` will sort all packages within OpenSearch Dashboards into batches based on
their mutual dependencies and run watch script for all packages in the correct order.

As with any other `kbn` command, you can use `--include` and `--exclude` filters to watch
As with any other `osd` command, you can use `--include` and `--exclude` filters to watch
only for a selected packages:

```
yarn kbn watch --include @kbn/pm --include kibana
yarn osd watch --include @osd/pm --include opensearch-dashboards
```

## Building packages for production

The production build process relies on both the Grunt setup at the root of the
Kibana project and code in `@kbn/pm`. The full process is described in
OpenSearch Dashboards project and code in `@osd/pm`. The full process is described in
`tasks/build/packages.js`.

## Development

This package is run from Kibana root, using `yarn kbn`. This will run the
This package is run from OpenSearch Dashboards root, using `yarn osd`. This will run the
"pre-built" (aka built and committed to git) version of this tool, which is
located in the `dist/` folder. This will also use the included version of Yarn
instead of using your local install of Yarn.

If you need to build a new version of this package, run `yarn build` in this
folder.

Even though this file is generated we commit it to Kibana, because it's used
Even though this file is generated we commit it to OpenSearch Dashboards , because it's used
_before_ dependencies are fetched (as this is the tool actually responsible for
fetching dependencies).

Expand All @@ -217,7 +217,7 @@ something we want to do).

#### `file:` dependencies in npm<5 and in yarn

When you add a dependency like `"foo": "file:../../kibana/packages/foo"`, both
When you add a dependency like `"foo": "file:../../opensearch-dashboards/packages/foo"`, both
npm<5 and yarn copies the files into the `node_modules` folder. This means you
can't easily make changes to the plugin while developing. Therefore this is a
no-go.
Expand Down Expand Up @@ -247,7 +247,7 @@ dependencies for these workspaces and hoist the dependencies to the root (to
> filesystem hierarchy.

So Yarn workspaces requires a shared root, which (at least currently) doesn't
fit Kibana, and it's therefore a no-go for now.
fit OpenSearch Dashboards , and it's therefore a no-go for now.

#### Lerna

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
10 changes: 5 additions & 5 deletions packages/kbn-pm/package.json → packages/osd-pm/package.json
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
{
"name": "@kbn/pm",
"name": "@osd/pm",
"main": "./dist/index.js",
"version": "1.0.0",
"license": "Apache-2.0",
"private": true,
"kibana": {
"opensearchDashboards": {
"devOnly": true
},
"scripts": {
"build": "webpack",
"kbn:watch": "webpack --watch --progress",
"osd:watch": "webpack --watch --progress",
"prettier": "prettier --write './src/**/*.ts'"
},
"devDependencies": {
Expand All @@ -35,7 +35,7 @@
"@types/strong-log-transformer": "^1.0.0",
"@types/tempy": "^0.2.0",
"@types/write-pkg": "^3.1.0",
"@kbn/dev-utils": "1.0.0",
"@osd/dev-utils": "1.0.0",
"@yarnpkg/lockfile": "^1.1.0",
"babel-loader": "^8.0.6",
"chalk": "^4.1.0",
Expand Down Expand Up @@ -69,7 +69,7 @@
"write-pkg": "^4.0.0"
},
"dependencies": {
"@kbn/utils": "1.0.0",
"@osd/utils": "1.0.0",
"tslib": "^2.0.0"
}
}
Loading