[6.x] Switch to Yarn (elastic#15485) (elastic#15955)

* Switch to Yarn (elastic#15485)

* switch to yarn

* cleanup misc references to npm

* [yarn] loosen dependency ranges so yarn will merge more deps

* fix linting error now that moment uses ESM

* [licenses] font-awesome changed the format of its license id

* Use local yarn

* Misc fixes

* eslintignore built yarn file

* Remove mkdir which doesn't do what it should do

* Check build without upgrading lots of versions

* Fix license check

* too many moments

* Better description

* Review fixes

* Lock to [email protected]

* More specific version locks

* Revert "More specific version locks"

This reverts commit 11ef811.

* Revert "Lock to [email protected]"

This reverts commit 3ade68c.

* rm yarn.lock; yarn

* Forcing a specific version of React, Angular, Moment

* Using vendored version of yarn in ci

* Use --frozen-lockfile

* fixes

* Update lockfile

.eslintignore

@@ -8,3 +8,4 @@
 /src/ui/public/utils/decode_geo_hash.js
 /src/core_plugins/timelion/public/webpackShims/jquery.flot.*
 /ui_framework/doc_site/build
+/tasks/vendor

.github/PULL_REQUEST_TEMPLATE.md

@@ -8,6 +8,6 @@ attention.
 - Have you signed the [contributor license agreement](https://www.elastic.co/contributor-agreement)?
 - Have you followed the [contributor guidelines](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md)?
 - If submitting code, have you included unit tests that cover the changes?
-- If submitting code, have you tested and built your code locally prior to submission with `npm test && npm run build`?
+- If submitting code, have you tested and built your code locally prior to submission with `yarn test && yarn build`?
 - If submitting code, is your pull request against master? Unless there is a good reason otherwise, we prefer pull requests against master and will backport as needed.
--->
+-->

.gitignore

@@ -35,7 +35,6 @@ selenium
 *.out
 ui_framework/doc_site/build
 !ui_framework/doc_site/build/index.html
-yarn.lock
 package-lock.json
 .yo-rc.json
 /.vscode

CONTRIBUTING.md

@@ -158,16 +158,18 @@ Install the version of node.js listed in the `.node-version` file _(this can be
 nvm install "$(cat .node-version)"
 ```
 
-Install `npm` dependencies
+Install the latest version of [yarn](https://yarnpkg.com).
+
+Install dependencies
 
 ```bash
-npm install
+yarn
 ```
 
 Start elasticsearch.
 
 ```bash
-npm run elasticsearch
+yarn elasticsearch
 ```
 
 > You'll need to have a `java` binary in `PATH` or set `JAVA_HOME`.
@@ -182,12 +184,12 @@ node scripts/makelogs
 
 Start the development server.
   ```bash
-  npm start
+  yarn start
   ```
 
 > On Windows, you'll need you use Git Bash, Cygwin, or a similar shell that exposes the `sh` command.  And to successfully build you'll need Cygwin optional packages zip, tar, and shasum.
 
-Now you can point your web browser to https://localhost:5601 and start using Kibana! When running `npm start`, Kibana will also log that it is listening on port 5603 due to the base path proxy, but you should still access Kibana on port 5601.
+Now you can point your web browser to https://localhost:5601 and start using Kibana! When running `yarn start`, Kibana will also log that it is listening on port 5603 due to the base path proxy, but you should still access Kibana on port 5601.
 
 #### Customizing `config/kibana.dev.yml`
 
@@ -203,7 +205,7 @@ In development mode, Kibana runs a customized version of [Webpack](http://webpac
 
 #### Setting Up SSL
 
-Kibana includes a self-signed certificate that can be used for development purposes: `npm start -- --ssl`.
+Kibana includes a self-signed certificate that can be used for development purposes: `yarn start --ssl`.
 
 ### Linting
 
@@ -230,23 +232,23 @@ Before running the tests you will need to install the projects dependencies as d
 Once that's done, just run:
 
 ```bash
-npm run test && npm run build -- --skip-os-packages
+yarn test && yarn build --skip-os-packages
 ```
 
 ### Debugging Unit Tests
 
-The standard `npm run test` task runs several sub tasks and can take several minutes to complete, making debugging failures pretty painful. In order to ease the pain specialized tasks provide alternate methods for running the tests.
+The standard `yarn test` task runs several sub tasks and can take several minutes to complete, making debugging failures pretty painful. In order to ease the pain specialized tasks provide alternate methods for running the tests.
 
-To execute both server and browser tests, but skip linting, use `npm run test:quick`.
+To execute both server and browser tests, but skip linting, use `yarn test:quick`.
 
 ```bash
-npm run test:quick
+yarn test:quick
 ```
 
-Use `npm run test:server` when you want to run only the server tests.
+Use `yarn test:server` when you want to run only the server tests.
 
 ```bash
-npm run test:server
+yarn test:server
 ```
 
 When you'd like to execute individual server-side test files, you can use the command below. Note that this command takes care of configuring Mocha with Babel compilation for you, and you'll be better off avoiding a globally installed `mocha` package. This command is great for development and for quickly identifying bugs.
@@ -261,16 +263,16 @@ You could also add the `--debug` option so that `node` is run using the `--debug
 node scripts/mocha --debug <file>
 ```
 
-With `npm run test:browser`, you can run only the browser tests. Coverage reports are available for browser tests by running `npm run test:coverage`. You can find the results under the `coverage/` directory that will be created upon completion.
+With `yarn test:browser`, you can run only the browser tests. Coverage reports are available for browser tests by running `yarn test:coverage`. You can find the results under the `coverage/` directory that will be created upon completion.
 
 ```bash
-npm run test:browser
+yarn test:browser
 ```
 
-Using `npm run test:dev` initializes an environment for debugging the browser tests. Includes an dedicated instance of the kibana server for building the test bundle, and a karma server. When running this task the build is optimized for the first time and then a karma-owned instance of the browser is opened. Click the "debug" button to open a new tab that executes the unit tests.
+Using `yarn test:dev` initializes an environment for debugging the browser tests. Includes an dedicated instance of the kibana server for building the test bundle, and a karma server. When running this task the build is optimized for the first time and then a karma-owned instance of the browser is opened. Click the "debug" button to open a new tab that executes the unit tests.
 
 ```bash
-npm run test:dev
+yarn test:dev
 ```
 
 In the screenshot below, you'll notice the URL is `localhost:9876/debug.html`. You can append a `grep` query parameter to this URL and set it to a string value which will be used to exclude tests which don't match. For example, if you changed the URL to `localhost:9876/debug.html?query=my test` and then refreshed the browser, you'd only see tests run which contain "my test" in the test description.
@@ -285,8 +287,8 @@ This should work super if you're using the [Kibana plugin generator](https://git
 To run the tests for just your particular plugin run the following command from your plugin:
 
 ```bash
-npm run test:server
-npm run test:browser -- --dev # remove the --dev flag to run them once and close
+yarn test:server
+yarn test:browser --dev # remove the --dev flag to run them once and close
 ```
 
 ### Cross-browser Compatibility
@@ -300,7 +302,7 @@ npm run test:browser -- --dev # remove the --dev flag to run them once and close
 * Open VMWare and go to Window > Virtual Machine Library. Unzip the virtual machine and drag the .vmx file into your Virtual Machine Library.
 * Right-click on the virtual machine you just added to your library and select "Snapshots...", and then click the "Take" button in the modal that opens. You can roll back to this snapshot when the VM expires in 90 days.
 * In System Preferences > Sharing, change your computer name to be something simple, e.g. "computer".
-* Run Kibana with `npm start -- --host=computer.local` (substituting your computer name).
+* Run Kibana with `yarn start --host=computer.local` (substituting your computer name).
 * Now you can run your VM, open the browser, and navigate to `http://computer.local:5601` to test Kibana.
 
 #### Running Browser Automation Tests
@@ -316,13 +318,13 @@ Packages are built using fpm, pleaserun, dpkg, and rpm.  fpm and pleaserun can b
 apt-get install ruby-dev rpm
 gem install fpm -v 1.5.0
 gem install pleaserun -v 0.0.24
-npm run build -- --skip-archives
+yarn build --skip-archives
 ```
 
 To specify a package to build you can add `rpm` or `deb` as an argument.
 
 ```bash
-npm run build -- --rpm
+yarn build --rpm
 ```
 
 Distributable packages can be found in `target/` after the build completes.

docs/development/core/development-basepath.asciidoc

@@ -79,9 +79,9 @@ To accomplish this the `serve` task does a few things:
 
 This proxy can sometimes have unintended side effects in development, so when
 needed you can opt out by passing the `--no-base-path` flag to the `serve` task
-or `npm start`.
+or `yarn start`.
 
 ["source","shell"]
 -----------
-npm start -- --no-base-path
------------
+yarn start --no-base-path
+-----------

docs/development/core/development-dependencies.asciidoc

@@ -10,14 +10,14 @@ Before you can use an external library with Kibana you have to install it. You
 do that using...
 
 [float]
-==== npm (preferred method)
+==== yarn (preferred method)
 
 Once you've http://npmsearch.com[found] a dependency you want to add, you can
 install it like so:
 
 ["source","shell"]
 -----------
-npm install --save some-neat-library
+yarn add some-neat-library
 -----------
 
 At the top of a javascript file, just import the library using it's name:
@@ -28,13 +28,13 @@ import someNeatLibrary from 'some-neat-library';
 -----------
 
 Just like working in node.js, front-end code can require node modules installed
-by npm without any additional configuration.
+by yarn without any additional configuration.
 
 [float]
 ==== webpackShims
 
 When a library you want to use does use es6 or common.js modules but is not
-available on npm, you can copy the source of the library into a webpackShim.
+available with yarn, you can copy the source of the library into a webpackShim.
 
 ["source","shell"]
 -----------
@@ -97,7 +97,7 @@ module.exports = window.angular;
 What this shim does is fairly simple if you go line by line:
 
 . makes sure that jQuery is loaded before angular (which actually runs the shim above)
-. load the angular.js file from the npm installation
+. load the angular.js file from the node_modules directory
 . load the angular-elastic plugin, a plugin we want to always be included whenever we import angular
 . use the `ui/modules` module to add the module exported by angular-elastic as a dependency to the `kibana` angular module
 . finally, export the window.angular variable. This means that writing `import angular from 'angular';` will properly set the angular variable to the angular library, rather than undefined which is the default behavior.

docs/development/core/development-functional-tests.asciidoc

@@ -12,20 +12,20 @@ There are three ways to run the tests depending on your goals:
 
 1. Easiest option:
 ** Description: Starts up Kibana & Elasticsearch servers, followed by running tests. This is much slower when running the tests multiple times because slow startup time for the servers. Recommended for single-runs.
-** `npm run test:ui`
+** `yarn test:ui`
 *** does everything in a single command, including running Elasticsearch and Kibana locally
 *** tears down everything after the tests run
 *** exit code reports success/failure of the tests
 
 2. Best for development:
 ** Description: Two commands, run in separate terminals, separate the components that are long-running and slow from those that are ephemeral and fast. Tests can be re-run much faster, and this still runs Elasticsearch & Kibana locally.
-** `npm run test:ui:server`
+** `yarn test:ui:server`
 *** starts Elasticsearch and Kibana servers
 *** slow to start
 *** can be reused for multiple executions of the tests, thereby saving some time when re-running tests
 *** automatically restarts the Kibana server when relevant changes are detected
 ** `node scripts/functional_test_runner`
-*** runs the tests against Kibana & Elasticsearch servers that were started `npm run test:ui:server`
+*** runs the tests against Kibana & Elasticsearch servers that were started `yarn test:ui:server`
 *** exit code reports success or failure of the tests
 
 3. Custom option:
@@ -405,4 +405,4 @@ const log = getService(‘log’);
 
 // log.debug only writes when using the `--debug` or `--verbose` flag.
 log.debug(‘done clicking menu’);
------------
+-----------

package.json

@@ -300,8 +300,13 @@
     "yeoman-generator": "1.1.1",
     "yo": "2.0.0"
   },
+  "resolutions": {
+    "angular": "1.6.5",
+    "moment": "2.13.0",
+    "react": "16.0.0"
+  },
   "engines": {
     "node": "6.12.2",
-    "npm": "3.10.10"
+    "yarn": "^1.3.2"
   }
 }

src/dev/ci_setup/setup.sh

@@ -10,7 +10,7 @@ cacheDir="${CACHE_DIR:-"/tmp/kibana"}"
 ### check that we seem to be in a kibana project
 ###
 if [ -f "$dir/package.json" ] && [ -f "$dir/.node-version" ]; then
-  echo "Setting up node.js and npm in $dir"
+  echo "Setting up node.js and yarn in $dir"
 else
   echo "src/dev/ci_setup/setup.sh must be run within a kibana repo"
   exit 1
@@ -41,13 +41,33 @@ fi
 
 
 ###
-### "install" node by extending the path with it's bin directory
+### "install" node into this shell
 ###
 export PATH="$nodeDir/bin:$PATH"
+hash -r
+
+###
+### setting up yarn
+###
+yarnVersion="1.3.2"
+yarnDir="$cacheDir/yarn/$yarnVersion"
+
+echo " -- using vendored version of yarn"
+mkdir -p "$yarnDir/bin"
+cp "$dir/tasks/vendor/yarn-1.3.2-with-ignore-fix.js" "$yarnDir/bin/yarn"
+
+###
+### "install" yarn into this shell
+###
+export PATH="$yarnDir/bin:$PATH"
+yarnGlobalDir="$(yarn global bin)"
+export PATH="$PATH:$yarnGlobalDir"
+hash -r
 
 
 ###
 ### install dependencies
 ###
 echo " -- installing node.js dependencies"
-npm install --cache "$cacheDir/npm"
+yarn config set cache-folder "$cacheDir/yarn"
+yarn --frozen-lockfile

src/ui/public/time_buckets/calc_auto_interval.js

@@ -1,4 +1,5 @@
-import moment, { duration as d } from 'moment';
+import moment from 'moment';
+const { duration: d } = moment;
 
 export function TimeBucketsCalcAutoIntervalProvider() {
   // these are the rounding rules used by roundInterval()

tasks/build/index.js

@@ -15,7 +15,7 @@ module.exports = function (grunt) {
       '_build:packageJson',
       '_build:readme',
       '_build:babelCache',
-      '_build:installNpmDeps',
+      '_build:installDependencies',
       '_build:notice',
       '_build:removePkgJsonDeps',
       'clean:testsFromModules',

tasks/build/install_dependencies.js

@@ -0,0 +1,13 @@
+import { exec } from 'child_process';
+module.exports = function (grunt) {
+  grunt.registerTask('_build:installDependencies', function () {
+    // We rely on a local version of Yarn that contains the bugfix from
+    // https://github.com/yarnpkg/yarn/pull/5059. Once this fix is merged
+    // and released we can use Yarn directly in the build.
+    const yarn = require.resolve('../vendor/yarn-1.3.2-with-ignore-fix.js');
+
+    exec(`${yarn} --production --ignore-optional --frozen-lockfile`, {
+      cwd: grunt.config.process('<%= root %>/build/kibana')
+    }, this.async());
+  });
+};

tasks/build/remove_pkg_json_deps.js

@@ -8,5 +8,7 @@ module.exports = function (grunt) {
       'build/kibana/package.json',
       JSON.stringify(pkg, null, '  ')
     );
+
+    grunt.file.delete('build/kibana/yarn.lock');
   });
 };

tasks/config/copy.js

@@ -3,6 +3,7 @@ module.exports = function () {
     devSource: {
       options: { mode: true },
       src: [
+        'yarn.lock',
         'src/**',
         '!src/**/__tests__/**',
         '!src/test_utils/**',

tasks/config/karma.js

@@ -102,11 +102,11 @@ module.exports = function (grunt) {
    *  (&shard_num=Y), are added to the testing bundle url and read by the
    *  test_harness/setup_test_sharding[1] module. This allows us to use a
    *  different number of shards in different scenarios (ie. running
-   *  `npm run test:browser` runs the tests in a single shard, effectively
+   *  `yarn test:browser` runs the tests in a single shard, effectively
    *  disabling sharding)
    *
    *  These same parameters can also be defined in the URL/query string of the
-   *  karma debug page (started when you run `npm run test:dev`).
+   *  karma debug page (started when you run `yarn test:dev`).
    *
    *  ## debugging
    *