[Spaces] Space-Aware Saved Objects #18862

legrego · 2018-05-07T14:01:10Z

This is an updated description that incorperates some of the discussions below

This PR introduces changes which allow the Spaces plugin to make saved objects "space aware". Effectively, this means wrapping the Saved Objects Client to alter or filter requests/responses.

Note Advanced UI Settings (i.e., saved objects of type config) are not in scope for this PR, and will be addressed separately.

Terminology:

SOC: Saved Objects Client

Saved Objects Client

`get`

The response from the base SOC is checked to see if the object belongs to the current space. If not, a 404 is thrown to indicate the object does not exist.

`bulk_get`

The response from the base SOC is checked to see if each object belongs to the current space. For each object that does not belong, its contents are replaced with a 404 response, which looks identical to the base SOC's 404 response for a missing object.

`create`

The spaceId is appended to the create request, so the base SOC will write the new object into the correct space.

`bulk_create`

The spaceId is appended to each space-aware object in the request, so the base SOC will write the new objects into the correct space.

`update`

Before allowing an update to be processed by the base SOC, we check to ensure that it belongs to the current space. If not, a 404 is thrown. We also ensure that the spaceId is not changed as a result of an update.

`delete`

Before allowing a delete to be processed by the base SOC, we check to ensure that it belongs to the current space. If not, a 404 is thrown.

`find`

Searching is arguably the most complex case for this PR, and is responsible for a bulk of the LOC (other than tests). When performing a find, we augment the ES query to ensure that each object belongs to the current space.

* Belonging to the current space

To figure out if an object belongs to the current space, the following check is performed:

1. Is the object's type space-aware?

Most saved object types are space-aware. There are a couple of exceptions as of this PR: space and config.
If the type is not space-aware, then ✅ this object belongs to the current space. This implies that objects that are not space aware belong to every space.

If the type is space-aware, then processing continues to step 2

2. Check the object's `spaceId`

Each saved object may have a spaceId assigned. This spaceId is compared against the spaceId that the user's request is executed within. If they match, then the object belongs to the current space.

caveat The Default Space is a special-case space that does not assign a spaceId to its underlying objects. This is done to maintain backwards compatibility, and makes bootstrapping Spaces much easier for upgrading installations. Given this, there is logic in place which accounts for this special-case. The most interesting example is when we build the query for the SOC's find operation. Rather than checking that the object has a particular spaceId, we have to check that the object does not have a spaceId assigned.

legrego · 2018-05-07T14:02:15Z

src/server/saved_objects/client/saved_objects_client.js

@@ -122,7 +123,8 @@ export class SavedObjectsClient {
        body: {
          type,
          updated_at: time,
-          [type]: attributes
+          [type]: attributes,
+          ...extraBodyProperties


Used to assign the spaceId

Should we put this first so it can't overwrite the other properties?

legrego · 2018-05-07T14:02:33Z

src/server/saved_objects/client/saved_objects_client.js

@@ -169,7 +171,8 @@ export class SavedObjectsClient {
        {
          type: object.type,
          updated_at: time,
-          [object.type]: object.attributes
+          [object.type]: object.attributes,
+          ...object.extraBodyProperties


Used to assign the spaceId

Let's move this one up "top" as well, like we did for create

Thanks -- don't know how I missed that...fixed!

legrego · 2018-05-07T14:02:59Z

src/server/saved_objects/client/saved_objects_client.js

@@ -300,6 +304,10 @@ export class SavedObjectsClient {
      }
    };

+    if (esOptions.body.query && typeof queryDecorator === 'function') {
+      esOptions.body.query = queryDecorator(esOptions.body.query);


Augments the ES query to restrict results to the active Space

If we only support functions, let's change this to throw an Error if something else is provided instead of silently ignoring it.

legrego · 2018-05-07T14:03:35Z

src/server/saved_objects/client/saved_objects_client.js

+
+    let docsToReturn = docs;
+    if (typeof options.documentFilter === 'function') {
+      docsToReturn = docs.filter(options.documentFilter);


Filters ES results to only show documents for the active Space

legrego · 2018-05-07T14:04:27Z

src/server/saved_objects/client/saved_objects_client.js

          id,
          type,
          ...time && { updated_at: time },
          version: doc._version,
          attributes: doc._source[type]
        };
+
+        if (typeof options.resultDecorator === 'function') {
+          return options.resultDecorator(savedObject, doc);


Hacky...this might not be needed. Adds the spaceId to each returned object

legrego · 2018-05-07T14:04:57Z

src/server/saved_objects/client/saved_objects_client.js

    const response = await this._callCluster('get', {
      id: this._generateEsId(type, id),
      type: this._type,
      index: this._index,
      ignore: [404]
    });

+    if (typeof options.responseInterceptor === 'function') {
+      options.responseInterceptor(response);


Filter the ES search result based on the active Space

legrego · 2018-05-07T17:08:09Z

Hey @archanid & @kobelb!

This is a brute-force approach to satisfying some of our requirements for the Saved Object Client. My goal for opening this PR is to start a conversation about how we want to approach SOC changes both in the near-term, and long-term with the new Saved Objects Service.

Here is the general approach for how this PR attempts to satisfy our requirements:

Get/MGet:

Filter result from ES to ensure the object(s) belong to the active Space. This unfortunately requires us to receive the entire ES query results on the Kibana server, and filter it there, but I don't see another approach without changing the ES API. I suppose alternatively, we could translate the get and mget requests to find operations?

Find:

Augment query to ensure that ES only returns objects that belong to the active Space.

Create/Update/Delete:

Add spaceId to the object being created

Update and Delete operations are a bit trickier:

Updates would require a get beforehand to ensure the object belongs to the active Space. This might be ok, but I wasn't sure about incurring that additional overhead. Thoughts?
Delete has the same problem - we either need to do a get beforehand, or change the delete operation to use delete by query. My assumption is that a simple get beforehand would be more efficient than a delete-by-query, but I'd still like to discuss this and get your thoughts.

kobelb · 2018-05-07T20:10:21Z

src/server/saved_objects/client/saved_objects_client.js

@@ -122,7 +123,8 @@ export class SavedObjectsClient {
        body: {
          type,
          updated_at: time,
-          [type]: attributes
+          [type]: attributes,
+          ...extraBodyProperties


Should we put this first so it can't overwrite the other properties?

kobelb · 2018-05-07T20:18:38Z

src/server/saved_objects/client/saved_objects_client.js

@@ -300,6 +304,10 @@ export class SavedObjectsClient {
      }
    };

+    if (esOptions.body.query && typeof queryDecorator === 'function') {
+      esOptions.body.query = queryDecorator(esOptions.body.query);


If we only support functions, let's change this to throw an Error if something else is provided instead of silently ignoring it.

kobelb · 2018-05-07T20:31:56Z

x-pack/plugins/spaces/server/lib/saved_objects_client/spaces_saved_objects_client.js

+    if (this._isTypeSpaceAware(options.type)) {
+      const spaceId = await this._getSpaceId();
+
+      spaceOptions.queryDecorator = (query) => {


Instead of using the decorator here, could we pass in the additional filter and let the SavedObjectsClient know how to append it? It's less generic, but let's us be less concerned with the internal implementation of the query itself.

kobelb · 2018-05-07T20:39:52Z

x-pack/plugins/spaces/server/lib/saved_objects_client/spaces_saved_objects_client.js

+    // ES 'mget' does not support queries, so we have to filter results after the fact.
+    const thisSpaceId = await this._getSpaceId();
+
+    return await this._client.bulkGet(objects, {


In a similar fashion to the above, how do you feel about passing in an extraSourceProperties array that we use to copy the additional properties from the document? Then we could just do the filter on the result without adding the extension point.

I like it! Done!

kobelb · 2018-05-07T20:43:40Z

x-pack/plugins/spaces/server/lib/saved_objects_client/spaces_saved_objects_client.js

+    let thisSpaceId;
+
+    if (this._isTypeSpaceAware(type)) {
+      thisSpaceId = await this._getSpaceId();


Same as above, if we added the extraSourceProperties then we could by default assign that property from the _source to the object and then just filter the response.

I like it! Done!

kobelb · 2018-05-07T20:44:43Z

x-pack/plugins/spaces/server/lib/saved_objects_client/spaces_saved_objects_client.js

+  }
+
+  async update(type, id, attributes, options = {}) {
+    return await this._client.update(type, id, attributes, options);


To be safe, let's specify the spaceId here always so that they don't accidentally write the document without a spaceId or to the wrong spaceId.

kobelb · 2018-05-08T20:44:46Z

src/server/saved_objects/client/saved_objects_client.js

+    const { docs } = response;
+
+    let docsToReturn = docs;
+    if (typeof options.documentFilter === 'function') {


Do we still need this?

I should be able to pull this logic up into the spaces_saved_object_client now that we've introduced extraSourceProperties

awesome, thanks!

kobelb

This is looking really good, let's make sure to discuss with @archanid when she returns

rhoboat · 2018-05-10T21:46:15Z

create

Add spaceId to the object being created

This is only if the object is being created in a space. Otherwise, if we're in the global space (no-space-land), no such thing is added to the object, right?

get + bulkGet

Changing internally to use find (search) is an interesting idea. I did an audit of where all get and bulkGet are currently used to see what would be affected, and it seems like we just need a collection of objects, similar to what find returns. Get/bulkGet is less complex than the typical find operation can be. So, looking at esOptions, everything that get requires (index, type, id, ignore) exists except for id which would be passed as a field or searchField. bulkGet calls with this, which would require some transformation if it were to use find. But it's possible to do the translation to search for both.

Scrap all that. After more research I think it's a bad idea to use search/find inside of our current get and bulkGet.

Search will be less efficient because it must look through all shards, whereas get knows the shard based on hash("DOCID") % NUM_OF_SHARDS, which search doesn't do. So we may be optimizing on the Kibana side, but increasing some complexity on the Elasticsearch side.
Current users of get expect the latest version of the document, whereas users of search do not. Behind the scenes, the index is refreshed if needed before doing the get. I'm not sure how many of our current users of get and mget even rely on having the latest document, but it could be a reason not to do this. Forcing a refresh before we do a search is an option, but feels also costly.

So we just have to weigh the tradeoffs. it might be a good idea if it looks like we'll be pulling a lot of stuff into memory much of the time. I'm not sure how to advise on something that may end up being a premature optimization in the end. Searches, even if they're across all shards, are pretty fast on Elasticsearch, as we know, since it's all about the search. Are we ok with not being able to guarantee the latest version of the document?

With the unanswered questions, I lean toward not changing the underlying functionality from get/mget to search. I would like to feel the pain of too many objects in memory before going that route.

update + delete

Doing a get before updates and deletes should be fine, IMO.

High level question: What about OLS metadata? This is just about Spaces. Have we thought about what needs to change for OLS to be supported in the current Saved Objects Client?

rhoboat · 2018-05-10T22:03:43Z

The optimization discussion above only would apply to mget usage, right? get is always getting a single document?

legrego · 2018-05-11T12:28:07Z

This is only if the object is being created in a space. Otherwise, if we're in the global space (no-space-land), no such thing is added to the object, right?

We only add the spaceId when Spaces are enabled. The spaceId will be appended to every created object that is space aware...that is to say, anything that goes through the saved object client. The one exception is the Default Space. This is a special, system generated Space that cannot be deleted, and represents all objects that were created prior to Spaces existing. Users can also create new objects in this space, and when they do, I expect that we will not be storing the spaceId -- even though the default space technically has an id.

[...] Scrap all that. After more research I think it's a bad idea to use search/find inside of our current get and bulkGet. [...]

This is a great insight, thank you for explaining the performance impacts here.
The theoritical options for get/mget that I was trying to describe are:

Have Elasticsearch filter the search based on spaceId
Have Elasticsearch return the results as it does today, and have the Saved Objects Client[/Service] filter the results based on spaceId before returning to the consumer.

Option 1 is not viable because the get and mget operations do not allow for that kind of filtering, and search/find could have a performance degredation.

Option 2 allows for fast retrieval from Elasticsearch, at the expense of additional work on the Kibana server. Kibana will be receiving and parsing objects that it has to discard. I don't expect that it'll be a huge dataset, but I suppose it's something to consider.

High level question: What about OLS metadata? This is just about Spaces. Have we thought about what needs to change for OLS to be supported in the current Saved Objects Client?

We haven't implemented anything at this point, but I think we have a vague idea of what needs to happen. The Security Plugin will likely take similar steps to augment search/find queries, and to filter get/mget results.

elasticmachine · 2018-07-23T20:52:12Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

legrego · 2018-07-25T12:12:27Z

@kobelb since this PR doesn't deal with securing access to Spaces, I'm inclined to leave this as-is (pending feedback), and introduce the Spaces Client within the PR that actually enforces security. This PR is already getting rather large, and I have a couple of other features waiting on this one.

kobelb · 2018-07-25T12:23:07Z

@legrego that makes sense, lemme give this another review here momentarily.

kobelb · 2018-07-25T15:14:55Z

src/server/saved_objects/service/saved_objects_client.js

@@ -102,6 +102,7 @@ export class SavedObjectsClient {
   * @param {object} [options={}]
   * @property {string} [options.id] - force id on creation, not recommended
   * @property {boolean} [options.overwrite=false]
+   * @property {object} [options.extraBodyProperties={}] - extra properties to append to the document body, outside of the object's type property


I'm prone to think we should rename these to extraDocumentProperties as well, what do you think?

kobelb · 2018-07-25T15:29:15Z

x-pack/plugins/spaces/server/lib/saved_objects_client/lib/query_filters.js

+    must: []
+  };
+
+  if (type) {


type should always be provided, or else we can't build a clause for the type, can we throw an explicit error if it's not provided here?

kobelb · 2018-07-25T15:30:39Z

x-pack/plugins/spaces/server/lib/saved_objects_client/lib/query_filters.test.js

+
+import { getSpacesQueryFilters } from './query_filters';
+
+test('returns no parameters when no types are provided', () => {


Shouldn't we be throwing an error if no types are provided because we don't know how to filter anything if that's the case?

kobelb · 2018-07-25T15:49:31Z

x-pack/plugins/spaces/server/lib/saved_objects_client/spaces_saved_objects_client.js

+      ...options
+    };
+
+    if (shouldAssignSpaceId) {


If the type is space-aware, and we're in the default space, we're not excluding the spaceOptions.extraDocumentProperties from specifying a space ID of their own.

great catch, thanks!

kobelb · 2018-07-25T15:54:48Z

x-pack/plugins/spaces/server/lib/saved_objects_client/spaces_saved_objects_client.js

+          }
+        };
+      }
+      return object;


Same situation with bulkCreate if we're in the default space and the type is space-aware, we'll want to ensure that we aren't letting the caller specify a space ID of their own using extraDocumentProperties.

kobelb · 2018-07-25T16:05:39Z

x-pack/plugins/spaces/server/lib/saved_objects_client/spaces_saved_objects_client.test.js

+});
+
+describe('#find', () => {
+  test(`creates ES query filters restricting objects to the current space`, async () => {


Can we augment this test with a type that isn't space aware, and add a test for when we're in the default space? We do have some coverage via query_filters.test.js, but it feels rather awkward to miss the coverage here. What do you think?

x-pack/plugins/spaces/server/lib/saved_objects_client/spaces_saved_objects_client.test.js

@@ -0,0 +1,745 @@
+/*


legrego · 2018-07-26T11:23:35Z

thanks, @kobelb! I think I addressed all of your feedback. I ended up refactoring spaces_saved_objects_client.test.js to group tests by the default space, and by another space. I made those changes in a separate commit so that the other diffs are a little more readable.

elasticmachine · 2018-07-26T12:32:20Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

kobelb

LGTM

…19497) This replaces the existing Modal with a smaller Popover which is less intrusive. The popover also features a search bar for finding the desired Space when there are 8 or more Spaces to choose from. ### Details When there are less than 8 spaces available, the selector will render a simple list of spaces. When there are >= 8 spaces available, the selector will also render a search bar to let users search for their space. ### Prerequisites - [x] Merge #18862 into `spaces-phase-1` ### Known Issues - elastic/eui#1043 (fixed in `v3.2.0`) - elastic/eui#1052 (fixed in `v3.2.1`) - Missing typdefs (not a blocker to merge): elastic/eui#1120

### Review notes This is generally ready for review. We are awaiting elastic/elasticsearch#32777 to improve handling when users do not have any access to Kibana, but this should not hold up the overall review for this PR. This PR is massive, there's no denying that. Here's what to focus on: 1) `x-pack/plugins/spaces`: This is, well, the Spaces plugin. Everything in here is brand new. The server code is arguably more important, but feel free to review whatever you see fit. 2) `x-pack/plugins/security`: There are large and significant changes here to allow Spaces to be securable. To save a bit of time, you are free to ignore changes in `x-pack/plugins/security/public`: These are the UI changes for the role management screen, which were previously reviewed by both us and the design team. 3) `x-pack/test/saved_object_api_integration` and `x-pack/test/spaces_api_integration`: These are the API test suites which verify functionality for: a) Both security and spaces enabled b) Only security enabled c) Only spaces enabled What to ignore: 1) As mentioned above, you are free to ignore changes in `x-pack/plugins/security/public` 2) Changes to `kibana/src/server/*`: These changes are part of a [different PR that we're targeting against master](#23378) for easier review. ## Saved Objects Client Extensions A bulk of the changes to the saved objects service are in the namespaces PR, but we have a couple of important changes included here. ### Priority Queue for wrappers We have implemented a priority queue which allows plugins to specify the order in which their SOC wrapper should be applied: `kibana/src/server/saved_objects/service/lib/priority_collection.ts`. We are leveraging this to ensure that both the security SOC wrapper and the spaces SOC wrapper are applied in the correct order (more details below). ### Spaces SOC Wrapper This wrapper is very simple, and it is only responsible for two things: 1) Prevent users from interacting with any `space` objects (use the Spaces client instead, described below) 2) Provide a `namespace` to the underlying Saved Objects Client, and ensure that no other wrappers/callers have provided a namespace. In order to accomplish this, the Spaces wrapper uses the priority queue to ensure that it is the last wrapper invoked before calling the underlying client. ### Security SOC Wrapper This wrapper is responsible for performing authorization checks. It uses the priority queue to ensure that it is the first wrapper invoked. To say another way, if the authorization checks fail, then no other wrappers will be called, and the base client will not be called either. This wrapper authorizes users in one of two ways: RBAC or Legacy. More details on this are below. ### Examples: `GET /s/marketing/api/saved_objects/index-pattern/foo` **When both Security and Spaces are enabled:** 1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function 2) The Security wrapper is invoked. a) Authorization checks are performed to ensure user can access this particular saved object at this space. 3) The Spaces wrapper is invoked. a) Spaces applies a `namespace` to be used by the underlying client 4) The underlying client/repository are invoked to retrieve the object from ES. **When only Spaces are enabled:** 1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function 2) The Spaces wrapper is invoked. a) Spaces applies a `namespace` to be used by the underlying client 3) The underlying client/repository are invoked to retrieve the object from ES. **When only Security is enabled:** (assume `/s/marketing` is no longer part of the request) 1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function 2) The Security wrapper is invoked. a) Authorization checks are performed to ensure user can access this particular saved object globally. 3) The underlying client/repository are invoked to retrieve the object from ES. ## Authorization Authorization changes for this project are centered around Saved Objects, and builds on the work introduced in RBAC Phase 1. ### Saved objects client #### Security without spaces When security is enabled, but spaces is disabled, then the authorization model behaves the same way as before: If the user is taking advantage of Kibana Privileges, then we check their privileges "globally" before proceeding. A "global" privilege check specifies `resources: ['*']` when calling the [ES _has_privileges api.](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-has-privileges.html). Legacy users (non-rbac) will continue to use the underlying index privileges for authorization. #### Security with spaces When both plugins are enabled, then the authorization model becomes more fine-tuned. Rather than checking privileges globally, the privileges are checked against a specific resource that matches the user's active space. In order to accomplish this, the Security plugin needs to know if Spaces is enabled, and if so, it needs to ask Spaces for the user's active space. The subsequent call to the `ES _has_privileges api` would use `resources: ['space:marketing']` to verify that the user is authorized at the `marketing` space. Legacy users (non-rbac) will continue to use the underlying index privileges for authorization. **NOTE** The legacy behavior implies that those users will have access to all spaces. The read/write restrictions are still enforced, but there is no way to restrict access to a specific space for legacy auth users. #### Spaces without security No authorization performed. Everyone can access everything. ### Spaces client Spaces, when enabled, prevents saved objects of type `space` from being CRUD'd via the Saved Objects Client. Instead, the only "approved" way to work with these objects is through the new Spaces client (`kibana/x-pack/plugins/spaces/lib/spaces_client.ts`). When security is enabled, the Spaces client performs its own set of authorization checks before allowing the request to proceed. The Spaces client knows which authorization checks need to happen for a particular request, but it doesn't know _how_ to check privileges. To accomplish this, the spaces client will delegate the check security's authorization service. #### FAQ: Why oh why can't you used the Saved Objects Client instead!? That's a great question! We did this primarily to simplify the authorization model (at least for our initial release). Accessing regular saved objects follows a predictible authorization pattern (described above). Spaces themselves inform the authorization model, and this interplay would have greatly increased the complexity. We are brainstorming ideas to obselete the Spaces client in favor of using the Saved Objects Client everywhere, but that's certainly out of scope for this release. ## Test Coverage ### Saved Objects API A bulk of the changes to enable spaces are centered around saved objects, so we have spent a majority of our time automating tests against the saved objects api. **`x-pack/test/saved_object_api_integration/`** contains the test suites for the saved objects api. There is a `common/suites` subfolder which contains a bulk of the test logic. The suites defined here are used in the following test configurations: 1) Spaces only: `./spaces_only` 2) Security and spaces: `./security_and_spaces` 3) Security only: `./security_only` Each of these test configurations will start up ES/Kibana with the appropriate license and plugin set. Each set runs through the entire test suite described in `common/suites`. Each test with in each suite is run multiple times with different inputs, to test the various permutations of authentication, authorization type (legacy vs RBAC), space-level privileges, and the user's active space. ### Spaces API Spaces provides an experimental public API. **`x-pack/test/spaces_api_integration`** contains the test suites for the Spaces API. Similar to the Saved Objects API tests described above, there is a `common/suites` folder which contains a bulk of the test logic. The suites defined here are used in the following test configurations: 1) Spaces only: `./spaces_only` 2) Security and spaces: `./security_and_spaces` ### Role Management UI We did not provide any new functional UI tests for role management, but the existing suite was updated to accomidate the screen rewrite. We do have a decent suite of jest unit tests for the various components that make up the new role management screen. They're nested within `kibana/x-pack/plugins/security/public/views/management/edit_role` ### Spaces Management UI We did not provide any new functional UI tests for spaces management, but the components that make up the screens are well-tested, and can be found within `kibana/x-pack/plugins/spaces/public/views/management/edit_space` ### Spaces Functional UI Tests There are a couple of UI tests that verify _basic_ functionality. They assert that a user can login, select a space, and then choose a different space once inside: `kibana/x-pack/test/functional/apps/spaces` ## Reference Notable child PRs are listed below for easier digesting. Note that some of these PRs are built on other PRs, so the deltas in the links below may be outdated. Cross reference with this PR when in doubt. ### UI - Reactify Role Management Screen: #19035 - Space Aware Privileges UI: #21049 - Space Selector (in Kibana Nav): #19497 - Recently viewed Widget: #22492 - Support Space rename/delete: #22586 ### Saved Objects Client - ~~Space Aware Saved Objects: #18862 - ~~Add Space ID to document id: #21372 - Saved object namespaces (supercedes #18862 and #21372): #22357 - Securing saved objects: #21995 - Dedicated Spaces client (w/ security): #21995 ### Other - Public Spaces API (experimental): #22501 - Telemetry: #20581 - Reporting: #21457 - Spencer's original Spaces work: #18664 - Expose `spaceId` to "Add Data" tutorials: #22760 Closes #18948 "Release Note: Create spaces within Kibana to organize dashboards, visualizations, and other saved objects. Secure access to each space when X-Pack Security is enabled"

This is generally ready for review. We are awaiting elastic/elasticsearch#32777 to improve handling when users do not have any access to Kibana, but this should not hold up the overall review for this PR. This PR is massive, there's no denying that. Here's what to focus on: 1) `x-pack/plugins/spaces`: This is, well, the Spaces plugin. Everything in here is brand new. The server code is arguably more important, but feel free to review whatever you see fit. 2) `x-pack/plugins/security`: There are large and significant changes here to allow Spaces to be securable. To save a bit of time, you are free to ignore changes in `x-pack/plugins/security/public`: These are the UI changes for the role management screen, which were previously reviewed by both us and the design team. 3) `x-pack/test/saved_object_api_integration` and `x-pack/test/spaces_api_integration`: These are the API test suites which verify functionality for: a) Both security and spaces enabled b) Only security enabled c) Only spaces enabled What to ignore: 1) As mentioned above, you are free to ignore changes in `x-pack/plugins/security/public` 2) Changes to `kibana/src/server/*`: These changes are part of a [different PR that we're targeting against master](elastic#23378) for easier review. A bulk of the changes to the saved objects service are in the namespaces PR, but we have a couple of important changes included here. We have implemented a priority queue which allows plugins to specify the order in which their SOC wrapper should be applied: `kibana/src/server/saved_objects/service/lib/priority_collection.ts`. We are leveraging this to ensure that both the security SOC wrapper and the spaces SOC wrapper are applied in the correct order (more details below). This wrapper is very simple, and it is only responsible for two things: 1) Prevent users from interacting with any `space` objects (use the Spaces client instead, described below) 2) Provide a `namespace` to the underlying Saved Objects Client, and ensure that no other wrappers/callers have provided a namespace. In order to accomplish this, the Spaces wrapper uses the priority queue to ensure that it is the last wrapper invoked before calling the underlying client. This wrapper is responsible for performing authorization checks. It uses the priority queue to ensure that it is the first wrapper invoked. To say another way, if the authorization checks fail, then no other wrappers will be called, and the base client will not be called either. This wrapper authorizes users in one of two ways: RBAC or Legacy. More details on this are below. `GET /s/marketing/api/saved_objects/index-pattern/foo` **When both Security and Spaces are enabled:** 1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function 2) The Security wrapper is invoked. a) Authorization checks are performed to ensure user can access this particular saved object at this space. 3) The Spaces wrapper is invoked. a) Spaces applies a `namespace` to be used by the underlying client 4) The underlying client/repository are invoked to retrieve the object from ES. **When only Spaces are enabled:** 1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function 2) The Spaces wrapper is invoked. a) Spaces applies a `namespace` to be used by the underlying client 3) The underlying client/repository are invoked to retrieve the object from ES. **When only Security is enabled:** (assume `/s/marketing` is no longer part of the request) 1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function 2) The Security wrapper is invoked. a) Authorization checks are performed to ensure user can access this particular saved object globally. 3) The underlying client/repository are invoked to retrieve the object from ES. Authorization changes for this project are centered around Saved Objects, and builds on the work introduced in RBAC Phase 1. When security is enabled, but spaces is disabled, then the authorization model behaves the same way as before: If the user is taking advantage of Kibana Privileges, then we check their privileges "globally" before proceeding. A "global" privilege check specifies `resources: ['*']` when calling the [ES _has_privileges api.](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-has-privileges.html). Legacy users (non-rbac) will continue to use the underlying index privileges for authorization. When both plugins are enabled, then the authorization model becomes more fine-tuned. Rather than checking privileges globally, the privileges are checked against a specific resource that matches the user's active space. In order to accomplish this, the Security plugin needs to know if Spaces is enabled, and if so, it needs to ask Spaces for the user's active space. The subsequent call to the `ES _has_privileges api` would use `resources: ['space:marketing']` to verify that the user is authorized at the `marketing` space. Legacy users (non-rbac) will continue to use the underlying index privileges for authorization. **NOTE** The legacy behavior implies that those users will have access to all spaces. The read/write restrictions are still enforced, but there is no way to restrict access to a specific space for legacy auth users. No authorization performed. Everyone can access everything. Spaces, when enabled, prevents saved objects of type `space` from being CRUD'd via the Saved Objects Client. Instead, the only "approved" way to work with these objects is through the new Spaces client (`kibana/x-pack/plugins/spaces/lib/spaces_client.ts`). When security is enabled, the Spaces client performs its own set of authorization checks before allowing the request to proceed. The Spaces client knows which authorization checks need to happen for a particular request, but it doesn't know _how_ to check privileges. To accomplish this, the spaces client will delegate the check security's authorization service. That's a great question! We did this primarily to simplify the authorization model (at least for our initial release). Accessing regular saved objects follows a predictible authorization pattern (described above). Spaces themselves inform the authorization model, and this interplay would have greatly increased the complexity. We are brainstorming ideas to obselete the Spaces client in favor of using the Saved Objects Client everywhere, but that's certainly out of scope for this release. A bulk of the changes to enable spaces are centered around saved objects, so we have spent a majority of our time automating tests against the saved objects api. **`x-pack/test/saved_object_api_integration/`** contains the test suites for the saved objects api. There is a `common/suites` subfolder which contains a bulk of the test logic. The suites defined here are used in the following test configurations: 1) Spaces only: `./spaces_only` 2) Security and spaces: `./security_and_spaces` 3) Security only: `./security_only` Each of these test configurations will start up ES/Kibana with the appropriate license and plugin set. Each set runs through the entire test suite described in `common/suites`. Each test with in each suite is run multiple times with different inputs, to test the various permutations of authentication, authorization type (legacy vs RBAC), space-level privileges, and the user's active space. Spaces provides an experimental public API. **`x-pack/test/spaces_api_integration`** contains the test suites for the Spaces API. Similar to the Saved Objects API tests described above, there is a `common/suites` folder which contains a bulk of the test logic. The suites defined here are used in the following test configurations: 1) Spaces only: `./spaces_only` 2) Security and spaces: `./security_and_spaces` We did not provide any new functional UI tests for role management, but the existing suite was updated to accomidate the screen rewrite. We do have a decent suite of jest unit tests for the various components that make up the new role management screen. They're nested within `kibana/x-pack/plugins/security/public/views/management/edit_role` We did not provide any new functional UI tests for spaces management, but the components that make up the screens are well-tested, and can be found within `kibana/x-pack/plugins/spaces/public/views/management/edit_space` There are a couple of UI tests that verify _basic_ functionality. They assert that a user can login, select a space, and then choose a different space once inside: `kibana/x-pack/test/functional/apps/spaces` Notable child PRs are listed below for easier digesting. Note that some of these PRs are built on other PRs, so the deltas in the links below may be outdated. Cross reference with this PR when in doubt. - Reactify Role Management Screen: elastic#19035 - Space Aware Privileges UI: elastic#21049 - Space Selector (in Kibana Nav): elastic#19497 - Recently viewed Widget: elastic#22492 - Support Space rename/delete: elastic#22586 - ~~Space Aware Saved Objects: elastic#18862 - ~~Add Space ID to document id: elastic#21372 - Saved object namespaces (supercedes elastic#18862 and elastic#21372): elastic#22357 - Securing saved objects: elastic#21995 - Dedicated Spaces client (w/ security): elastic#21995 - Public Spaces API (experimental): elastic#22501 - Telemetry: elastic#20581 - Reporting: elastic#21457 - Spencer's original Spaces work: elastic#18664 - Expose `spaceId` to "Add Data" tutorials: elastic#22760 Closes elastic#18948 "Release Note: Create spaces within Kibana to organize dashboards, visualizations, and other saved objects. Secure access to each space when X-Pack Security is enabled"

commit fc83aae2528e23c4cac8291468244e010fc430e9 Merge: 37ee2a1cc9 a5ed541b6a Author: maryia-lapata <[email protected]> Date: Tue Oct 16 11:00:34 2018 +0300 Merge branch 'feature/translations/tagcloud' of https://github.com/maryia-lapata/kibana into feature/translations/tagcloud commit 37ee2a1cc94baa436ec0f28ab2db14f13b01dfa9 Author: maryia-lapata <[email protected]> Date: Tue Oct 16 10:58:51 2018 +0300 Translations for Tag Cloud commit a5ed541b6aeb56cd106d8ec2236ba4aff08b2100 Merge: 74bf3f44bb b21337c4c9 Author: maryia-lapata <[email protected]> Date: Tue Oct 16 10:56:09 2018 +0300 Merge branch 'feature/translations/tagcloud' of https://github.com/maryia-lapata/kibana into feature/translations/tagcloud commit 74bf3f44bbb6d198e76c64727adc937049bc953c Author: maryia-lapata <[email protected]> Date: Tue Oct 16 10:55:03 2018 +0300 Translations for Tag Cloud commit b21337c4c9b3e42e77f4e9dc4fc4719887cf37c6 Author: maryia-lapata <[email protected]> Date: Tue Oct 16 10:45:31 2018 +0300 Revert changes commit 964ee059861b1f9fb2809093fa5995718bf48f6c Merge: 44f88c0de0 8fe71f888f Author: maryia-lapata <[email protected]> Date: Tue Oct 16 10:19:26 2018 +0300 Merge branch 'master' into feature/translations/tagcloud commit 8fe71f888f879e85bfab3a76df8df79fbfdd7e95 Author: Josh Dover <[email protected]> Date: Mon Oct 15 22:39:39 2018 +0100 Skip failing dashboard reporting test (#24040) commit d22bdfec461bc602d3d5ee36e06944162fed4c21 Author: Spencer <[email protected]> Date: Mon Oct 15 14:34:30 2018 -0700 [yarn] Upgrade to 1.10.1 (#23971) * [yarn] upgrade required version * [yarn] update lock files to include integrity * [yarn] coalesce locked readable-stream versions to avoid yarn bug commit 0b71747e78f4e3ee1c66b1096c024a6550c3dcb1 Author: Ryan Keairns <[email protected]> Date: Mon Oct 15 14:41:01 2018 -0500 fix sidebar scrolling in firefox (#24011) commit 5d19ace725518740f74590634bd665a5fa46b4cb Author: Brandon Kobel <[email protected]> Date: Mon Oct 15 18:04:27 2018 +0100 Switching to using a fork (#23422) commit aba586fb6924125f54affffb24a185d7c1c72690 Author: Larry Gregory <[email protected]> Date: Mon Oct 15 17:15:23 2018 +0100 Delete objects belonging to removed space (#23640) * delete objects belonging to removed space * remove unused parameters commit b3a15d4f5b31a07299d3c99d8ff69bb980ce0c09 Author: Catherine Liu <[email protected]> Date: Mon Oct 15 12:05:47 2018 -0400 Sets private:true in canvas package.json (#24022) commit 2eb449e0425e6ad310c42119712c56080380467b Author: Catherine Liu <[email protected]> Date: Mon Oct 15 10:34:14 2018 -0400 Moved squel from devDependencies to dependencies (#23849) commit d4d0911968f8847c1379864db2a44a5027f3acd0 Author: Brandon Kobel <[email protected]> Date: Mon Oct 15 15:11:12 2018 +0100 Fixing the spaces audit logger when security is explicitly disabled (#23878) commit 527178771ab784b1c76178e552adcdc592c9abbf Author: Lee Drengenberg <[email protected]> Date: Thu Oct 11 05:14:41 2018 -0500 fix building Canvas plugin on Windows (#23920) commit e9b5abe1b5a433165e90f5c236b49050f72ab76e Author: Joe Fleming <[email protected]> Date: Thu Oct 11 09:12:01 2018 +0100 fix: Router can render function or class components (#23372) * fix: router can render function or class components * fix: correctly define state and change the first load detection, since this.state is always set now * chore: DRY up navigation code * tests: disable listener cleanup test there's no way to know when the listener is going to get cleaned up anymore :( commit 44f88c0de014ee269b427359e2d84a2d1b6f82fc Author: maryia-lapata <[email protected]> Date: Wed Oct 10 17:26:59 2018 +0300 Use one-time binding for aria-label attributes commit dfe534156713b5d76c2f0d53703ffd90fc6f4c8d Merge: c1b4fb4dbd ec2f025312 Author: maryia-lapata <[email protected]> Date: Wed Oct 10 17:23:46 2018 +0300 Merge branch 'master' into feature/translations/tagcloud commit c1b4fb4dbd97936702226be19f2d949d788650e9 Merge: aeb94a0d40 4246530213 Author: maryia-lapata <[email protected]> Date: Wed Oct 10 17:19:28 2018 +0300 Merge branch 'master' into feature/translations/tagcloud commit ec2f0253122af9f9c0677da837014d01b48a1627 Author: Chris Davies <[email protected]> Date: Wed Oct 10 09:15:48 2018 -0400 [WIP] Fix flaky reporting test that is failing due to a CSS animation (#23907) Add a wait for the reporting flyout menu to animate. This workaround should be removed if we figure out how to disable animations in our test suite. commit c722e41213b2fae3208a9e16ed6119fded7bccdf Author: Lukas Olson <[email protected]> Date: Wed Oct 10 14:12:34 2018 +0100 Fixes #2180 commit 14e4e1744c53d60a046f75af442149ad5779461d Author: Leanid Shutau <[email protected]> Date: Wed Oct 10 15:46:53 2018 +0300 [I18n] Add one-time binding to angularjs i18n (#23499) * Add one-time binding to angularjs i18n * Add watcher for values property * Watch values field only if it is provided * Fix ci commit 424653021323c2a1312084baf60e1efed9caac7f Author: Brandon Kobel <[email protected]> Date: Wed Oct 10 04:27:15 2018 -0700 Skipping SAML tests, ES master is throwing NPEs (#23936) commit 6fea8859ee2551458fac2aea3da427eac7a0df32 Author: Catherine Liu <[email protected]> Date: Wed Oct 10 11:20:28 2018 +0100 Adds super select to font picker (#23855) * Adds super select to font picker. Removes fauxSelect component * Removed import for font_picker.scss commit 665c26606e633538274a7480a1b875a0dd7a2502 Author: Ryan Keairns <[email protected]> Date: Wed Oct 10 05:05:14 2018 -0500 removes unused less styles (#23759) commit aeb94a0d405284f95ee1af1c16488bad4fe40db3 Author: maryia-lapata <[email protected]> Date: Tue Oct 9 19:04:13 2018 +0300 Revert wrapping by I18nProvider commit c63267df8839b26858de0e63ad2a1f1de2b17a95 Author: maryia-lapata <[email protected]> Date: Tue Oct 9 08:26:55 2018 +0300 Update ids commit b647b178ccf237dc7465dd7799e1a484c9dba724 Merge: e7617a9485 c6911d43d9 Author: maryia-lapata <[email protected]> Date: Tue Oct 9 08:21:09 2018 +0300 Merge branch 'master' into feature/translations/tagcloud commit c6911d43d9444dcf9d56f6e6e2fdf8300d9cf308 Author: Catherine Liu <[email protected]> Date: Mon Oct 8 21:54:19 2018 +0100 fixes early return condition in dom_preview (#23894) commit e7617a94852d64b55ddb9fff00eb6f3324568bf3 Merge: a26a7e67d2 2a9cc02d34 Author: maryia-lapata <[email protected]> Date: Mon Oct 8 23:19:31 2018 +0300 Merge branch 'master' into feature/translations/tagcloud commit a26a7e67d2582a0c8f889cbdf93e28effbbe176d Author: maryia-lapata <[email protected]> Date: Mon Oct 8 13:33:23 2018 +0300 Use i18n from core commit 2a9cc02d347ad0f33e82a1f11c9a4a241ae16c8d Author: Josh Dover <[email protected]> Date: Mon Oct 8 02:52:06 2018 -0500 Prevent header popovers from scrolling with page content (#23850) commit 6217204c2f6c6637cb1386a53ad431a537a32618 Merge: e8f470c9cd 70c4e718a0 Author: maryia-lapata <[email protected]> Date: Mon Oct 8 08:25:09 2018 +0300 Merge branch 'master' into feature/translations/tagcloud commit 70c4e718a00ba8e1795feaab07a1ef8060a24564 Author: Aleh Zasypkin <[email protected]> Date: Fri Oct 5 21:00:26 2018 +0200 Add Kibana bootstrap step to generate types exposed by the core and its plugins. (#23686) commit 0dcef1ed1d1c9c694f75e43256bdc43c1559524e Author: Chris Roberson <[email protected]> Date: Fri Oct 5 13:22:35 2018 -0400 Fix tests for #23013 (#23883) commit 2780c0a80301beec6bb7f6716481f85161a019cd Author: Brandon Kobel <[email protected]> Date: Fri Oct 5 09:27:17 2018 -0700 Fixing the behavior when scrolling of the spaces popover (#23851) commit 2ada2403b06db7d08aec1fb37efcc7ef09b4354a Author: Walter Rafelsberger <[email protected]> Date: Fri Oct 5 17:44:52 2018 +0200 [ML] Makes mlExplorerDashboardService independent of angularjs (#23874) This is a refactor of mlExplorerDashboardService to make it available via import instead of angularjs dependency injection. This way it's also not necessary anymore to pass it on as a prop to ExplorerSwimlane, the component can now import the service by itself. commit c4ee9dd87eff168d41d786c902766d56f55f99c0 Author: Walter Rafelsberger <[email protected]> Date: Fri Oct 5 17:07:45 2018 +0200 [ML] Anomaly Explorer Rare/Population Charts (#23423) This PR introduces custom charts for detectors that use a rare function (Event Distribution Chart) as well as detectors that use an over field (Population Distribution Chart). commit 584100198f998969c938b6276161c4413d6c6632 Author: Catherine Liu <[email protected]> Date: Fri Oct 5 07:47:30 2018 -0700 Change single select combo box to plain text in esdocs (#23853) commit 2caf6ecb4f32b65537e196627dacf735ebaff994 Author: Chris Roberson <[email protected]> Date: Fri Oct 5 08:46:21 2018 -0400 [Monitoring] CCR UI (#23013) * Initial version of CCR monitoring UI * Adding missing files * Use icons * Use new column header text * Update tests * Basic of shard detail page * Do these in parallel * Disable time picker on ccr page * Remove summary for now * Remove unnecessary code here * Fix a few things on the shard page * Only send down what we need * update snapshot * Handle no ccr_stats documents * Ensure we fetch the latest * Updates * Format the time * Add api integration tests * Adding pagination and sorting * Updated query logic * Change this back * Add specific information about the follower and leader lag ops * Update tests * UI updates * Address PR issues * Fix tests * Update shapshots * Add timestamp * Update tests * Add a few snapshot tests * Use timezone formatter * Fix tests * Fix aligment of shard table * PR feedback * Update snapshots * Update snapshot commit a648d0bff3de6236da8c006da49dc063a480cef7 Author: Ryan Keairns <[email protected]> Date: Fri Oct 5 07:39:51 2018 -0500 Reporting SASS - Remove less styles (#23782) * remove less styles * remove less import commit 95b9851a08f5a2809a4960df09359adff8995e8b Author: Melissa Alvarez <[email protected]> Date: Fri Oct 5 12:44:07 2018 +0100 [ML] Ensure Calendar creation navigation tabs are keyboard/screen reader accessible (#23832) * Calendar nav links keyboard accessible * Prevent default click behavior * use event default value * remove unnecessary default param commit e8f470c9cda2a97b6c1af4051629db8bfc4183fb Author: maryia-lapata <[email protected]> Date: Fri Oct 5 13:57:08 2018 +0300 Fix review comments commit 13944bb5c0f453d9c46b3c84f3a0b6581d29e6fa Author: pavel06081991 <[email protected]> Date: Fri Oct 5 10:06:10 2018 +0300 i18n remove extra span tags (#23529) remove extra span tags generated by FormattedMessage component translate missed labels commit c5bbb41bd68177897c96f72745c83d71b4cccb17 Author: maryia-lapata <[email protected]> Date: Fri Oct 5 08:42:39 2018 +0300 Update ids commit 57b1a6ce715b3ff231d21cac3e5621745ddd82b3 Author: [email protected] <[email protected]> Date: Thu Oct 4 22:33:49 2018 -0700 Management core Less to Sass (#23596) Converts management's less to sass. Makes minor adjustments to those pages for some design cleanup. commit 9de0385ff1a623f25ff07c270924b1fa72caa353 Merge: cf9759a89b 03202be64a Author: maryia-lapata <[email protected]> Date: Fri Oct 5 07:46:56 2018 +0300 Merge branch 'master' into feature/translations/tagcloud commit 03202be64a128f0378264818f9346866e2c7405d Author: Josh Dover <[email protected]> Date: Thu Oct 4 16:12:44 2018 -0500 Fix regression with ML breadcrumbs in old UI (#23756) commit 42abc7df4a7e10a4159f1afe7859f9aff2f5af52 Author: Ryan Keairns <[email protected]> Date: Thu Oct 4 15:19:22 2018 -0500 change progress element titles to sentence casing (#23820) commit 85c62afc2fc215542dc4e2d05d195650f4789f30 Author: lcawl <[email protected]> Date: Thu Oct 4 13:13:34 2018 -0700 [DOCS] Fixes broken link in monitoring page commit 2fe176c6b2b97d9c47510f13c2fa44601b9f1932 Author: Brandon Kobel <[email protected]> Date: Thu Oct 4 11:55:20 2018 -0700 Reenable X-Pack Functional Tests (#23836) * Make saved object client error while Kibana index is migrating * Tidy up a bit, and refactor the way the `isMigrated` check is accessed * Remove unused interface declaration * Remove default migrator from saved objects repository constructor * Fix repository migrator isComplete check * Wrap callCluster and delay it until migrations have completed... * Fix inaccurate comment * Ensure migrations wait for elasticsearch to go green prior to running * Reenabling tests * Add tests for callCluster being wrapped in the repository, fix the es_archiver's call to migrate index. * Fixing esArchiver's usage of migrations * Disabling spaces for the phanton api BWC tests * don't throw if authorization mode is already initialized * Adding spaces to the reporting historical archives * Loading empty_kibana for grok debugger tests * Enabling reporting tests * Altering the method in which we logout users to be more fault tolerant * Actually doing what I said before... * Skipping Dashboard Preserve Layout, it likes to fail a lot * Skipping dashboard view mode tests * Putting logout back how it was, trying to make the security tests run properly when we don't have dashboard mode tests * Running subsection of tests that are failing * Don't bail, run them all * Disabling canvas, breaks logout * Fixing spaces create legacy error assertion * Putting comment about why we're disabling spaces for the functional tests commit 3a9deb0850f6620e1544e4ec8298f86bbd3a586b Author: Lisa Cawley <[email protected]> Date: Thu Oct 4 11:31:15 2018 -0700 [DOCS] Update Kibana monitoring tasks (#23736) commit 8b0b5b3ac663c24a1bc03e00bbad057501ebb3d4 Author: Stacey Gammon <[email protected]> Date: Thu Oct 4 13:27:48 2018 -0400 Tests: Wait for dashboard save button to be enabled before clicking. (#23539) * Fixes #21446 An attempt to fix the above by making sure the click only happens when the button is enabled. * Fix wrong function name * Fix mistakes commit 37bed9b51be92b13378278e5768f08f23b6588b0 Author: [email protected] <[email protected]> Date: Thu Oct 4 09:35:29 2018 -0700 Eui 4.4.1 (#23790) Updates EUI to 4.4.1 and includes some minor homepage changes around icons. commit c2bae26e874ee58cdefa5fff7aa60392d079e4c1 Author: Jonathan Budzenski <[email protected]> Date: Thu Oct 4 11:10:20 2018 -0500 [tests/browser] generate css before testsBundle, include css (#23794) commit 95edbcdfbf6d3358bd50d6802859966639c29c46 Author: Leanid Shutau <[email protected]> Date: Thu Oct 4 11:30:59 2018 +0300 [I18n] Update TS types in i18n engine (#23754) * [I18n] Export i18n service type * Add InjectedIntl export and context type commit b2baf32fba2f09b034324489bd0c2bbb21bcb668 Author: Aleh Zasypkin <[email protected]> Date: Thu Oct 4 09:18:40 2018 +0200 Expose core config schema validation system as `@kbn/config-schema` package. (#23609) commit 125e4fa6ad03c18e8686f098c3d2cf7e0f59bd54 Author: Larry Gregory <[email protected]> Date: Wed Oct 3 19:10:20 2018 -0400 don't throw if authorization mode is already initialized (#23791) commit b6b6ebb5c49e7891694ce936bef0ecf6f0afeb4b Author: Chris Davies <[email protected]> Date: Wed Oct 3 17:26:35 2018 -0400 Make saved object client await migrations prior to calling Elasticsearch (#23709) * Make saved object client error while Kibana index is migrating * Tidy up a bit, and refactor the way the `isMigrated` check is accessed * Remove unused interface declaration * Remove default migrator from saved objects repository constructor * Fix repository migrator isComplete check * Wrap callCluster and delay it until migrations have completed... * Fix inaccurate comment * Ensure migrations wait for elasticsearch to go green prior to running * Add tests for callCluster being wrapped in the repository, fix the es_archiver's call to migrate index. * Fix es_archiver's kbnServer mock commit 84d4b0dc7358954171ebd835cfd585ed3f46cc6c Author: Shaunak Kashyap <[email protected]> Date: Wed Oct 3 11:49:38 2018 -0700 Relax check to account for metricbeat-indexed doc format (#23730) With Metricbeat shipping Elasticsearch monitoring data (instead of internal collection by Elasticsearch), there are some subtle changes to the format of monitoring docs that are indexed into `.monitoring-es-6-*`. One such change is that metricbeat won't index fields with `null` values; instead it simply doesn't index such fields at all. As a result, in the context of Elasticsearch monitoring docs, when it comes to docs with `type` = `shards` representing unassigned shards, the `shard.node` field was being indexed as `null` by internal Elasticearch collection, whereas the field was absent when the doc was indexed by Metricbeat. Since both cases represent the same case — the shard being unassigned — this PR relaxes the check in the UI code to look for either case. ### Sample `shards` document indexed by internal ES collection ```js { "_index":".monitoring-es-6-2018.10.02", "_type":"doc", "_id":"WUf_htOeSXOJQmiesyF5Bw:_na:metricbeat-7.0.0-alpha1-2018.10.01:0:r", "_source":{ "cluster_uuid":"zXO1GjA6SJGsrPnCPkOoyA", "timestamp":"2018-10-02T03:54:43.364Z", "interval_ms":10000, "type":"shards", "source_node":null, "state_uuid":"WUf_htOeSXOJQmiesyF5Bw", "shard":{ "state":"UNASSIGNED", "primary":false, "node":null, "relocating_node":null, "shard":0, "index":"metricbeat-7.0.0-alpha1-2018.10.01" } } } ``` ### Sample `shards` document indexed by Metricbeat collection ```js { "_index":".monitoring-es-6-mb-2018.10.02", "_type":"doc", "_id":"FhDRTPjkQJqsgawYbxjQzw:_na:metricbeat-7.0.0-alpha1-2018.10.01:0:r", "_source":{ "@timestamp":"2018-10-02T04:00:03.361Z", "interval_ms":10000, "shard":{ "state":"UNASSIGNED", "primary":false, "index":"metricbeat-7.0.0-alpha1-2018.10.01", "shard":0 }, "state_uuid":"FhDRTPjkQJqsgawYbxjQzw", "beat":{ "hostname":"Shaunaks-MBP-2", "version":"7.0.0-alpha1", "name":"Shaunaks-MBP-2" }, "timestamp":"2018-10-02T04:00:03.375Z", "type":"shards", "metricset":{ "name":"shard", "module":"elasticsearch", "host":"localhost:9200", "rtt":14254, "namespace":"elasticsearch.shard" }, "host":{ "name":"Shaunaks-MBP-2" }, "cluster_uuid":"zXO1GjA6SJGsrPnCPkOoyA" } } ``` commit e7290b90aa2a545d4a1683700f5dce577b4f554f Author: Alex F <[email protected]> Date: Wed Oct 3 13:55:04 2018 -0400 eCommerce Sample Data (#23214) :shipit: commit 9f10f6c696ad97c0ac6e71e938ba7e8690cbd401 Author: Larry Gregory <[email protected]> Date: Wed Oct 3 12:43:44 2018 -0400 Handle case where space name is made entirely of whitespace (#23691) * handle case where space name is made entirely of whitespace * update space name validation commit b10992c182d68fd8a1645e20b087927b3a0fb311 Author: Catherine Liu <[email protected]> Date: Wed Oct 3 09:32:53 2018 -0700 Added checks in dom_preview to fix style null bug (#23706) * Added checks in dom_preview to fix style null bug * Added early return in dom preview commit c3d48a005125c46f6d30ae539cbd610229bb2f5a Author: Catherine Liu <[email protected]> Date: Wed Oct 3 09:32:13 2018 -0700 [WIP] Removes server functions from webpack bundle (#23290) * Removed server functions from webpack bundle. Copies server files from canvas_plugins_src to canvas_plugins * Moved server functions * Installed CopyWebpackPlugin to copy server functions from canvas_plugin_src to canvas_plugin * Added canvas_plugin_src to cluster manager ignore list * Revert plugins task changes * ignores __tests__ folder * Added task to delete canvas_plugin before build * Fixed bug in canvas:plugins:build-prod * Updated yarn lock commit 9ba4c9ac6b83f96321444873b89fed4ae29c4899 Author: Chris Roberson <[email protected]> Date: Wed Oct 3 12:26:54 2018 -0400 [Monitoring] Ensure we use the provided node id in the query (#23715) * Ensure we use the right parameter name * Update test fixture to use second node commit 937e07c5f167663bafed0beb189b2f5ffce10e8c Author: Brandon Kobel <[email protected]> Date: Wed Oct 3 09:09:12 2018 -0700 Limiting maximum number of Spaces (#23673) * Limiting the number of spaces * Adding docs * Adding forgotten fixture * Fixing tslint error * Adjusting docs * Changing test descriptions from Boom.badRequest to bad request * Updating error snapshots commit c0eec4dd602e0e96cc7c91ceb9d382bf530d9184 Author: Ryan Keairns <[email protected]> Date: Wed Oct 3 11:03:22 2018 -0500 misc ui bug fixes (#23629) commit 88c5c6d93ca0a9f0e61a32beadb73be2045aff74 Author: Ryan Keairns <[email protected]> Date: Wed Oct 3 10:56:12 2018 -0500 Watcher - convert LESS to SASS (#23252) * convert watch less to sass * add temp workaround for loading new styles * use new style path commit fd050fbcd3eec5c2ca650c8cb6297fac5b0cbf92 Author: Court Ewing <[email protected]> Date: Wed Oct 3 11:35:46 2018 -0400 docs: note about permissions for grok debugger (#23664) commit 557fc7a66f14b64fd9be80680bc9942c01941686 Author: Pete Harverson <[email protected]> Date: Wed Oct 3 16:09:52 2018 +0100 [ML] Indicate multi-bucket anomalies in results dashboards (#23746) commit c993ad3996ef6f21739847e070f412dd21bdf1f5 Author: Leanid Shutau <[email protected]> Date: Wed Oct 3 17:57:04 2018 +0300 [I18n] Add HOC injecting i18n provider (#23683) * add injectI18nProvider HOC * Fix propTypes typo * Typescriptify wrapper * Add tests * Fix tests * Resolve comments commit 2f62fd69783e1ab99d0aa80b3eac706459903d3d Author: Leanid Shutau <[email protected]> Date: Wed Oct 3 17:56:07 2018 +0300 [I18n] Fix types paths for kbn-i18n package (#23744) * [I18n] Fix types paths for kbn-i18n package * Remove module field from package.json commit 1d7adee4856c8469b4743726ca163508ad2bb35c Author: Thomas Watson <[email protected]> Date: Wed Oct 3 13:49:34 2018 +0200 chore: fix spelling of APM Server (#23729) commit 1311d89b24db4afa7f5e3423296c7d8aa94ee652 Author: Melissa Alvarez <[email protected]> Date: Wed Oct 3 11:19:20 2018 +0100 [ML] Ensure charts loaded in Anomaly Explorer match swimlane selection (#23690) * Only consider last request.Prevent promise race condition * Reminder for regression test commit 57b4b144fc207b0f6f93bfc5dbe318c2aafb86ea Author: Robert Monfera <[email protected]> Date: Wed Oct 3 09:08:48 2018 +0200 Feat: group resize for horizontal constituents (#23553) Feat: group resize for horizontal constituents commit 52df40e42f18d77f424b4a839affc2df79626c95 Author: Robert Monfera <[email protected]> Date: Wed Oct 3 07:23:22 2018 +0200 Fix: browser back button after workpad switch should be handled specially (#23619) commit 2da50a9085046c55c74ae46b139cb2b13ebeca53 Author: Catherine Liu <[email protected]> Date: Tue Oct 2 19:18:23 2018 -0700 Fix: setState warnings in Canvas (#23671) * Added check for mounted workpad loader before setState calls * Added check for mounted page manager before setState calls * Added check for mounted arg form before setState calls * Resets onmousemove and onmouseup handlers when workpad page unmounts commit 17f11ccc53ab843497499a7eae30a531db45ab34 Author: Nathan Reese <[email protected]> Date: Tue Oct 2 17:04:26 2018 -0600 do not call set state when unmounted (#23711) commit 30929fad79255d9166f17c3fd9051e9ef75ee4fa Author: Catherine Liu <[email protected]> Date: Tue Oct 2 14:11:20 2018 -0700 Fix: page preview default font color (#23672) * Changes default font color to black in page_preview * Switched to euiTextColor commit 981e98c2fee50192ce3bae499af1b57ce2f2cb00 Author: Larry Gregory <[email protected]> Date: Tue Oct 2 16:45:28 2018 -0400 Fix space privilege associations when editing roles (#23638) This PR updates the role management screen so that changes to space privileges are correctly tracked when adding/updating/deleting both new and existing privilege associations. We were not tracking state correctly when both existing and in-progress privileges existed on screen. Closes #23541 commit e44113393a13e1e093e0d2a94ca867de9316bc79 Author: Nathan Reese <[email protected]> Date: Tue Oct 2 13:55:27 2018 -0600 Fix child controls don't work after parent reset (#23616) * Fix child controls don't work after parent reset * do not clear value on disable - this breaks values provided from kibana filters as highlighted by the broken functional test commit cce9a682de7b058cf293e4f627514e613e7e768f Author: Chris Davies <[email protected]> Date: Tue Oct 2 15:54:45 2018 -0400 Fix a bug with reindex timing out during migration of largish indices (#23397) Modify migrations to poll for realias completion to avoid a timeout on larger indices commit 2040cd501cfbcb86c72536bc6ecb44d29440a6dd Author: Nathan Reese <[email protected]> Date: Tue Oct 2 13:53:48 2018 -0600 Use EuiPanel to for dashboard panels (#22078) * Use EuiPanel to for dashboard panels * Fixed styles (#27) * Fixed styles - Removed extraneous panel styles no longer needed - Fixed overflow issue in FF * Couple classname adjustments * removed styles in expanded mode * remove styles in expanded mode commit 49798bc8adb9b3c9832fe82bf7cb4fb540bf3457 Author: Josh Dover <[email protected]> Date: Tue Oct 2 14:09:47 2018 -0500 Add K7 header navigation (#23300) * Add basic support for new K7 navigation * Make visibility and app title work * Allow nav controls on right side of navbar * Use render callback w/ el * Add support for multiple sides * Remove fake spaces nav control * Breadcrumb support * Hide breadcrumbs in plugins when k7design is enabled: * Fix units * Rename k7 -> header * Add tests * Fix tests * Fix loading indicator * PR comments * Move ts-ignore * Use canvasApp icon type commit f74b4bfdac76d61fcb013ad925a3e9e975532c57 Author: Chris Davies <[email protected]> Date: Tue Oct 2 14:09:54 2018 -0400 Fixes relative timezone bug #18133 for Chromium reports (#23652) commit e9d23f64f7807ad66aed60a2a7d0cf720dfac668 Author: Larry Gregory <[email protected]> Date: Tue Oct 2 13:31:42 2018 -0400 Fix spaces table rendering in IE (#23608) This fixes table rendering in IE where we display the Space Avatar alongside the Space Name. The solution is to[ render them in separate columns](https://github.com/elastic/kibana/issues/23546#issuecomment-425108806), instead of a single column. Screenshots from IE: ![fixed spaces cutoff](https://user-images.githubusercontent.com/3493255/46208213-036db700-c2f8-11e8-9a43-67bb42b7c788.png) ![fixed spaces cutoff 2](https://user-images.githubusercontent.com/3493255/46208216-0668a780-c2f8-11e8-94e0-454c51d543e2.png) Closes #23546 commit 6932cf2b175bda495a0335515fb58bdcd10040b3 Author: Ryan Keairns <[email protected]> Date: Tue Oct 2 12:30:29 2018 -0500 Search Profiler - convert LESS to SASS (#23588) * converts less to sass * IE fixes and misc tweaks * feedback * use bem css class naming commit 5c6ebc76f43909de8314056f8d0ad0342bd3de24 Author: Larry Gregory <[email protected]> Date: Tue Oct 2 13:29:50 2018 -0400 Fix error handling on role management screen (#23583) Fixes #23542 - old error handling was not working when API calls to create/update roles returned an error commit 4c1c04cb4075487eef104ccb3097e5de99455518 Author: Stacey Gammon <[email protected]> Date: Tue Oct 2 12:19:28 2018 -0400 Reporting test readme (#23507) * Reporting test readme * Use full urls * more full paths * Don't use link to session folder, it's not in repo. * updates * Consolidate all reporting information into the readme and link from main x-pack readme. * be consistent with Note: styling * Add windows steps for downloading the correct packages. commit a839f7f4034c2764f6a162ba6286afd4747b6ac2 Author: Brandon Kobel <[email protected]> Date: Tue Oct 2 09:09:05 2018 -0700 When we get a 403 trying to get the telemetry document, assume we (#23631) haven't opted into telemetry commit 3d50ef741ac44ea59f91460fda99a7996425e67e Author: Nathan Reese <[email protected]> Date: Tue Oct 2 09:58:17 2018 -0600 Fix sample data install toasts error when user navigates away from home app while installing (#23574) * do not call functions on after component has been unmounted * use isDefault method when checking if defaultIndex config is set commit cea1301127a5611c840924f81d06645ff75388c7 Author: Josh Dover <[email protected]> Date: Tue Oct 2 10:19:33 2018 -0500 Remove elasticsearch package from kbn-es (#23662) commit 8d028663216e81179871af672c84af44d5e46de8 Author: Tim Sullivan <[email protected]> Date: Tue Oct 2 06:57:14 2018 -0700 [Reporting] Chromium wait until domcontentloaded not networkidle0 (#23586) Kibana now keeps a constant connection between the browser and the server from Canvas's websocket. When there's a reverse proxy between the server and the browser, the fallback is XHR polling. This open polling connection was keeping the network alive all the time, never idle, which resulted in the Chromium browser driver kept waiting. Eventually, the Report job would fail with a timeout error. commit 52723502bc1e0f8570bb513c7ac7e3b9205b1807 Author: Chris Roberson <[email protected]> Date: Tue Oct 2 09:49:54 2018 -0400 [Beats] Add space.id to all filebeat and metricbeat tutorials (#22998) * Add space id to all filebeat and metricbeat tutorials * Do not show if default or does not exist. Also, move to a helper method as the logic is fairly complex now. * Add comment * Provide a boolean indicating if the current space is the default one on the context object * Remove debug * PR feedback * Fix prettier issue commit a002ee436982f7ba9535aaf66fa0694fc3c076d4 Author: Aliaksandr Yankouski <[email protected]> Date: Tue Oct 2 01:55:15 2018 -0700 i18n engine typescript migration (#22441) * configure typescript build, add necessary dependencies, change extensions, react migration * migrate lib files in root * update tests snapshots, resolve core loader, helper * fix types for core components * fix angular components * fix angular staff * use Messages type * first-upper-case letter while using classs * use stable latest babel, fix ts issues * optimize .babelrc * update lock file * Fix x-pack/yarn.lock * fix issue with unknown babel plugin * add babel-config.js file with babel configuration for i18n engine build process instead of .babelrc file to fix jest issue * Resolve comments * Fix babel config * Fix packages incompatibility issue * Fix tslint errors * Fix tests * Resolve comments * Fix types commit 6b3bc45b9aa41e0e2a73a2bc62752af63801e9ac Author: Jonathan Budzenski <[email protected]> Date: Mon Oct 1 16:15:22 2018 -0500 [field caps] filter nested and object fields (#23658) * [field caps] filter nested and object fields * update type list test * update snapshots commit 1f380267316e9a89f1bf07d1522031d70fbdc84e Author: Larry Gregory <[email protected]> Date: Mon Oct 1 07:09:33 2018 -0400 Spaces Phase 1 (#21408) ### Review notes This is generally ready for review. We are awaiting https://github.com/elastic/elasticsearch/issues/32777 to improve handling when users do not have any access to Kibana, but this should not hold up the overall review for this PR. This PR is massive, there's no denying that. Here's what to focus on: 1) `x-pack/plugins/spaces`: This is, well, the Spaces plugin. Everything in here is brand new. The server code is arguably more important, but feel free to review whatever you see fit. 2) `x-pack/plugins/security`: There are large and significant changes here to allow Spaces to be securable. To save a bit of time, you are free to ignore changes in `x-pack/plugins/security/public`: These are the UI changes for the role management screen, which were previously reviewed by both us and the design team. 3) `x-pack/test/saved_object_api_integration` and `x-pack/test/spaces_api_integration`: These are the API test suites which verify functionality for: a) Both security and spaces enabled b) Only security enabled c) Only spaces enabled What to ignore: 1) As mentioned above, you are free to ignore changes in `x-pack/plugins/security/public` 2) Changes to `kibana/src/server/*`: These changes are part of a [different PR that we're targeting against master](https://github.com/elastic/kibana/pull/23378) for easier review. ## Saved Objects Client Extensions A bulk of the changes to the saved objects service are in the namespaces PR, but we have a couple of important changes included here. ### Priority Queue for wrappers We have implemented a priority queue which allows plugins to specify the order in which their SOC wrapper should be applied: `kibana/src/server/saved_objects/service/lib/priority_collection.ts`. We are leveraging this to ensure that both the security SOC wrapper and the spaces SOC wrapper are applied in the correct order (more details below). ### Spaces SOC Wrapper This wrapper is very simple, and it is only responsible for two things: 1) Prevent users from interacting with any `space` objects (use the Spaces client instead, described below) 2) Provide a `namespace` to the underlying Saved Objects Client, and ensure that no other wrappers/callers have provided a namespace. In order to accomplish this, the Spaces wrapper uses the priority queue to ensure that it is the last wrapper invoked before calling the underlying client. ### Security SOC Wrapper This wrapper is responsible for performing authorization checks. It uses the priority queue to ensure that it is the first wrapper invoked. To say another way, if the authorization checks fail, then no other wrappers will be called, and the base client will not be called either. This wrapper authorizes users in one of two ways: RBAC or Legacy. More details on this are below. ### Examples: `GET /s/marketing/api/saved_objects/index-pattern/foo` **When both Security and Spaces are enabled:** 1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function 2) The Security wrapper is invoked. a) Authorization checks are performed to ensure user can access this particular saved object at this space. 3) The Spaces wrapper is invoked. a) Spaces applies a `namespace` to be used by the underlying client 4) The underlying client/repository are invoked to retrieve the object from ES. **When only Spaces are enabled:** 1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function 2) The Spaces wrapper is invoked. a) Spaces applies a `namespace` to be used by the underlying client 3) The underlying client/repository are invoked to retrieve the object from ES. **When only Security is enabled:** (assume `/s/marketing` is no longer part of the request) 1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function 2) The Security wrapper is invoked. a) Authorization checks are performed to ensure user can access this particular saved object globally. 3) The underlying client/repository are invoked to retrieve the object from ES. ## Authorization Authorization changes for this project are centered around Saved Objects, and builds on the work introduced in RBAC Phase 1. ### Saved objects client #### Security without spaces When security is enabled, but spaces is disabled, then the authorization model behaves the same way as before: If the user is taking advantage of Kibana Privileges, then we check their privileges "globally" before proceeding. A "global" privilege check specifies `resources: ['*']` when calling the [ES _has_privileges api.](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-has-privileges.html). Legacy users (non-rbac) will continue to use the underlying index privileges for authorization. #### Security with spaces When both plugins are enabled, then the authorization model becomes more fine-tuned. Rather than checking privileges globally, the privileges are checked against a specific resource that matches the user's active space. In order to accomplish this, the Security plugin needs to know if Spaces is enabled, and if so, it needs to ask Spaces for the user's active space. The subsequent call to the `ES _has_privileges api` would use `resources: ['space:marketing']` to verify that the user is authorized at the `marketing` space. Legacy users (non-rbac) will continue to use the underlying index privileges for authorization. **NOTE** The legacy behavior implies that those users will have access to all spaces. The read/write restrictions are still enforced, but there is no way to restrict access to a specific space for legacy auth users. #### Spaces without security No authorization performed. Everyone can access everything. ### Spaces client Spaces, when enabled, prevents saved objects of type `space` from being CRUD'd via the Saved Objects Client. Instead, the only "approved" way to work with these objects is through the new Spaces client (`kibana/x-pack/plugins/spaces/lib/spaces_client.ts`). When security is enabled, the Spaces client performs its own set of authorization checks before allowing the request to proceed. The Spaces client knows which authorization checks need to happen for a particular request, but it doesn't know _how_ to check privileges. To accomplish this, the spaces client will delegate the check security's authorization service. #### FAQ: Why oh why can't you used the Saved Objects Client instead!? That's a great question! We did this primarily to simplify the authorization model (at least for our initial release). Accessing regular saved objects follows a predictible authorization pattern (described above). Spaces themselves inform the authorization model, and this interplay would have greatly increased the complexity. We are brainstorming ideas to obselete the Spaces client in favor of using the Saved Objects Client everywhere, but that's certainly out of scope for this release. ## Test Coverage ### Saved Objects API A bulk of the changes to enable spaces are centered around saved objects, so we have spent a majority of our time automating tests against the saved objects api. **`x-pack/test/saved_object_api_integration/`** contains the test suites for the saved objects api. There is a `common/suites` subfolder which contains a bulk of the test logic. The suites defined here are used in the following test configurations: 1) Spaces only: `./spaces_only` 2) Security and spaces: `./security_and_spaces` 3) Security only: `./security_only` Each of these test configurations will start up ES/Kibana with the appropriate license and plugin set. Each set runs through the entire test suite described in `common/suites`. Each test with in each suite is run multiple times with different inputs, to test the various permutations of authentication, authorization type (legacy vs RBAC), space-level privileges, and the user's active space. ### Spaces API Spaces provides an experimental public API. **`x-pack/test/spaces_api_integration`** contains the test suites for the Spaces API. Similar to the Saved Objects API tests described above, there is a `common/suites` folder which contains a bulk of the test logic. The suites defined here are used in the following test configurations: 1) Spaces only: `./spaces_only` 2) Security and spaces: `./security_and_spaces` ### Role Management UI We did not provide any new functional UI tests for role management, but the existing suite was updated to accomidate the screen rewrite. We do have a decent suite of jest unit tests for the various components that make up the new role management screen. They're nested within `kibana/x-pack/plugins/security/public/views/management/edit_role` ### Spaces Management UI We did not provide any new functional UI tests for spaces management, but the components that make up the screens are well-tested, and can be found within `kibana/x-pack/plugins/spaces/public/views/management/edit_space` ### Spaces Functional UI Tests There are a couple of UI tests that verify _basic_ functionality. They assert that a user can login, select a space, and then choose a different space once inside: `kibana/x-pack/test/functional/apps/spaces` ## Reference Notable child PRs are listed below for easier digesting. Note that some of these PRs are built on other PRs, so the deltas in the links below may be outdated. Cross reference with this PR when in doubt. ### UI - Reactify Role Management Screen: https://github.com/elastic/kibana/pull/19035 - Space Aware Privileges UI: https://github.com/elastic/kibana/pull/21049 - Space Selector (in Kibana Nav): https://github.com/elastic/kibana/pull/19497 - Recently viewed Widget: https://github.com/elastic/kibana/pull/22492 - Support Space rename/delete: https://github.com/elastic/kibana/pull/22586 ### Saved Objects Client - ~~Space Aware Saved Objects: https://github.com/elastic/kibana/pull/18862~~ - ~~Add Space ID to document id: https://github.com/elastic/kibana/pull/21372~~ - Saved object namespaces (supercedes #18862 and #21372): https://github.com/elastic/kibana/pull/22357 - Securing saved objects: https://github.com/elastic/kibana/pull/21995 - Dedicated Spaces client (w/ security): https://github.com/elastic/kibana/pull/21995 ### Other - Public Spaces API (experimental): https://github.com/elastic/kibana/pull/22501 - Telemetry: https://github.com/elastic/kibana/pull/20581 - Reporting: https://github.com/elastic/kibana/pull/21457 - Spencer's original Spaces work: https://github.com/elastic/kibana/pull/18664 - Expose `spaceId` to "Add Data" tutorials: https://github.com/elastic/kibana/pull/22760 Closes #18948 "Release Note: Create spaces within Kibana to organize dashboards, visualizations, and other saved objects. Secure access to each space when X-Pack Security is enabled" commit 76c0a0a5463d328a0fb4ee36af51f37a2a6158ba Author: Josh Dover <[email protected]> Date: Fri Sep 28 11:51:06 2018 -0500 Handle different junit XML formats (#23617) commit c44075f253d0302c403f97756abb796fbf9301b4 Author: Josh Dover <[email protected]> Date: Fri Sep 28 11:47:36 2018 -0500 Update kbn-pm build (#23621) commit abb3fcf53ec4e841f54697751179995e5bb3897c Author: [email protected] <[email protected]> Date: Fri Sep 28 09:36:31 2018 -0700 [PROPOSAL] Make Kibana's PR template a checklist (#23511) Kibana now uses a checklist for its PR template. The contributing docs were updated with more detail about release process. commit 86caf52a5747176e34f7b896b9b198254a7176fc Author: Catherine Liu <[email protected]> Date: Fri Sep 28 08:56:16 2018 -0700 Feat: Progress Elements (#23176) * Adds progress function and elements Added progress elements Added progress view Added unit tests for progress common function Fixed prop type in toggle arg Renamed vert -> vertical and horiz -> horizontal Adjusted progress element dimensions Removed check for null context in progress function Refactored progress shapes Added unicorn shape Adds labelPosition arg Added tests for labelPosition * Added percentage column to demodata * Updated elements to use percent_uptime in demodata * Updated demodata percent values * Refactored progress to use SVGs instead of shape defs * Added barWeight arg to progress function * Removed labelPosition arg. Set static label position for each progress shape * Added label to unicorn shape * Fixed element images commit 046430f876f6024aef3fd585d6c238f3c908493a Author: Rashid Khan <[email protected]> Date: Fri Sep 28 07:41:00 2018 -0700 Remove debug messages in Canvas (#23615) commit 34abe9762b14d3638daa06b6721d5bde24fd028e Author: Walter Rafelsberger <[email protected]> Date: Fri Sep 28 14:07:11 2018 +0200 [ML] Fix view link regression. (#23604) Fixes a regression introduced in #23494. The view link was broken because it expects a callback with an action instead of just the link. commit 7f1ee07405ff30dbac9513bba3e2fd614916f52e Author: Martijn Rondeel <[email protected]> Date: Fri Sep 28 10:47:27 2018 +0200 Add ElastAlert Kibana Plugin to known plugins list (#23598) * Add ElastAlert Kibana Plugin to known plugins list * Add author of ElastAlert plugin commit 37d3e54bd91ddf3488052944f4fd0ee3894b4d7f Author: Catherine Liu <[email protected]> Date: Thu Sep 27 16:36:55 2018 -0700 Added dataurl=null to default image expression (#23582) commit 16b4f85151ba3109f054d47801db0c90105679c9 Author: Brian Gaddis <[email protected]> Date: Thu Sep 27 18:28:34 2018 -0400 Fix error with reporting urls generated prior to 6.2 (when there was no layout parameter) (#23508) * removed passing of ID in to creation of layout and added a new test without a layout param * renamed test constant so it reflects what we are testing and added comment * Recommended Changes * removed */ from end of line commit 91c7fbc9f242f5c7495c0201022c6c5e3b568620 Author: Caroline Horn <[email protected]> Date: Thu Sep 27 18:11:26 2018 -0400 More Kibana plugin LESS 2 SASS (#23413) This PR removes the LESS files for dev_tools, context, console, and inspector_views and replaces them with Sass. commit 6bb4355c3963f7390cf68e51587710978c4aa5dc Author: Justin Kambic <[email protected]> Date: Thu Sep 27 16:58:23 2018 -0400 [Logstash Management] Euify pipeline (#22902) * Begin replacing pipeline editor KUI elements. * WIP build out EUI rendering of Create Pipeline view. * Add settings components. * Add close functionality. * Add save functionality. * Add temporary dependency hack for testing purposes until EUI XY Chart replaces jquery-flot. * Add delete pipeline button/capability. * Add delete modal. * Remove TODO comment. * Added toasts. * Switch to global toast system. * Add toast for inactive license and readonly state. * Remove pipeline edit template. * Add notify on PUT and DELETE errors. * Add null check for username prop of securityService return value. * Add disable save button if invalid ID. * Remove pipeline id field when editing existing pipeline. * Remove obsolete code. * Move PipelineEditor component to dedicated file. * Add EUI table to pipeline list view. * Add search to pipelines table. * Add create/delete pipelines buttons. * Add pagination stubs. Complete after EUI bug resolved. * Added unselectable for non-centrally-managed pipelines. * Add clone button to pipelines list. * Add min page height. Fix bug with edit pipeline link. * Remove obsolete pipeline list code. * Remove obsolete tooltip, edit, list code. * Disable create pipeline if id is empty. * Move PipelineList component to dedicated file. * Add empty state to pipeline list. Add selection messages. * Update loading message. * Move methods to more logical positions in component. * Add info alerts to pipeline list. * Remove obsolete angular template. * Remove obsolete imports from pipeline list directive. * Define UpgradeFailure component. * Move UpgradeFailure subcomponents to dedicated files. Write tests. * Move PipelineEditor subcomponents to dedicated files. * Write tests for pipeline editor subcomponents. * Move bare strings into constants. * Move PipelineEditor constant values into constants file. * Break subcomponents of InfoAlerts component into dedicated files. * Remove obsolete constants. * WIP - write tests for PipelineList, break table into separate component, add error empty prompt message. * Move ConfirmDeleteModal component to dedicated file and test. * Add TODO comment. * Add test tags to react components. * Add 'data-test-subj' prop to fields for func tests. Minor layout update. Run prettier on some files. * Add data-test-subj prop to button. Disable two tests until pagination is re-added. * Re-enabled pagination for pipeline list. * Remove wallaby hack. * Update pagination options, remove obsolete code. * Fix bug introduced in refactor to display delete button at appropriate time. * Handle max_bytes setting correctly. Add theme/mode to code editor. * Update snapshot for new pagination. * Remove angular template for UpgradeFailure view. * Move bare text from Modal functional component to constant file. Update test + snapshot. * Ran prettier on all changed documents. * Remove obsolete TODO comments. * Re-enable disabled functional test. Clean up TODO code. * Fix unresolved promise in functional tests. * Pipeline delete button hidden unless enabled, move to left. * Make filter title more readable. * Apply width to clone column on Pipeline List. * Modify pipeline edit view to use 's'-size icons. * Change pipeline editor delete button to empty button. * Move pipeline edit actions to bottom left of form. * Add propTypes for PipelineEditor. * Update test snapshots. * Update pipeline list delete button func test. * Add pipeline edit test. Add heading to pipeline edit page. * Move constant files to modules where they are consumed. * Move UPGRADE_FAILURE constants into module that consumes them. * Remove redundant tests and remove text constant imports from tests. * Give initial values to id and description text fields to make them controlled components. * Clean up pipeline ID form regex validation and add tests. commit 95f48c584eb881e20afc95c5f5b8ee2d712cc59a Author: Nathan Reese <[email protected]> Date: Thu Sep 27 14:49:41 2018 -0600 Migrate report listing management to react and EUI (#22928) * EUIify report management page * wire ReportListing component together * fetch jobs and display content in EuiPage * display jobs in table * add title and remove page size dropdown * format date and display date in status column * add poller * add download button * report error button * remove old reporting table * fix page styling * create type for job * remove job queue service * remove angular-paging dependency from x-pack * make download lib, update job notification hack to use jobQueueClient * fix some more typescript stuff * remove last angular service * make report object type subdued color and small text * update import in canvas * stricter typing * fix stuff lost in branch merge * add return types to JobQueueClient * wrap javascript code in {} in JSX commit 0ff498d5c4609c456e867e17bec2f84237d6d75f Author: Walter Rafelsberger <[email protected]> Date: Thu Sep 27 21:10:11 2018 +0200 [ML] Improve Explorer Chart labels. (#23494) Improves the display of the Explorer Chart labels to fix the following issues: - Long chart labels could be cut off, so it's not possible to tell what entity fields a chart is referring to. A workaround is to hover the info icon tooltip but that's really slow and cumbersome if you have to do it for every chart. - The list of entity fields and its values is an unformatted text blob which makes it hard to read and tell which values refer to which field. Changes: - If any of the chart labels is longer than 60 chars, the entity fields will wrap to a new line (for all charts to a achieve a consistent look). - Entity fields use EuiBadge and some custom formatting to make it easier to see field/value pairs. - If the detector description is too long, it still uses ellipsis for text-overflow: - If the entity badges are too long, they will be just cut off to the right. There's no simple CSS fix for that, we cannot use ellipsis and we don't want to wrap those badges again because then multiple charts could have different heights. I experimented with gradients but that turned out to be somewhat unreliable. I still consider this a good enough improvement compare to the previous version and would like to leave a tweak for that to a follow up PR. - If there are mixed detectors with and without entity fields and the existing one wrap, multiple charts are aligned considered the height of the entity fields on display: - Additionally, this changes the link to the single series viewer from custom code using a Font Awesome icon to use EuiButtonEmpty with the same EUI based icon and a tooltip. commit 186cea2d743444d8a591214846e14427d7819cc8 Author: Lisa Cawley <[email protected]> Date: Thu Sep 27 10:09:14 2018 -0700 [DOCS] Adds TLS info to licensing page (#20638) commit b778d53e9e3503d8f91460d5d35b0d5e5cd3fd26 Author: Josh Dover <[email protected]> Date: Thu Sep 27 12:08:37 2018 -0500 Fix plugin generator when using hacks and SCSS [ci skip] (#23579) commit 7e4e0cb84cd856526f4ba3256e5b8bd249c932c3 Author: Walter Rafelsberger <[email protected]> Date: Thu Sep 27 18:00:03 2018 +0200 [ML] Fixes Anomaly Explorer IE11 issues (#23558) Fixes two issues in IE11 for Anomaly Explorer: - The format of the string returned from element.attr('transform') is different in IE11 so the regex based on it would fail. This fixes the issue and adds tests for the different formats. The code was also changed to gracefully return NaN in case the regex wouldn't return results, the previous version triggered a JS error. - The migration of the swimlanes to React caused the cell selection to malfunction in IE11. This fixes it by updating the dragSelect library to use the new method setSelectables. The previous method we used (addSelectables) didn't play well with how React rerenders the swimlanes. Note this lib update using the new method will require to run yarn kbn bootstrap. commit ecaf26edd08d7e7bb43dcf498b2ae59cd91e9805 Author: CJ Cenizal <[email protected]> Date: Thu Sep 27 07:16:52 2018 -0700 Add Vanilla JS example to kbn-i18n README (#23556) Rename Node.js to Vanilla JS and give example of internationalizing a string constant. commit 73f955db1927e72567290ab1b325a4c52f35acc6 Author: Aleh Zasypkin <[email protected]> Date: Thu Sep 27 11:54:09 2018 +0200 Upgrade eslint/tslint/prettier plugin versions. (#23470) commit ecbcbb612a9e5b5b466148b5b0a64b8200ed8dd4 Author: Maryia Lapata <[email protected]> Date: Thu Sep 27 11:53:16 2018 +0300 Translate metric_vis (#23187) * Translate metric_vis * Close span tag * Remove space code * Update ids * Translations for color mode list commit 7c23374f2c7acb972610af450c777cab8a8f2084 Author: Robert Monfera <[email protected]> Date: Wed Sep 26 23:33:49 2018 +0200 Feat: ad-hoc grouping (#23249) * Feat: ad-hoc grouping * Feat: deleting ad-hoc group constituents * Chore: deleted the former removeElement action * Feat: make group snap to guides * Feat: make group snap to guides 2 * Feat: make group snap to guides 3 commit 78e212e32d007112e5e7eedf034595a046f57a84 Author: liza-mae <[email protected]> Date: Wed Sep 26 14:35:16 2018 -0600 Add argument passing to jenkins cloud job (#23538) commit 265a32417b175d3cb7fbbfcadd00297c72900a64 Author: CJ Cenizal <[email protected]> Date: Wed Sep 26 12:54:44 2018 -0700 Add SearchError for surfacing courier search errors. (#23382) commit 1df298131cb83c9adaaca0557e5cf8e37182c04a Author: Ryan Keairns <[email protected]> Date: Wed Sep 26 13:36:57 2018 -0500 fix home page width for IE11 (#23491) commit 5bf68d67aac161ff18ff2eb99db0cd05cb5fc753 Author: Brandon Kobel <[email protected]> Date: Wed Sep 26 08:29:48 2018 -0700 Saved Object Namespaces (#23378) * Use an instance of SavedObjectsSerializer for migrations and the repository * Fixing spelling of serialization * Making the serializer conditionally include and prepend id with ns * Adding repository tests for the namespaces * Implementing find * Modifying the SOCs to pass the options with the namespace * Centralizing omitting the namespace when using serializer.rawToSavedObject * Passing the schema through to the SavedObjectRepositoryProvider * Changing the schema to work with undefined ui exports schemas * Adding schema tests * Making the complimentary serialization test use the namespace * Fixing uiExports * Fixing some tests * Fixing included fields for the find * Fixing include field tests, they're checking length also... * Updating Repository test after adding namespace to always included fields * Renaming UIExportsSavedObjectTypeSchema to SavedObjectsSchemaDefinition * Completing rename... forgot to save usages * Fixing issue with the serialization.isRawSavedObject and the trailing : commit 3c806b86b4901ad27fea6dbd8e29d70785058801 Author: Josh Dover <[email protected]> Date: Wed Sep 26 10:12:24 2018 -0500 Setup yarn in current shell for jenkins test report script (#23531) * Setup yarn in current shell [skip ci] * Use setup.sh commit 832b896877e4ec23cda3fea995cca4df5739f36a Author: Tim Roes <[email protected]> Date: Wed Sep 26 16:53:23 2018 +0200 Remove last mentions of spy panels (#23527) commit 143e7d8ee5d6631c8a1eb64715dbb8699160ff8e Author: Catherine Liu <[email protected]> Date: Tue Sep 25 14:46:39 2018 -0700 Removed tr hover style in datatable (#23305) commit 3c6b382b061b299f03d121a6f07c8a5265d6a7f5 Author: Aleh Zasypkin <[email protected]> Date: Tue Sep 25 18:58:37 2018 +0200 Correctly pass `timestamp` from the core to the legacy Kibana. Do not try to stop legacy Hapi server if it does not exist. (#23436) commit 9b6d0b1f30ab2c93443578d315690204b894d9d2 Author: Luke Elmers <[email protected]> Date: Tue Sep 25 10:47:48 2018 -0600 Remove deprecation notice from ascending sort for terms (#23421) commit fceddf8610ace354e12fe0f830e8ec5f9486698a Author: Matt Bargar <[email protected]> Date: Tue Sep 25 12:45:22 2018 -0400 Mention license change for autocomplete commit 4773798114bb35ec8ea4d13c08cce59aca20b39f Author: Melissa Alvarez <[email protected]> Date: Tue Sep 25 16:44:42 2018 +0100 Add context to job picker for accessibility (#23483) commit 8a4088fd80bce6ca868df455478a542adb215c17 Author: James Gowdy <[email protected]> Date: Tue Sep 25 16:41:06 2018 +0100 [ML] Fixing duplicate influencers when cloning a job via a wizard (#23484) commit 90d0d1caa72f97e423bb0fd92c4974872d98037d Author: James Gowdy <[email protected]> Date: Tue Sep 25 15:19:07 2018 +0100 [ML] Fixing issue when editing script fields in advanced job creator (#23475) commit 110c987c89e0c9c6546e363de805cb9ce056bdde Author: Maryia Lapata <[email protected]> Date: Tue Sep 25 14:35:31 2018 +0300 Update versions of @babel/parser and @babel/types (#23268) Update versions of @babel/parser, @babel/types, eslint, babel-eslint commit b4e023086a96917f5f1494d0f1078fa58fcc78e7 Author: Ryan Keairns <[email protected]> Date: Mon Sep 24 15:48:37 2018 -0500 removes unused less styles for ace editor (#23425) commit e477ca3fdf07355cb27ad9852d47309f492bdee8 Author: liza-mae <[email protected]> Date: Mon Sep 24 13:57:23 2018 -0600 Cleanup from PR 22608, remove esInstallDir (#23450) commit d8b4d4b0603653e08001a3a1c2641cd275fa49c2 Author: James Gowdy <[email protected]> Date: Mon Sep 24 20:11:35 2018 +0100 [ML] Fixing missing field when cloning a distinct count job (#23439) commit 1b763d8ba6f968782eeea8397f2cf9730643bbc4 Author: Nathan Reese <[email protected]> Date: Mon Sep 24 12:00:39 2018 -0600 display hits and total hits for courier inspector requests (#23434) * display hits and total hits for courier inspector requests * update Hits help text to explain difference between total hits * fix functional test commit 5d9d7242e51b6dcc84e7fada7340437597c12df3 Author: liza-mae <[email protected]> Date: Mon Sep 24 11:39:09 2018 -0600 Add option to functional test server to run elasticsearch from instal… (#22608) * Add option to functional test server to run elasticsearch from install dir * Fix variable * Fix server CLI test * Updates to include install path in esFrom command line option * Fix snapshot * Update args/cli tests * Keep default snapshot in args/help commit f2bb7dbf9d90ec30ce373edac63506fb349334d9 Author: Chris Roberson <[email protected]> Date: Mon Sep 24 12:59:18 2018 -0400 [Monitoring] APM Monitoring UI (#22975) * Merge in boilerplate branch * Manually copy over the specific metrics and UIs * Add api integration tests * Fix tests * Remove unused metrics * Update snapshot * Fix tests * Remove types agg * Use ApmClusterMetric * provide description for apm-server monitoring metrics (#23331) * Vis LESS to SASS (cont.) (#23199) * Tweak migrations integraiton tests to have a stable sort (#23265) * Fix: plugin api route with security enabled (#23334) Closes https://github.com/elastic/kibana/issues/23266 This is more of a quick fix than the final solution. The issue was that Canvas tries to check the plugins API without checking to see if the user it logged in. As a result, instead of the plugins response, it gets the HTML from the login page and that causes an error to be thrown when attempting to parse the results. For now, this PR just disables the auth requirement on the Canvas plugin API endpoint. * [migrations/tests] sort results before assertion (#23347) There have been several failures in this test, seemingly caused by a lack of sorting in the results. It makes sense that since both migrations are run simultaneously that sometimes one would succeed and sometimes another would, so I've just sorted the results before checking. ![image](https://user-images.githubusercontent.com/1329312/45791153-44e9cc80-bc3d-11e8-88c4-760d4c7b35bd.png) cc: @chrisdavies * [ML] Moves custom URL editor Add button and form to top of flyout (#23326) * [ML] Moves custom URL editor Add button and form to top of flyout * [ML] Edits to custom URL editor class name * Graph LESS to SASS (#23348) * Developer documentation for integrating with the telemetry service (#23295) * Developer documentation for integrating with the telemetry service * open…

Crude and incomplete impl of Space-Aware Saved Objects Client

f208d3c

legrego added WIP Work in progress Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! labels May 7, 2018

legrego commented May 7, 2018

View reviewed changes

legrego requested a review from kobelb May 7, 2018 14:09

kobelb reviewed May 7, 2018

View reviewed changes

Code review updates

7538bf3

legrego force-pushed the space-aware-saved-objects branch from 0c5cefd to 7538bf3 Compare May 8, 2018 18:52

Missed one - move extraBodyProperties to the top

bb3e511

kobelb reviewed May 8, 2018

View reviewed changes

Remove documentFilter from bulkGet

0e2e4e8

legrego mentioned this pull request May 9, 2018

RBAC - Phase 2 (Spaces) #18948

Closed

33 tasks

rhoboat mentioned this pull request May 10, 2018

[Spaces] Space Avatar with selector in main Kibana menu #18609

Merged

1 task

rhoboat closed this May 10, 2018

rhoboat reopened this May 10, 2018

legrego added 10 commits May 21, 2018 13:26

Merge branch 'spaces-phase-1' into space-aware-saved-objects

a6287cc

Make config document id independent of Kibana version

7db0a4a

Merge branch 'spaces-phase-1' into space-aware-saved-objects

192d9c2

Merge branch 'spaces-phase-1' into space-aware-saved-objects

1c4afd8

cleanup and fixes following initial rbac phase 1 merge

d573457

remove unused/migrated files

d99cec7

remove unused code

7e2d1e3

Merge branch 'spaces-phase-1' into space-aware-saved-objects

862752b

partial updates for space aware saved objects and tests

a63128c

working get & find functional tests

858eff0

address PR feedback

8b1ff6b

elastic deleted a comment from elasticmachine Jul 23, 2018

legrego added 2 commits July 23, 2018 13:54

Consolidate space url parsing on the server

299efb8

fix base path parsing

3d0f6c9

legrego mentioned this pull request Jul 23, 2018

[Spaces] Replace Space Selector modal with a less intrusive popover #19497

Merged

1 task

elastic deleted a comment from legrego Jul 25, 2018

kobelb reviewed Jul 25, 2018

View reviewed changes

legrego added 5 commits July 25, 2018 16:05

refactor spaces saved objects client tests

1747a74

fix bug routing to the correct space

78a54d1

address PR feedback

2149f87

remove stray console.log

85b315f

augment find tests to include global types

9c69568

elastic deleted a comment from elasticmachine Jul 26, 2018

kobelb approved these changes Jul 26, 2018

View reviewed changes

legrego merged commit 252e407 into elastic:spaces-phase-1 Jul 26, 2018

legrego mentioned this pull request Jul 30, 2018

Spaces Phase 1 #21408

Merged


		import { getSpacesQueryFilters } from './query_filters';

		test('returns no parameters when no types are provided', () => {

[Spaces] Space-Aware Saved Objects #18862

[Spaces] Space-Aware Saved Objects #18862

Conversation

legrego commented May 7, 2018 • edited Loading

Terminology:

Saved Objects Client

get

bulk_get

create

bulk_create

update

delete

find

* Belonging to the current space

1. Is the object's type space-aware?

2. Check the object's spaceId

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

legrego commented May 7, 2018

Get/MGet:

Find:

Create/Update/Delete:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kobelb left a comment

Choose a reason for hiding this comment

rhoboat commented May 10, 2018 • edited Loading

create

get + bulkGet

update + delete

rhoboat commented May 10, 2018

legrego commented May 11, 2018

elasticmachine commented Jul 23, 2018

💔 Build Failed

legrego commented Jul 25, 2018

kobelb commented Jul 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as resolved.

legrego commented Jul 26, 2018

elasticmachine commented Jul 26, 2018

💔 Build Failed

kobelb left a comment

Choose a reason for hiding this comment

legrego commented May 7, 2018 •

edited

Loading

`get`

`bulk_get`

`create`

`bulk_create`

`update`

`delete`

`find`

2. Check the object's `spaceId`

rhoboat commented May 10, 2018 •

edited

Loading