[POC] SO Metadata events streams

[sebelga]

This PR is a POC for the content management "metadata events".

Goal

We want to add a new sorting capability to the saved object list view tables to help our users quickly find the content they are looking for. We want to be able to sort by “Total views” (ASC and DESC) and also “Total views in the last 30 / 15 / 7 days” (ASC and DESC).

In this PR I demo how we could add views counters (for last "7_days", "14_days", ...) and be able to sort the Dashboard list view table with those counters.

How to test

• Add the sample data (this will add 3 dashboards)
• Navigate to the dashboard list view. There should be a column with the view count + a drop down in the table header to change the granularity of the counts
• Click a dashboard to view it and go back to the dashboard list. Its counter should now be 1

In order to test the granularity we need to manually add an event in the past. In a terminal send the following curl command (note: set the correct url of your local dev Kibana)

curl -X "POST" "http://localhost:5601/<unique>/api/user_content/event/viewed:kibana" \
     -H 'kbn-xsrf: kibana' \
     -H 'Content-Type: application/json; charset=utf-8' \
     -u 'elastic:changeme' \
     -d $'{
  "@timestamp": "2022-05-15T10:00:00.000Z", # change this date if needed
  "soId": "1bcfb990-d859-11ec-8a6b-b125ca46a567", # change if needed
  "soType": "dashboard"
}'

You should now have an event in the past and be able to test the other granularity ("30 days", "45 days"...)

Issue related: #9202

[sebelga]

I am not sure why we have duplicate mappings definition with the server. This would probably need to be refactored to have a single mappings declaration

[kibana-ci]

💔 Build Failed

• Buildkite Build
• Commit: 982d5ea
• Interpreting CI Failures
• Storybooks Preview
• Webpack Bundle Analyzer

Failed CI Steps

• Jest Tests #3
• Jest Tests #4
• Jest Tests #6
• Jest Integration Tests #1
• Jest Integration Tests #2
• FTR Configs #5
• FTR Configs #5
• FTR Configs #26
• FTR Configs #26
• Checks
• Check Types

Test Failures

• [job] [logs] Jest Tests #3 / after fetch initialFilter
• [job] [logs] Jest Tests #3 / after fetch renders all table rows
• [job] [logs] Jest Tests #3 / after fetch renders call to action when no dashboards exist
• [job] [logs] Jest Tests #3 / after fetch renders call to action with continue when no dashboards exist but one is in progress
• [job] [logs] Jest Tests #3 / after fetch renders warning when listingLimit is exceeded
• [job] [logs] Jest Tests #3 / after fetch showWriteControls
• [job] [logs] Jest Tests #3 / after fetch When given a title that matches multiple dashboards, filter on the title
• [job] [logs] FTR Configs #5 / analytics analytics service analytics service: public side should see both events enqueued and sent to the shipper
• [job] [logs] FTR Configs #5 / analytics analytics service analytics service: public side should see both events enqueued and sent to the shipper
• [job] [logs] FTR Configs #26 / apis saved_objects _bulk_create should return 200 with individual responses
• [job] [logs] FTR Configs #26 / apis saved_objects _bulk_create should return 200 with individual responses
• [job] [logs] Jest Integration Tests #1 / Fleet preconfiguration reset Preconfigured cloud policy Adding APM to a preconfigured agent policy after first setup Create a .fleet-policies document with the APM package policy
• [job] [logs] Jest Integration Tests #1 / Fleet preconfiguration reset Preconfigured cloud policy Adding APM to a preconfigured agent policy after first setup Create correct package policies
• [job] [logs] Jest Integration Tests #1 / Fleet preconfiguration reset Preconfigured cloud policy Adding APM to a preconfigured agent policy after first setup Works and preconfigure correctly agent policies
• [job] [logs] Jest Integration Tests #1 / Fleet preconfiguration reset Preconfigured cloud policy Adding package policy id to a preconfigured package policy after first setup Create correct package policies and use the name of package policies instead of id
• [job] [logs] Jest Integration Tests #1 / Fleet preconfiguration reset Preconfigured cloud policy Adding package policy id to a preconfigured package policy after first setup Works and preconfigure correctly agent policies
• [job] [logs] Jest Integration Tests #1 / Fleet setup preconfiguration with multiple instances Kibana preconfiguration setup sets up Fleet correctly when multiple Kibana instaces are started in serial
• [job] [logs] Jest Integration Tests #1 / Fleet setup preconfiguration with multiple instances Kibana preconfiguration setup sets up Fleet correctly when multiple Kibana instances are started at the same time
• [job] [logs] Jest Tests #6 / SavedObjectsClient #create makes HTTP call with ID
• [job] [logs] Jest Tests #6 / SavedObjectsClient #create makes HTTP call without ID
• [job] [logs] Jest Tests #6 / SavedObjectsService #setup() #addClientWrapper registers the wrapper to the clientProvider
• [job] [logs] Jest Tests #6 / SavedObjectsService #setup() #getTypeRegistry returns the internal type registry of the service
• [job] [logs] Jest Tests #6 / SavedObjectsService #setup() #registerType registers the type to the internal typeRegistry
• [job] [logs] Jest Tests #6 / SavedObjectsService #setup() #setClientFactoryProvider registers the factory to the clientProvider
• [job] [logs] Jest Tests #6 / SavedObjectsService #setup() #setClientFactoryProvider throws if a factory is already registered
• [job] [logs] Jest Tests #6 / SavedObjectsService #setup() calls registerRoutes with the correct kibanaVersion
• [job] [logs] Jest Tests #6 / SavedObjectsService #setup() register the deprecation provider
• [job] [logs] Jest Tests #6 / SavedObjectsService #setup() registers the deprecation provider with the correct kibanaVersion
• [job] [logs] Jest Tests #6 / SavedObjectsService #start() #createInternalRepository creates a repository using the admin user
• [job] [logs] Jest Tests #6 / SavedObjectsService #start() #createScopedRepository creates a repository including hidden types when specified
• [job] [logs] Jest Tests #6 / SavedObjectsService #start() #createScopedRepository creates a repository scoped to the user
• [job] [logs] Jest Tests #6 / SavedObjectsService #start() #getTypeRegistry returns the internal type registry of the service
• [job] [logs] Jest Tests #6 / SavedObjectsService #start() calls KibanaMigrator with correct version
• [job] [logs] Jest Tests #6 / SavedObjectsService #start() resolves with KibanaMigrator after waiting for migrations to complete
• [job] [logs] Jest Tests #6 / SavedObjectsService #start() skips KibanaMigrator migrations when migrations.skip=true
• [job] [logs] Jest Tests #6 / SavedObjectsService #start() skips KibanaMigrator migrations when pluginsInitialized=false
• [job] [logs] Jest Tests #6 / SavedObjectsService #start() throws when calling setup APIs once started
• [job] [logs] Jest Tests #6 / SavedObjectsService #start() waits for all es nodes to be compatible before running migrations
• [job] [logs] Jest Tests #4 / TableListView render list view
• [job] [logs] Jest Integration Tests #2 / trace execution context does not affect "x-opaque-id" set by user
• [job] [logs] Jest Integration Tests #2 / trace execution context does not break on non-ASCII characters within execution context
• [job] [logs] Jest Integration Tests #2 / trace execution context execution context is the same for all the lifecycle events
• [job] [logs] Jest Integration Tests #2 / trace execution context execution context is uniq for concurrent requests
• [job] [logs] Jest Integration Tests #2 / trace execution context execution context is uniq for concurrent requests when "x-opaque-id" provided
• [job] [logs] Jest Integration Tests #2 / trace execution context execution context is uniq for sequential requests
• [job] [logs] Jest Integration Tests #2 / trace execution context parses the parent context if present
• [job] [logs] Jest Integration Tests #2 / trace execution context passes "x-opaque-id" if no execution context is registered
• [job] [logs] Jest Integration Tests #2 / trace execution context propagates context to Elasticsearch scoped client
• [job] [logs] Jest Integration Tests #2 / trace execution context propagates context to Elasticsearch unscoped client
• [job] [logs] Jest Integration Tests #2 / trace execution context sets execution context for a sync request handler
• [job] [logs] Jest Integration Tests #2 / trace execution context sets execution context for an async request handler
• [job] [logs] Jest Integration Tests #2 / trace execution context supports UTF-8 characters
• [job] [logs] Jest Integration Tests #2 / trace execution context withContext sets execution context for a nested function
• [job] [logs] Jest Integration Tests #2 / trace x-opaque-id can be overridden during Elasticsearch client call
• [job] [logs] Jest Integration Tests #2 / trace x-opaque-id ExecutionContext Service is disabled does not pass context if ExecutionContext Service is disabled
• [job] [logs] Jest Integration Tests #2 / trace x-opaque-id ExecutionContext Service is disabled passed to Elasticsearch scoped client calls even if ExecutionContext Service is disabled
• [job] [logs] Jest Integration Tests #2 / trace x-opaque-id generated and attached to Elasticsearch scoped client calls if not specifed
• [job] [logs] Jest Integration Tests #2 / trace x-opaque-id generated and attached to Elasticsearch unscoped client calls if not specifed
• [job] [logs] Jest Integration Tests #2 / trace x-opaque-id passed to Elasticsearch scoped client calls

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
userContent	-	10	+10

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
core	956	977	+21
kibanaReact	213	223	+10
metadataEventsStreams	-	24	+24
savedObjects	151	152	+1
userContent	-	31	+31
total			+87

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id	before	after	diff
userContent	-	4	+4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
dashboard	425.4KB	425.6KB	+120.0B
kibanaReact	209.6KB	209.6KB	+2.0B
total			+122.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
core	33	37	+4
userContent	-	4	+4
total			+8

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
core	301.1KB	301.3KB	+151.0B
dashboard	65.9KB	66.1KB	+218.0B
kibanaReact	61.9KB	63.5KB	+1.6KB
savedObjects	28.9KB	29.1KB	+160.0B
userContent	-	3.6KB	+3.6KB
total			+5.8KB

Saved Objects .kibana field count

Every field in each saved object type adds overhead to Elasticsearch. Kibana needs to keep the total field count below Elasticsearch's default limit of 1000 fields. Only specify field mappings for the fields you wish to search on or query. See https://www.elastic.co/guide/en/kibana/master/development-plugin-saved-objects.html#_mappings

id	before	after	diff
dashboard	22	29	+7

Unknown metric groups

API count

id	before	after	diff
core	2519	2549	+30
kibanaReact	250	260	+10
metadataEventsStreams	-	24	+24
savedObjects	192	193	+1
userContent	-	36	+36
total			+101

ESLint disabled in files

id	before	after	diff
apm	14	13	-1
securitySolution	71	70	-1
synthetics	7	6	-1
total			-3

ESLint disabled line counts

id	before	after	diff
apm	84	81	-3
enterpriseSearch	9	7	-2
metadataEventsStreams	-	1	+1
synthetics	62	56	-6
userContent	-	6	+6
total			-4

Total ESLint disabled count

id	before	after	diff
apm	98	94	-4
enterpriseSearch	9	7	-2
metadataEventsStreams	-	1	+1
securitySolution	514	513	-1
synthetics	69	62	-7
userContent	-	6	+6
total			-7

History

• 💔 Build #48987 failed aae9289
• 💔 Build #48054 failed 7cc782f
• 💔 Build #47882 failed 637287ff68e2280b1ba59ec5558cf6d12a24c84d
• 💔 Build #47869 failed b2361a353e83a34c9706c0b1e70478880f1f0ea8

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream