[7.x] [Ingest Manager] Unify install* under installPackage (#82916) #82998
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Summary
* Add `installPackage` with `installSource` param, to provide a single interface the `install*` functions.
```diff
- const res = await installPackageFromRegistry({
+ const res = await installPackage({
+ installSource: 'registry',
```
and
```diff
- const res = await installPackageByUpload({
+ const res = await installPackage({
+ installSource: 'upload',
```
* Push some repeated work (`install`, `removable`) from `install*` into `_installPackage`. Which also simplifies its interface.
### installPackage
For now `installPackage` checks the `installSource` and calls the same `install*` functions to prevent any change in behavior but there's still a lot of overlap between `installPackageFromRegistry` & `installPackageByUpload`. I think we can bring them together into `installPackage` using the same branching on `installSource`.
### local checks with curl
<details><summary>curl request/responses for happy path:</summary>
```
## zip:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"response":[{"id":"apache-Logs-Apache-Dashboard-ecs","type":"dashboard"},{"id":"apache-Metrics-Apache-HTTPD-server-status-ecs","type":"dashboard"},{"id":"Apache-HTTPD-CPU-ecs","type":"visualization"},{"id":"Apache-HTTPD-Hostname-list-ecs","type":"visualization"},{"id":"Apache-HTTPD-Load1-slash-5-slash-15-ecs","type":"visualization"},{"id":"Apache-HTTPD-Scoreboard-ecs","type":"visualization"},{"id":"Apache-HTTPD-Total-accesses-and-kbytes-ecs","type":"visualization"},{"id":"Apache-HTTPD-Uptime-ecs","type":"visualization"},{"id":"Apache-HTTPD-Workers-ecs","type":"visualization"},{"id":"Apache-access-unique-IPs-map-ecs","type":"visualization"},{"id":"Apache-browsers-ecs","type":"visualization"},{"id":"Apache-error-logs-over-time-ecs","type":"visualization"},{"id":"Apache-operating-systems-ecs","type":"visualization"},{"id":"Apache-response-codes-of-top-URLs-ecs","type":"visualization"},{"id":"Apache-response-codes-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-ecs","type":"search"},{"id":"Apache-access-logs-ecs","type":"search"},{"id":"Apache-errors-log-ecs","type":"search"}]}
## Uploaded packages can be deleted as expected:
curl -X DELETE -u elastic:changeme http://localhost:5601/api/fleet/epm/packages/apache-0.1.4 -H 'kbn-xsrf: xxx'
{"response":[{"id":"apache-Logs-Apache-Dashboard-ecs","type":"dashboard"},{"id":"apache-Metrics-Apache-HTTPD-server-status-ecs","type":"dashboard"},{"id":"Apache-HTTPD-CPU-ecs","type":"visualization"},{"id":"Apache-HTTPD-Hostname-list-ecs","type":"visualization"},{"id":"Apache-HTTPD-Load1-slash-5-slash-15-ecs","type":"visualization"},{"id":"Apache-HTTPD-Scoreboard-ecs","type":"visualization"},{"id":"Apache-HTTPD-Total-accesses-and-kbytes-ecs","type":"visualization"},{"id":"Apache-HTTPD-Uptime-ecs","type":"visualization"},{"id":"Apache-HTTPD-Workers-ecs","type":"visualization"},{"id":"Apache-access-unique-IPs-map-ecs","type":"visualization"},{"id":"Apache-browsers-ecs","type":"visualization"},{"id":"Apache-error-logs-over-time-ecs","type":"visualization"},{"id":"Apache-operating-systems-ecs","type":"visualization"},{"id":"Apache-response-codes-of-top-URLs-ecs","type":"visualization"},{"id":"Apache-response-codes-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-ecs","type":"search"},{"id":"Apache-access-logs-ecs","type":"search"},{"id":"Apache-errors-log-ecs","type":"search"}]}
## Now upload
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.tar.gz -H 'kbn-xsrf: xyz' -H 'Content-Type: application/gzip'
{"response":[{"id":"apache-Metrics-Apache-HTTPD-server-status-ecs","type":"dashboard"},{"id":"apache-Logs-Apache-Dashboard-ecs","type":"dashboard"},{"id":"Apache-access-unique-IPs-map-ecs","type":"visualization"},{"id":"Apache-HTTPD-CPU-ecs","type":"visualization"},{"id":"Apache-HTTPD-Load1-slash-5-slash-15-ecs","type":"visualization"},{"id":"Apache-response-codes-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-Workers-ecs","type":"visualization"},{"id":"Apache-HTTPD-Hostname-list-ecs","type":"visualization"},{"id":"Apache-error-logs-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-Scoreboard-ecs","type":"visualization"},{"id":"Apache-HTTPD-Uptime-ecs","type":"visualization"},{"id":"Apache-operating-systems-ecs","type":"visualization"},{"id":"Apache-HTTPD-Total-accesses-and-kbytes-ecs","type":"visualization"},{"id":"Apache-browsers-ecs","type":"visualization"},{"id":"Apache-response-codes-of-top-URLs-ecs","type":"visualization"},{"id":"Apache-access-logs-ecs","type":"search"},{"id":"Apache-errors-log-ecs","type":"search"},{"id":"Apache-HTTPD-ecs","type":"search"},{"id":"logs-apache.error-0.1.4","type":"ingest_pipeline"},{"id":"logs-apache.access-0.1.4","type":"ingest_pipeline"},{"id":"logs-apache.error","type":"index_template"},{"id":"metrics-apache.status","type":"index_template"},{"id":"logs-apache.access","type":"index_template"}]}
```
</details>
<details><summary>curl request/responses for archive errors:</summary>
```
## Wrong content type:
### tar.gz with application/zip:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.tar.gz -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Error during extraction of package: Error: end of central directory record signature not found. Assumed content type was application/zip, check if this matches the archive type."}
### zip with application/gzip:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/gzip'
{"statusCode":400,"error":"Bad Request","message":"Archive seems empty. Assumed content type was application/gzip, check if this matches the archive type."}
## Invalid packages
### Two top-level directories:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_two_toplevels_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Package contains more than one top-level directory."}
### No manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_no_manifest_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Package must contain a top-level manifest.yml file."}
### Invalid YAML in manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_manifest_invalid_yaml_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Could not parse top-level package manifest: YAMLException: bad indentation of a mapping entry at line 2, column 7:\n name: apache\n ^."}
### Mandatory field missing in manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_manifest_missing_field_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Invalid top-level package manifest: one or more fields missing of name, version, description, type, categories, format_version"}
### Top-level directory doesn't match name and version from manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_toplevel_mismatch_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Name thisIsATypo and version 0.1.4 do not match top-level directory apache-0.1.4"}
```
</details>
#### TS type check examples on `installPackage`
<details><summary>screenshots</summary>
<img width="379" alt="Screen Shot 2020-11-08 at 4 00 14 PM" src="https://user-images.githubusercontent.com/57655/98484251-1d1e9f80-21dc-11eb-93f8-601036b45355.png">
<img width="890" alt="Screen Shot 2020-11-08 at 4 00 21 PM" src="https://user-images.githubusercontent.com/57655/98484252-1db73600-21dc-11eb-88d1-5faa498f94fc.png">
<img width="396" alt="Screen Shot 2020-11-08 at 4 01 06 PM" src="https://user-images.githubusercontent.com/57655/98484253-1db73600-21dc-11eb-8e2a-10a5762f4a95.png">
<img width="441" alt="Screen Shot 2020-11-08 at 4 01 25 PM" src="https://user-images.githubusercontent.com/57655/98484254-1db73600-21dc-11eb-9d9a-c1620dcad11e.png">
<img width="879" alt="Screen Shot 2020-11-08 at 4 02 54 PM" src="https://user-images.githubusercontent.com/57655/98484255-1db73600-21dc-11eb-8f36-7da3e9256feb.png">
</details>
### Checklist
- [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
jfsiii
force-pushed
the
backport/7.x/pr-82916
branch
from
a5cd375 to
4201f2e
Compare
💚 Build SucceededMetrics [docs]
History
To update your PR or re-run it, just comment with: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backports the following commits to 7.x: