[Security Solution] Add tags by OpenAPI bundler #189621
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maximpn
added
release_note:skip
|
Hi @xcrzx, You may notice this PR removes existing tags only for some certain API domains. It feels like we still could use tagging inside API domains later on for example in code generation. Specs for Lists API, Exceptions API and Endpoint Exceptions API just had the same value and don't bring too much value that's why I removed them. WDYT? |
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
I initially added tags to group API endpoints based on their domains, so in standard OpenAPI viewers, they wouldn't appear as one big list. Since the bundler and merger now handle producing a correctly categorized schema, I'm okay with removing the old tags from the sources. |
xcrzx
reviewed
Aug 1, 2024
packages/kbn-securitysolution-exceptions-common/scripts/openapi_bundle.js
Outdated
Show resolved
Hide resolved
packages/kbn-securitysolution-exceptions-common/scripts/openapi_bundle.js
Outdated
Show resolved
Hide resolved
maximpn
force-pushed
the
assign-tags-by-oas-bundler
branch
from
d5772d1 to
6125ec2
Compare
maximpn
force-pushed
the
assign-tags-by-oas-bundler
branch
3 times, most recently
from
b51e691 to
30c7336
Compare
maximpn
changed the title
xcrzx
approved these changes
Aug 2, 2024
maximpn
force-pushed
the
assign-tags-by-oas-bundler
branch
from
11032a0 to
593981e
Compare
patrykkopycinski
approved these changes
Aug 2, 2024
Files by Code Ownerelastic/security-defend-workflows
elastic/security-detection-engine
elastic/security-detection-rule-management
elastic/security-generative-ai
elastic/security-solution
|
e40pud
approved these changes
Aug 2, 2024
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]
History
To update your PR or re-run it, just comment with: cc @maximpn |
lcawl
pushed a commit
to lcawl/kibana
that referenced
this pull request
Aug 20, 2024
**Resolves:** elastic#183375 This PR implements functionality assigning a provided tag to OpenAPI `Operation object`s in the result bundle. Specified tag is also added as the only root level OpenAPI tag. This approach allows to produce domain bundles having a single tag assigned. At the next step domain bundles are merged together into single Kibana bundle where tags will allow to properly display grouping at Bump.sh (API reference documentation platform). Bump.sh (our new API reference documentation platform) uses OpenAPI tags for grouping API endpoints. It supports only one tag per endpoint. This PR facilitates preparation of Kibana OpenAPI bundle to be uploaded to Bump.sh by implementing functionality assigning a provided tag to OpenAPI `Operation object`s in the result domain bundles. It's implemented by providing an optional configuration option `assignTag` whose format is OpenAPI Tag Object. When `assignTag` isn't specified the bundler merges existing tags. Consider the following bundling configuration ```js const { bundle } = require('@kbn/openapi-bundler'); bundle({ // ... options: { assignTag: { name: 'Some Domain API tag name', description: 'Some Domain API description', externalDocs: { url: 'https://some-external-documentation-url', description: 'External documentation description', } }, }); ``` and source OpenAPI specs **spec1.schema.yaml** ```yaml openapi: 3.0.3 info: title: Spec1 version: '2023-10-31' paths: /api/some_api: get: tags: ['Some local tag'] responses: 200: content: 'application/json': schema: type: string ``` **spec2.schema.yaml** ```yaml openapi: 3.0.3 info: title: Spec2 version: '2023-10-31' paths: /api/some_api: post: tags: ['Some global tag'] responses: 200: content: 'application/json': schema: type: string tags: - name: Some global tag ``` **spec2.schema.yaml** ```yaml openapi: 3.0.3 info: title: Spec3 version: '2023-10-31' paths: /api/another_api: get: responses: 200: content: 'application/json': schema: type: string ``` After bundling above OpenAPI specs with the provided bundling script we'll get the following **domain-bundle.schema.yaml** ```yaml openapi: 3.0.3 info: title: Bundled document version: '2023-10-31' paths: /api/some_api: get: tags: ['Some Domain API tag name'] responses: 200: content: 'application/json': schema: type: string post: tags: ['Some Domain API tag name'] responses: 200: content: 'application/json': schema: type: string /api/another_api: get: tags: ['Some Domain API tag name'] responses: 200: content: 'application/json': schema: type: string tags: - name: Some Domain API tag name description: Some Domain API description externalDocs: url: 'https://some-external-documentation-url' description: External documentation description ```
lcawl
added a commit
that referenced
this pull request
Aug 20, 2024
…189262) (#190541) # Backport This will backport the bundler aspects of the following commits from `main` to `8.15`: - [[HTTP/OAS] Merge OpenAPI specs by using `kbn-openapi-bundler` (#189262)](#189262) - #189621 - #189482 - #189348 - #189472 - #188110 - #188812 <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-08-13T10:45:35Z","message":"[HTTP/OAS] Merge OpenAPI specs by using `kbn-openapi-bundler` (#189262)\n\n**Addresses:** https://github.com/elastic/kibana/issues/186356\r\n**Relates to:** https://github.com/elastic/kibana/issues/184428\r\n\r\n## Summary\r\n\r\nThis PR adds a merging JS script based on the utility implemented in #186356. Resulted OpenAPI bundle as committed in `oas_docs/output/kibana.serverless.bundled.yaml`.\r\n\r\n## Details\r\n\r\nhttps://github.com//pull/188110 implements and exposes `merge` utility design to merge source OpenAPI specs without processing. It's has only a programmatic API. To merge OpenAPI specs it's required to add a JS script like below\r\n\r\n```js\r\nconst { merge } = require('@kbn/openapi-bundler');\r\n\r\n(async () => {\r\n await merge({\r\n sourceGlobs: [/* a list of source globs goes here */],\r\n outputFilePath: 'path/to/the/output/file.yaml',\r\n });\r\n})();\r\n```\r\n\r\nThe JS script added in this PR includes source OpenAPI specs presented in `oas_docs/makefile` plus Security Solution OpenAPI specs based on https://github.com/elastic/kibana/issues/184428.\r\n\r\n**To run** the script use the following command from Kibana root folder\r\n\r\n```bash\r\nnode ./oas_docs/scripts/merge_serverless_oas.js \r\n```\r\n\r\n## Known linting issues with Security Solution OpenAPI specs\r\n\r\nRunning Spectral OpenAPI linter on the result bundle shows a number of errors caused by `no-$ref-siblings` rule. This caused by the current code generator implementation which requires `default` property to be set next to `$ref` though it's not correct for OpenAPI `3.0.3` while it's allowed in `3.1`. It seems that Bump.sh handles such cases properly though by properly showing a default value.\r\n\r\nWe need to analyze the problem and decide if/when we should fix it.\r\n\r\nThe rest of warnings look fixable and will be addressed in the next stage after setting up linter rules.\r\n\r\n## Next steps\r\n\r\nSince `@kbn/openapi-bundler` package is tailored specifically for Kibana it should replace Redocly currently used to merge OpenAPI specs. It also means `oas_docs/makefile` should be superseded by JS script(s) using `merge` utility form `@kbn/openapi-bundler` package.\r\n\r\n`@kbn/openapi-bundler` SHOULD NOT replace OpenAPI linters since it doesn't perform thorough linting. It's good if we continue adopting `spectral-cli` for linting purposes.","sha":"7a2e7bef9644eaf04fd820d16b8ee137b3c00f2b","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Core","release_note:skip","backport:skip","docs","Team: SecuritySolution","Feature:OAS","v8.16.0"],"number":189262,"url":"https://github.com/elastic/kibana/pull/189262","mergeCommit":{"message":"[HTTP/OAS] Merge OpenAPI specs by using `kbn-openapi-bundler` (#189262)\n\n**Addresses:** https://github.com/elastic/kibana/issues/186356\r\n**Relates to:** https://github.com/elastic/kibana/issues/184428\r\n\r\n## Summary\r\n\r\nThis PR adds a merging JS script based on the utility implemented in #186356. Resulted OpenAPI bundle as committed in `oas_docs/output/kibana.serverless.bundled.yaml`.\r\n\r\n## Details\r\n\r\nhttps://github.com//pull/188110 implements and exposes `merge` utility design to merge source OpenAPI specs without processing. It's has only a programmatic API. To merge OpenAPI specs it's required to add a JS script like below\r\n\r\n```js\r\nconst { merge } = require('@kbn/openapi-bundler');\r\n\r\n(async () => {\r\n await merge({\r\n sourceGlobs: [/* a list of source globs goes here */],\r\n outputFilePath: 'path/to/the/output/file.yaml',\r\n });\r\n})();\r\n```\r\n\r\nThe JS script added in this PR includes source OpenAPI specs presented in `oas_docs/makefile` plus Security Solution OpenAPI specs based on https://github.com/elastic/kibana/issues/184428.\r\n\r\n**To run** the script use the following command from Kibana root folder\r\n\r\n```bash\r\nnode ./oas_docs/scripts/merge_serverless_oas.js \r\n```\r\n\r\n## Known linting issues with Security Solution OpenAPI specs\r\n\r\nRunning Spectral OpenAPI linter on the result bundle shows a number of errors caused by `no-$ref-siblings` rule. This caused by the current code generator implementation which requires `default` property to be set next to `$ref` though it's not correct for OpenAPI `3.0.3` while it's allowed in `3.1`. It seems that Bump.sh handles such cases properly though by properly showing a default value.\r\n\r\nWe need to analyze the problem and decide if/when we should fix it.\r\n\r\nThe rest of warnings look fixable and will be addressed in the next stage after setting up linter rules.\r\n\r\n## Next steps\r\n\r\nSince `@kbn/openapi-bundler` package is tailored specifically for Kibana it should replace Redocly currently used to merge OpenAPI specs. It also means `oas_docs/makefile` should be superseded by JS script(s) using `merge` utility form `@kbn/openapi-bundler` package.\r\n\r\n`@kbn/openapi-bundler` SHOULD NOT replace OpenAPI linters since it doesn't perform thorough linting. It's good if we continue adopting `spectral-cli` for linting purposes.","sha":"7a2e7bef9644eaf04fd820d16b8ee137b3c00f2b"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/189262","number":189262,"mergeCommit":{"message":"[HTTP/OAS] Merge OpenAPI specs by using `kbn-openapi-bundler` (#189262)\n\n**Addresses:** https://github.com/elastic/kibana/issues/186356\r\n**Relates to:** https://github.com/elastic/kibana/issues/184428\r\n\r\n## Summary\r\n\r\nThis PR adds a merging JS script based on the utility implemented in #186356. Resulted OpenAPI bundle as committed in `oas_docs/output/kibana.serverless.bundled.yaml`.\r\n\r\n## Details\r\n\r\nhttps://github.com//pull/188110 implements and exposes `merge` utility design to merge source OpenAPI specs without processing. It's has only a programmatic API. To merge OpenAPI specs it's required to add a JS script like below\r\n\r\n```js\r\nconst { merge } = require('@kbn/openapi-bundler');\r\n\r\n(async () => {\r\n await merge({\r\n sourceGlobs: [/* a list of source globs goes here */],\r\n outputFilePath: 'path/to/the/output/file.yaml',\r\n });\r\n})();\r\n```\r\n\r\nThe JS script added in this PR includes source OpenAPI specs presented in `oas_docs/makefile` plus Security Solution OpenAPI specs based on https://github.com/elastic/kibana/issues/184428.\r\n\r\n**To run** the script use the following command from Kibana root folder\r\n\r\n```bash\r\nnode ./oas_docs/scripts/merge_serverless_oas.js \r\n```\r\n\r\n## Known linting issues with Security Solution OpenAPI specs\r\n\r\nRunning Spectral OpenAPI linter on the result bundle shows a number of errors caused by `no-$ref-siblings` rule. This caused by the current code generator implementation which requires `default` property to be set next to `$ref` though it's not correct for OpenAPI `3.0.3` while it's allowed in `3.1`. It seems that Bump.sh handles such cases properly though by properly showing a default value.\r\n\r\nWe need to analyze the problem and decide if/when we should fix it.\r\n\r\nThe rest of warnings look fixable and will be addressed in the next stage after setting up linter rules.\r\n\r\n## Next steps\r\n\r\nSince `@kbn/openapi-bundler` package is tailored specifically for Kibana it should replace Redocly currently used to merge OpenAPI specs. It also means `oas_docs/makefile` should be superseded by JS script(s) using `merge` utility form `@kbn/openapi-bundler` package.\r\n\r\n`@kbn/openapi-bundler` SHOULD NOT replace OpenAPI linters since it doesn't perform thorough linting. It's good if we continue adopting `spectral-cli` for linting purposes.","sha":"7a2e7bef9644eaf04fd820d16b8ee137b3c00f2b"}}]}] BACKPORT--> --------- Co-authored-by: Maxim Palenov <[email protected]>
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Sep 25, 2024
…PI (elastic#193019) **Addresses:** elastic#183375 ## Summary This PR adds a missing domain tag to Endpoint Exceptions API. The rest API endpoints got their tags in elastic#189621. (cherry picked from commit 09374ad) # Conflicts: # packages/kbn-securitysolution-endpoint-exceptions-common/docs/openapi/ess/security_solution_endpoint_exceptions_api_2023_10_31.bundled.schema.yaml # packages/kbn-securitysolution-endpoint-exceptions-common/docs/openapi/serverless/security_solution_endpoint_exceptions_api_2023_10_31.bundled.schema.yaml
maximpn
referenced
this pull request
Sep 25, 2024
…ions API (#193019) (#193984) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Add a missing domain tag to Endpoint Exceptions API (#193019)](#193019) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-19T11:23:11Z","message":"[Security Solution] Add a missing domain tag to Endpoint Exceptions API (#193019)\n\n**Addresses:** https://github.com/elastic/kibana/issues/183375\r\n\r\n## Summary\r\n\r\nThis PR adds a missing domain tag to Endpoint Exceptions API. The rest API endpoints got their tags in https://github.com/elastic/kibana/pull/189621.","sha":"09374ad2ce5ba96b61a1d300264d2465f34e2326","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","docs","Team:Detections and Resp","Team: SecuritySolution","backport:prev-minor","Team:Detection Engine","v8.16.0"],"number":193019,"url":"https://github.com/elastic/kibana/pull/193019","mergeCommit":{"message":"[Security Solution] Add a missing domain tag to Endpoint Exceptions API (#193019)\n\n**Addresses:** https://github.com/elastic/kibana/issues/183375\r\n\r\n## Summary\r\n\r\nThis PR adds a missing domain tag to Endpoint Exceptions API. The rest API endpoints got their tags in https://github.com/elastic/kibana/pull/189621.","sha":"09374ad2ce5ba96b61a1d300264d2465f34e2326"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193019","number":193019,"mergeCommit":{"message":"[Security Solution] Add a missing domain tag to Endpoint Exceptions API (#193019)\n\n**Addresses:** https://github.com/elastic/kibana/issues/183375\r\n\r\n## Summary\r\n\r\nThis PR adds a missing domain tag to Endpoint Exceptions API. The rest API endpoints got their tags in https://github.com/elastic/kibana/pull/189621.","sha":"09374ad2ce5ba96b61a1d300264d2465f34e2326"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <[email protected]>
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Nov 12, 2024
…PI (elastic#193019) **Addresses:** elastic#183375 This PR adds a missing domain tag to Endpoint Exceptions API. The rest API endpoints got their tags in elastic#189621. (cherry picked from commit 09374ad)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves: #183375
Summary
This PR implements functionality assigning a provided tag to OpenAPI
Operation objects in the result bundle. Specified tag is also added as the only root level OpenAPI tag. This approach allows to produce domain bundles having a single tag assigned. At the next step domain bundles are merged together into single Kibana bundle where tags will allow to properly display grouping at Bump.sh (API reference documentation platform).Details
Bump.sh (our new API reference documentation platform) uses OpenAPI tags for grouping API endpoints. It supports only one tag per endpoint.
This PR facilitates preparation of Kibana OpenAPI bundle to be uploaded to Bump.sh by implementing functionality assigning a provided tag to OpenAPI
Operation objects in the result domain bundles. It's implemented by providing an optional configuration optionassignTagwhose format is OpenAPI Tag Object. WhenassignTagisn't specified the bundler merges existing tags.Example
Consider the following bundling configuration
and source OpenAPI specs
spec1.schema.yaml
spec2.schema.yaml
spec2.schema.yaml
After bundling above OpenAPI specs with the provided bundling script we'll get the following
domain-bundle.schema.yaml