Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set spaces and roles CRUD APIs to public #193534

Merged
merged 32 commits into from
Oct 3, 2024

Conversation

jeramysoucy
Copy link
Contributor

@jeramysoucy jeramysoucy commented Sep 20, 2024

Closes #192153

Summary

This PR explicitly sets the spaces and roles CRUD operation HTTP API endpoints to public in both stateful and serverless offerings, and additionally, switches to the versioned router to register these endpoints.

Prior to this PR, the access level was not explicitly set, thus any endpoints registered in serverless were by default internal. CRUD operations for spaces and roles are being set to public to support the rollout of custom roles in serverless, which coincides with enabling multiple spaces.

Note

  • Currently, roles APIs are only available in serverless via a feature flag (xpack.security.roleManagementEnabled)
  • Spaces APIs are already registered in serverless, however, the maximum number of spaces is by default 1, rendering create and delete operations unusable. By overriding xpack.spaces.maxSpaces to a number greater than 1 (stateful default is 1000), it will effectively enable use of the spaces CRUD operations in serverless.

Tests

  • x-pack/test_serverless/api_integration/test_suites/common/management/multiple_spaces_enabled.ts
  • x-pack/test_serverless/api_integration/test_suites/common/management/spaces.ts
  • x-pack/test_serverless/api_integration/test_suites/common/platform_security/authorization.ts
  • x-pack/test_serverless/api_integration/test_suites/common/platform_security/roles_routes_feature_flag.ts
  • Unit tests for each endpoint (to account for versioned router)
  • Flaky Test Runner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7002

Manual Testing

  1. Start ES & Kibana in serverless mode with config options to enable role management and multiple spaces

Elasticsearch:

xpack.security.authc.native_roles.enabled: true

KIbana:

 xpack.security.roleManagementEnabled: true
 xpack.spaces.maxSpaces: 100
  1. Issue each CRUD HTTP API without including the internal origin header ('x-elastic-internal-origin') and verify you do not receive a 400 with the message "method [get|post|put|delete] exists but is not available with the current configuration"
  2. Repeat steps 1 & 2 from the current head of main and verify that you DO receive a 400 with the message "method [get|post|put|delete] exists but is not available with the current configuration"

Regression testing - ensure that interfaces which leverage spaces and roles APIs are functioning properly

  • Spaces management
  • Space navigation
  • Roles management

@jeramysoucy jeramysoucy added Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! Feature:Security/Spaces Platform Security - Spaces feature release_note:skip Skip the PR/issue when compiling release notes Feature:Security/Authorization Platform Security - Authorization backport:skip This commit does not require backporting Project:Serverless Work as part of the Serverless project for its initial release v8.16.0 labels Sep 20, 2024
@jeramysoucy jeramysoucy changed the title Sets spaces and roles CRUD API to public Set spaces and roles CRUD API to public Sep 23, 2024
@jeramysoucy jeramysoucy changed the title Set spaces and roles CRUD API to public Set spaces and roles CRUD APIs to public Sep 23, 2024
@jeramysoucy jeramysoucy marked this pull request as ready for review September 24, 2024 07:13
@jeramysoucy jeramysoucy requested review from a team as code owners September 24, 2024 07:13
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-security (Team:Security)

@jeramysoucy jeramysoucy marked this pull request as draft September 24, 2024 07:31
@kibanamachine
Copy link
Contributor

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7002

[✅] x-pack/test_serverless/api_integration/test_suites/search/config.feature_flags.ts: 50/50 tests passed.
[✅] x-pack/test_serverless/api_integration/test_suites/observability/config.feature_flags.ts: 50/50 tests passed.
[✅] x-pack/test_serverless/api_integration/test_suites/security/config.feature_flags.ts: 50/50 tests passed.

see run history

@@ -5,7 +5,7 @@ set -euo pipefail
source .buildkite/scripts/common/util.sh

echo --- Capture OAS snapshot
cmd="node scripts/capture_oas_snapshot --include-path /api/status --include-path /api/alerting/rule/ --include-path /api/alerting/rules --include-path /api/actions"
cmd="node scripts/capture_oas_snapshot --include-path /api/status --include-path /api/alerting/rule/ --include-path /api/alerting/rules --include-path /api/actions --include-path /api/security/role --include-path /api/spaces"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @elastic/kibana-core for awareness

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #30 / cases security and spaces enabled: basic Common migrations migrations 7.11.1 -> latest stack version "before all" hook for "adds rule info to only alert comments for 7.12"

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/core-http-browser 37 36 -1
spaces 71 73 +2
total +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
security 591.0KB 591.1KB +38.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
security 67.6KB 67.7KB +162.0B
spaces 30.8KB 31.0KB +201.0B
total +363.0B
Unknown metric groups

API count

id before after diff
spaces 266 269 +3

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@jeramysoucy jeramysoucy merged commit 26f2928 into elastic:main Oct 3, 2024
21 checks passed
tiansivive pushed a commit to tiansivive/kibana that referenced this pull request Oct 7, 2024
Closes elastic#192153

## Summary

This PR sets the spaces and roles CRUD operation HTTP API endpoints to
public in both stateful and serverless offerings, and additionally,
switches to the versioned router to register these endpoints.

Prior to this PR, the access level was not explicitly set, thus any
endpoints registered in serverless were by default internal. CRUD
operations for spaces and roles are being set to public to support the
rollout of custom roles in serverless, which coincides with enabling
multiple spaces.

### Note
- Currently, roles APIs are only available in serverless via a feature
flag (`xpack.security.roleManagementEnabled`)
- Spaces APIs are already registered in serverless, however, the maximum
number of spaces is by default 1, rendering create and delete operations
unusable. By overriding `xpack.spaces.maxSpaces` to a number greater
than 1 (stateful default is 1000), it will effectively enable use of the
spaces CRUD operations in serverless.

## Tests
-
x-pack/test_serverless/api_integration/test_suites/common/management/multiple_spaces_enabled.ts
-
x-pack/test_serverless/api_integration/test_suites/common/management/spaces.ts
-
x-pack/test_serverless/api_integration/test_suites/common/platform_security/authorization.ts
-
x-pack/test_serverless/api_integration/test_suites/common/platform_security/roles_routes_feature_flag.ts
- Unit tests for each endpoint (to account for versioned router)
- Flaky Test Runner:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7002

## Manual Testing
1. Start ES & Kibana in serverless mode with config options to enable
role management and multiple spaces

Elasticsearch:
```
xpack.security.authc.native_roles.enabled: true
```
 KIbana:
```
 xpack.security.roleManagementEnabled: true
 xpack.spaces.maxSpaces: 100
```
3. Issue each CRUD HTTP API without including the internal origin header
('x-elastic-internal-origin') and verify you do not receive a 400 with
the message "method [get|post|put|delete] exists but is not available
with the current configuration"
4. Repeat steps 1 & 2 from the current head of main and verify that you
DO receive a 400 with the message "method [get|post|put|delete] exists
but is not available with the current configuration"

Regression testing - ensure that interfaces which leverage spaces and
roles APIs are functioning properly
- Spaces management
- Space navigation
- Roles management

---------

Co-authored-by: kibanamachine <[email protected]>
@jen-huang
Copy link
Contributor

@jeramysoucy Should this be backported to 8.x? I see both backport:skip and v8.16.0 labels here 😄 (the backport tool marked this PR as a potential missing backport on one of my PRs)

@jeramysoucy
Copy link
Contributor Author

@jen-huang Sorry about that. This doesn't need to be backported. I removed the v8.16 tag.

@jeramysoucy
Copy link
Contributor Author

@jen-huang Sorry to change this now, but after a good point from a team member, we will backport to prev-minor.

@jeramysoucy jeramysoucy added backport:prev-major Backport to (8.x, 8.17, 8.16) the previous major branch and other branches in development and removed backport:skip This commit does not require backporting labels Oct 15, 2024
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.15, 8.x

https://github.com/elastic/kibana/actions/runs/11341061398

@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.15, 8.x

https://github.com/elastic/kibana/actions/runs/11341061349

@jeramysoucy jeramysoucy added backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) and removed backport:prev-major Backport to (8.x, 8.17, 8.16) the previous major branch and other branches in development labels Oct 15, 2024
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11341066309

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.x Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 193534

Questions ?

Please refer to the Backport tool documentation

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
8.15 Backport failed because of merge conflicts
8.x Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 193534

Questions ?

Please refer to the Backport tool documentation

@jeramysoucy
Copy link
Contributor Author

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

jeramysoucy added a commit to jeramysoucy/kibana that referenced this pull request Oct 15, 2024
Closes elastic#192153

## Summary

This PR sets the spaces and roles CRUD operation HTTP API endpoints to
public in both stateful and serverless offerings, and additionally,
switches to the versioned router to register these endpoints.

Prior to this PR, the access level was not explicitly set, thus any
endpoints registered in serverless were by default internal. CRUD
operations for spaces and roles are being set to public to support the
rollout of custom roles in serverless, which coincides with enabling
multiple spaces.

### Note
- Currently, roles APIs are only available in serverless via a feature
flag (`xpack.security.roleManagementEnabled`)
- Spaces APIs are already registered in serverless, however, the maximum
number of spaces is by default 1, rendering create and delete operations
unusable. By overriding `xpack.spaces.maxSpaces` to a number greater
than 1 (stateful default is 1000), it will effectively enable use of the
spaces CRUD operations in serverless.

## Tests
-
x-pack/test_serverless/api_integration/test_suites/common/management/multiple_spaces_enabled.ts
-
x-pack/test_serverless/api_integration/test_suites/common/management/spaces.ts
-
x-pack/test_serverless/api_integration/test_suites/common/platform_security/authorization.ts
-
x-pack/test_serverless/api_integration/test_suites/common/platform_security/roles_routes_feature_flag.ts
- Unit tests for each endpoint (to account for versioned router)
- Flaky Test Runner:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7002

## Manual Testing
1. Start ES & Kibana in serverless mode with config options to enable
role management and multiple spaces

Elasticsearch:
```
xpack.security.authc.native_roles.enabled: true
```
 KIbana:
```
 xpack.security.roleManagementEnabled: true
 xpack.spaces.maxSpaces: 100
```
3. Issue each CRUD HTTP API without including the internal origin header
('x-elastic-internal-origin') and verify you do not receive a 400 with
the message "method [get|post|put|delete] exists but is not available
with the current configuration"
4. Repeat steps 1 & 2 from the current head of main and verify that you
DO receive a 400 with the message "method [get|post|put|delete] exists
but is not available with the current configuration"

Regression testing - ensure that interfaces which leverage spaces and
roles APIs are functioning properly
- Spaces management
- Space navigation
- Roles management

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit 26f2928)

# Conflicts:
#	oas_docs/output/kibana.serverless.yaml
#	oas_docs/output/kibana.yaml
jeramysoucy added a commit that referenced this pull request Oct 16, 2024
# Backport

This will backport the following commits from `main` to `8.x`:
- [Set spaces and roles CRUD APIs to public
(#193534)](#193534)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Jeramy
Soucy","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-03T14:28:54Z","message":"Set
spaces and roles CRUD APIs to public (#193534)\n\nCloses
#192153\r\n\r\n## Summary\r\n\r\nThis PR sets the spaces and roles CRUD
operation HTTP API endpoints to\r\npublic in both stateful and
serverless offerings, and additionally,\r\nswitches to the versioned
router to register these endpoints.\r\n\r\nPrior to this PR, the access
level was not explicitly set, thus any\r\nendpoints registered in
serverless were by default internal. CRUD\r\noperations for spaces and
roles are being set to public to support the\r\nrollout of custom roles
in serverless, which coincides with enabling\r\nmultiple
spaces.\r\n\r\n### Note\r\n- Currently, roles APIs are only available in
serverless via a feature\r\nflag
(`xpack.security.roleManagementEnabled`)\r\n- Spaces APIs are already
registered in serverless, however, the maximum\r\nnumber of spaces is by
default 1, rendering create and delete operations\r\nunusable. By
overriding `xpack.spaces.maxSpaces` to a number greater\r\nthan 1
(stateful default is 1000), it will effectively enable use of
the\r\nspaces CRUD operations in serverless.\r\n\r\n##
Tests\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/management/multiple_spaces_enabled.ts\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/management/spaces.ts\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/platform_security/authorization.ts\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/platform_security/roles_routes_feature_flag.ts\r\n-
Unit tests for each endpoint (to account for versioned router)\r\n-
Flaky Test
Runner:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7002\r\n\r\n##
Manual Testing\r\n1. Start ES & Kibana in serverless mode with config
options to enable\r\nrole management and multiple
spaces\r\n\r\nElasticsearch:\r\n```\r\nxpack.security.authc.native_roles.enabled:
true\r\n```\r\n KIbana:\r\n```\r\n xpack.security.roleManagementEnabled:
true\r\n xpack.spaces.maxSpaces: 100\r\n```\r\n3. Issue each CRUD HTTP
API without including the internal origin
header\r\n('x-elastic-internal-origin') and verify you do not receive a
400 with\r\nthe message \"method [get|post|put|delete] exists but is not
available\r\nwith the current configuration\"\r\n4. Repeat steps 1 & 2
from the current head of main and verify that you\r\nDO receive a 400
with the message \"method [get|post|put|delete] exists\r\nbut is not
available with the current configuration\"\r\n\r\nRegression testing -
ensure that interfaces which leverage spaces and\r\nroles APIs are
functioning properly\r\n- Spaces management\r\n- Space navigation\r\n-
Roles management\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>","sha":"26f2928b0887c9fda4403c0ce3fcc332b7c0e69a","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Security","Feature:Security/Spaces","release_note:skip","Feature:Security/Authorization","v9.0.0","backport:prev-minor","Project:Serverless"],"number":193534,"url":"https://github.com/elastic/kibana/pull/193534","mergeCommit":{"message":"Set
spaces and roles CRUD APIs to public (#193534)\n\nCloses
#192153\r\n\r\n## Summary\r\n\r\nThis PR sets the spaces and roles CRUD
operation HTTP API endpoints to\r\npublic in both stateful and
serverless offerings, and additionally,\r\nswitches to the versioned
router to register these endpoints.\r\n\r\nPrior to this PR, the access
level was not explicitly set, thus any\r\nendpoints registered in
serverless were by default internal. CRUD\r\noperations for spaces and
roles are being set to public to support the\r\nrollout of custom roles
in serverless, which coincides with enabling\r\nmultiple
spaces.\r\n\r\n### Note\r\n- Currently, roles APIs are only available in
serverless via a feature\r\nflag
(`xpack.security.roleManagementEnabled`)\r\n- Spaces APIs are already
registered in serverless, however, the maximum\r\nnumber of spaces is by
default 1, rendering create and delete operations\r\nunusable. By
overriding `xpack.spaces.maxSpaces` to a number greater\r\nthan 1
(stateful default is 1000), it will effectively enable use of
the\r\nspaces CRUD operations in serverless.\r\n\r\n##
Tests\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/management/multiple_spaces_enabled.ts\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/management/spaces.ts\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/platform_security/authorization.ts\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/platform_security/roles_routes_feature_flag.ts\r\n-
Unit tests for each endpoint (to account for versioned router)\r\n-
Flaky Test
Runner:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7002\r\n\r\n##
Manual Testing\r\n1. Start ES & Kibana in serverless mode with config
options to enable\r\nrole management and multiple
spaces\r\n\r\nElasticsearch:\r\n```\r\nxpack.security.authc.native_roles.enabled:
true\r\n```\r\n KIbana:\r\n```\r\n xpack.security.roleManagementEnabled:
true\r\n xpack.spaces.maxSpaces: 100\r\n```\r\n3. Issue each CRUD HTTP
API without including the internal origin
header\r\n('x-elastic-internal-origin') and verify you do not receive a
400 with\r\nthe message \"method [get|post|put|delete] exists but is not
available\r\nwith the current configuration\"\r\n4. Repeat steps 1 & 2
from the current head of main and verify that you\r\nDO receive a 400
with the message \"method [get|post|put|delete] exists\r\nbut is not
available with the current configuration\"\r\n\r\nRegression testing -
ensure that interfaces which leverage spaces and\r\nroles APIs are
functioning properly\r\n- Spaces management\r\n- Space navigation\r\n-
Roles management\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>","sha":"26f2928b0887c9fda4403c0ce3fcc332b7c0e69a"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193534","number":193534,"mergeCommit":{"message":"Set
spaces and roles CRUD APIs to public (#193534)\n\nCloses
#192153\r\n\r\n## Summary\r\n\r\nThis PR sets the spaces and roles CRUD
operation HTTP API endpoints to\r\npublic in both stateful and
serverless offerings, and additionally,\r\nswitches to the versioned
router to register these endpoints.\r\n\r\nPrior to this PR, the access
level was not explicitly set, thus any\r\nendpoints registered in
serverless were by default internal. CRUD\r\noperations for spaces and
roles are being set to public to support the\r\nrollout of custom roles
in serverless, which coincides with enabling\r\nmultiple
spaces.\r\n\r\n### Note\r\n- Currently, roles APIs are only available in
serverless via a feature\r\nflag
(`xpack.security.roleManagementEnabled`)\r\n- Spaces APIs are already
registered in serverless, however, the maximum\r\nnumber of spaces is by
default 1, rendering create and delete operations\r\nunusable. By
overriding `xpack.spaces.maxSpaces` to a number greater\r\nthan 1
(stateful default is 1000), it will effectively enable use of
the\r\nspaces CRUD operations in serverless.\r\n\r\n##
Tests\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/management/multiple_spaces_enabled.ts\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/management/spaces.ts\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/platform_security/authorization.ts\r\n-\r\nx-pack/test_serverless/api_integration/test_suites/common/platform_security/roles_routes_feature_flag.ts\r\n-
Unit tests for each endpoint (to account for versioned router)\r\n-
Flaky Test
Runner:\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7002\r\n\r\n##
Manual Testing\r\n1. Start ES & Kibana in serverless mode with config
options to enable\r\nrole management and multiple
spaces\r\n\r\nElasticsearch:\r\n```\r\nxpack.security.authc.native_roles.enabled:
true\r\n```\r\n KIbana:\r\n```\r\n xpack.security.roleManagementEnabled:
true\r\n xpack.spaces.maxSpaces: 100\r\n```\r\n3. Issue each CRUD HTTP
API without including the internal origin
header\r\n('x-elastic-internal-origin') and verify you do not receive a
400 with\r\nthe message \"method [get|post|put|delete] exists but is not
available\r\nwith the current configuration\"\r\n4. Repeat steps 1 & 2
from the current head of main and verify that you\r\nDO receive a 400
with the message \"method [get|post|put|delete] exists\r\nbut is not
available with the current configuration\"\r\n\r\nRegression testing -
ensure that interfaces which leverage spaces and\r\nroles APIs are
functioning properly\r\n- Spaces management\r\n- Space navigation\r\n-
Roles management\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine
<[email protected]>","sha":"26f2928b0887c9fda4403c0ce3fcc332b7c0e69a"}}]}]
BACKPORT-->

---------

Co-authored-by: kibanamachine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Feature:Security/Authorization Platform Security - Authorization Feature:Security/Spaces Platform Security - Spaces feature Project:Serverless Work as part of the Serverless project for its initial release release_note:skip Skip the PR/issue when compiling release notes Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! v8.16.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Set spaces CRUD endpoints to public in serverless
10 participants