Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Serverless Security Functional Tests.x-pack/test_serverless/functional/test_suites/common/index_management/index_templates·ts - serverless common UI Index Management Index Templates "before all" hook for "renders the index templates tab" #167130

Closed
kibanamachine opened this issue Sep 25, 2023 · 3 comments · Fixed by #166936
Closed

Failing test: Serverless Security Functional Tests.x-pack/test_serverless/functional/test_suites/common/index_management/index_templates·ts - serverless common UI Index Management Index Templates "before all" hook for "renders the index templates tab" #167130

kibanamachine opened this issue Sep 25, 2023 · 3 comments · Fixed by #166936
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Sep 25, 2023 •
edited
Loading

A test failed on a tracked branch

Error: timed out waiting for logout button visible -- last error: Error: retry.try timeout: Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="userMenuButton"])
Wait timed out after 10021ms
    at /var/lib/buildkite-agent/builds/kb-n2-4-spot-9e476d268ceadb55/elastic/kibana-on-merge/kibana/node_modules/selenium-webdriver/lib/webdriver.js:929:17
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at onFailure (retry_for_success.ts:17:9)
    at retryForSuccess (retry_for_success.ts:59:13)
    at RetryService.try (retry.ts:31:12)
    at Proxy.clickByCssSelector (find.ts:417:5)
    at TestSubjects.click (test_subjects.ts:164:5)
    at user_menu.js:47:9
    at runAttempt (retry_for_success.ts:29:15)
    at retryForSuccess (retry_for_success.ts:68:21)
    at RetryService.try (retry.ts:31:12)
    at UserMenu._ensureMenuOpen (user_menu.js:46:7)
    at UserMenu.logoutLinkExists (user_menu.js:28:7)
    at security_page.ts:267:19
    at runAttempt (retry_for_success.ts:29:15)
    at retryForSuccess (retry_for_success.ts:68:21)
    at retryForTruthy (retry_for_truthy.ts:27:3)
    at RetryService.waitFor (retry.ts:59:5)
    at SecurityPageObject.login (security_page.ts:265:5)
    at Object.login (svl_common_page.ts:18:14)
    at Context.<anonymous> (index_templates.ts:20:7)
    at Object.apply (wrap_function.js:73:16)
    at onFailure (retry_for_success.ts:17:9)
    at retryForSuccess (retry_for_success.ts:59:13)
    at RetryService.try (retry.ts:31:12)
    at UserMenu._ensureMenuOpen (user_menu.js:46:7)
    at UserMenu.logoutLinkExists (user_menu.js:28:7)
    at security_page.ts:267:19
    at runAttempt (retry_for_success.ts:29:15)
    at retryForSuccess (retry_for_success.ts:68:21)
    at retryForTruthy (retry_for_truthy.ts:27:3)
    at RetryService.waitFor (retry.ts:59:5)
    at SecurityPageObject.login (security_page.ts:265:5)
    at Object.login (svl_common_page.ts:18:14)
    at Context.<anonymous> (index_templates.ts:20:7)
    at Object.apply (wrap_function.js:73:16)
    at onFailure (retry_for_truthy.ts:39:13)
    at retryForSuccess (retry_for_success.ts:59:13)
    at retryForTruthy (retry_for_truthy.ts:27:3)
    at RetryService.waitFor (retry.ts:59:5)
    at SecurityPageObject.login (security_page.ts:265:5)
    at Object.login (svl_common_page.ts:18:14)
    at Context.<anonymous> (index_templates.ts:20:7)
    at Object.apply (wrap_function.js:73:16)

First failure: CI Build - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Sep 25, 2023
@botelastic botelastic bot added the needs-team Issues missing a team label label Sep 25, 2023
@kibanamachine kibanamachine added the Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more label Sep 25, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/platform-deployment-management (Team:Deployment Management)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Sep 25, 2023
@dmlemeshko dmlemeshko mentioned this issue Sep 25, 2023
Merged
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

delanni pushed a commit that referenced this issue Sep 28, 2023
@dmlemeshko @kibanamachine
[ftr] pass password for UI login + improve login/logout steps with pr…
2eddd54 
…oper wait logic (#166936)

## Summary

Hopefully

closes #167104
closes #167130
closes #167100 
closes #167013
closes #166964

Fixing a few issues with login/logout:

1. Failed to login in "before" hook
<img width="1336" alt="Screenshot 2023-09-25 at 12 37 45"
src="https://github.com/elastic/kibana/assets/10977896/e3b2830e-7b0d-4467-9b90-261b385bf71e">

My theory is that we are loading `/login` route too soon while log out
was not completed yet.
When we navigate to `https://localhost:5620/logout` there are multiple
url re-directions with final page being Cloud login form. This PR makes
sure we wait for this form to be displayed + 2500 ms extra to avoid
"immediate" /login navigation

2. Failed login on MKI:
Updating login via UI for serverless to pass password valid for
deployment: currently FTR uses `changeme` for both Kibana CI & MKI.

3. ES activate user profile call returning 500
We saw some login failures that are preceded with the following logs:
```
[00:03:27]           │ debg Find.clickByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:03:27]           │ debg Find.findByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:03:27]           │ debg Find.waitForDeletedByCssSelector('.kibanaWelcomeLogo') with timeout=10000
[00:03:27]           │ proc [kibana] [2023-09-19T07:08:26.126+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic)
[00:03:27]           │ info [o.e.x.s.s.SecurityIndexManager] [ftr] security index does not exist, creating [.security-profile-8] with alias [.security-profile]
[00:03:27]           │ proc [kibana] [2023-09-19T07:08:26.140+00:00][ERROR][plugins.security.user-profile] Failed to activate user profile: {"error":{"root_cause":[{"type":"validation_exception","reason":"Validation Failed: 1: this action would add [1] shards, but this cluster currently has [27]/[27] maximum normal shards open;"}],"type":"validation_exception","reason":"Validation Failed: 1: this action would add [1] shards, but this cluster currently has [27]/[27] maximum normal shards open;"},"status":400}.
[00:03:27]           │ proc [kibana] [2023-09-19T07:08:26.140+00:00][ERROR][http] 500 Server Error
[00:03:27]           │ warn browser[SEVERE] http://localhost:5620/internal/security/login - Failed to load resource: the server responded with a status of 500 (Internal Server Error)
```

User activation happens during `POST internal/security/login` call to
Kibana server. ~~The only improvement that we can do from FTR
perspective is to call this end-point via API to makes sure user is
activated and only after proceed with UI login.~~
While working on issue #4 and talking to @jeramysoucy I believe retrying
login via UI will work here as well. We are checking if we are still on
login page (similar to incorrect password login), waiting 2500 ms and
pressing login button again.

4. Failed to login with Kibana reporting UNEXPECTED_SESSION_ERROR and
been re-directed to Cloud login page
```
proc [kibana] [2023-09-25T11:35:12.794+00:00][INFO ][plugins.security.authentication] Authentication attempt failed: UNEXPECTED_SESSION_ERROR
```

Temporary solution is to retry login from scratch (navigation to Kibana
login page & re-login )



Flaky-test-runner for functional obtl tests 50x
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3215


This PR is not fixing random 401 response when user navigates to some
apps with direct url

---------

Co-authored-by: Kibana Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more
Projects
None yet
Milestone
No milestone
2 participants
@elasticmachine @kibanamachine