[Archive Migration] x-pack-banners/multispace

[wayneseymour]

Summary

I've the before() fn loading the new kbn archive
containing one config, for the default space.
Then, the fn creates a new space and changes the ui settings of this new space,
w/o using an archive.

Helps with: #102552

Bugs Filed:
#140307
#140309

[elasticmachine]

Pinging @elastic/kibana-qa (Team:QA)

[LeeDr]

@wayneseymour I don't think that kbn_archive really does anything for this test. It only contains a config saved object and Kibana would have already created a config object when it starts (and will ignore the one you load).

[LeeDr]

I'm trying it locally without the kbn_archive

[LeeDr]

Interesting that this test won't pass for me locally. It fails here because the save button doesn't appear on the page after changing the banner text in advanced settings;

await PageObjects.settings.setAdvancedSettingsTextArea(
        'banners:textContent',
        'default space banner text'
      );

[LeeDr]

I'm also getting tons of this error in the server log while running the test. I don't see any reason for this test to change the default route for the space. It doesn't seem to have anything to do with the test.

 proc [kibana] [2022-07-07T14:09:55.579-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:09:55.579-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:09:55.847-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:10:00.326-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 info [o.e.x.s.a.r.TransportPutRoleAction] [ftr] updated role [system_indices_superuser]
 info [o.e.x.s.a.u.TransportPutUserAction] [ftr] updated user [system_indices_superuser]
 info [o.e.x.s.a.u.TransportPutUserAction] [ftr] added user [test_user]
 proc [kibana] [2022-07-07T14:11:38.164-05:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic)
 proc [kibana] [2022-07-07T14:11:38.837-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:38.889-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:40.261-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:40.262-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:40.262-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:40.263-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:50.436-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
 proc [kibana] [2022-07-07T14:11:50.458-05:00][WARN ][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..

[LeeDr]

I'm not sure how the test passes in CI, but locally it doesn't seem to work as the tests describe. The difference is mainly on the last test 'displays the global banner on the login page' where instead of displaying the global banner, it actually displays the default space banner.

Steps to reproduce:

1. Start Elasticsearch and Kibana with node scripts/functional_tests_server.js --config x-pack/test/banners_functional/config.ts
   (the config file sets
   '--xpack.banners.placement=top',
   '--xpack.banners.textContent="global banner text"
   )
2. log in to Kibana as elastic user. I see the global banner text as expected. I'm not really why I see the space selector since there's only the default space at this time.
3. click on the default space
4. click on the D and Manage spaces and create a new space space1
5. go to Advanced Settings (still in the default space)
6. change banners:textContent to default space banner, save changes, reload page, see the new banner text
7. switch to space1, see global banner text
8. logout and log back in
9. on the space selector page, the banner is default space banner not global banner text

Either this is a bug, or the test is incorrect?

[LeeDr]

It appears that these tests are not running in buildkite. If you go to the passing buildkite job and look at

Pick Test Group Run Order. .buildkite/scripts/steps/test/pick_test_group_run_order.sh

and then at
ftr_run_order.json

and then look for the test pathx-path/test/banners_functional it's not there.

[LeeDr]

buildkite doesn't seem to be running these tests. There's some ftr config somewhere that it's probably missing from.

It's in the section here labeled # Configs that exist but weren't running in CI when this file was introduced

https://github.com/elastic/kibana/blob/main/.buildkite/ftr_configs.yml#L52

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[wayneseymour]

buildkite doesn't seem to be running these tests. There's some ftr config somewhere that it's probably missing from.

It's in the section here labeled # Configs that exist but weren't running in CI when this file was introduced

https://github.com/elastic/kibana/blob/main/.buildkite/ftr_configs.yml#L52

Yeah, now I see it is defined within the disabled stanza of ftr_configs.yml

[LeeDr]

Looks like @pgayvallet added these tests in #94449 a little over a year ago. Things have change since then quite a bit from Jenkins to Buildkite and how we run tests. But I think there still would have been some config file change to add the banners_functional tests to CI that I don't see in that PR. So I think these never ran in CI.
The settings_page method setAdvancedSettingsTextArea was also added in that PR and doesn't appear to be used anywhere else. I had problems with it locally but I think I have a fix for it (sending the return key to it after entering the text).
And then there's the actual functionality in the product doesn't seem to match the test descriptions. It could be a bug.

[pgayvallet]

Just putting a safety block (given I previously approved) so we don't forget about that:

[wayneseymour]

This should be reverted once #140688 get merged

I could have sworn this test was a duplicate. Am I crazy? lol

[LeeDr]

Canyou change back from global_banner_text to global banner text as was originally in the test?

[pgayvallet]

Can you change back from global_banner_text to global banner text as was originally in the test?

Nope. See #140688 (comment)

[LeeDr]

Can you change back from global_banner_text to global banner text as was originally in the test?

Nope. See #140688 (comment)

Couldn't this PR just add the double-quotes here and include in this PR? https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/bin/scripts/kibana#L29

Or we want to do that in a separate PR?

[pgayvallet]

Couldn't this PR just add the double-quotes here and include in this PR?

We can, yes (even if I'm not sure to see the issue using a string without whitespaces in our FTR tests)

[LeeDr]

LGTM

[pgayvallet]

LGTM when the duplicate test suite has been removed from .buildkite/ftr_configs.yml

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 896249c

Metrics [docs]

✅ unchanged

History

• 💚 Build #75539 succeeded 77da79d
• 💚 Build #75478 succeeded 31c211a
• 💔 Build #75350 failed 4a689f3
• 💔 Build #75238 failed 010c0a5
• 💚 Build #74799 succeeded ca19e27
• 💚 Build #74350 succeeded b55c3c6

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @wayneseymour