[CI][Tests] Add BWC tests for 2.7 and 2.8

[manasvinibs]

Description

Test data generated with:

yarn test:bwc -o https://artifacts.opensearch.org/releases/bundle/opensearch/2.6.0/opensearch-2.6.0-linux-x64.tar.gz -d https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.6.0/opensearch-dashboards-2.6.0-linux-x64.tar.gz -g true

yarn test:bwc -o https://artifacts.opensearch.org/releases/bundle/opensearch/2.7.0/opensearch-2.7.0-linux-x64.tar.gz -d https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.7.0/opensearch-dashboards-2.7.0-linux-x64.tar.gz -g true

yarn test:bwc -o https://artifacts.opensearch.org/releases/bundle/opensearch/2.6.0/opensearch-2.6.0-linux-x64.tar.gz -d https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.6.0/opensearch-dashboards-2.6.0-linux-x64.tar.gz -g true -s true

yarn test:bwc -o https://artifacts.opensearch.org/releases/bundle/opensearch/2.7.0/opensearch-2.7.0-linux-x64.tar.gz -d https://artifacts.opensearch.org/releases/bundle/opensearch-dashboards/2.7.0/opensearch-dashboards-2.7.0-linux-x64.tar.gz -g true -s true

Issues Resolved

#3776

Check List

• All tests pass
  • yarn test:jest
  • yarn test:jest_integration
  • yarn test:ftr
• New functionality includes testing.
• New functionality has been documented.
• Update CHANGELOG.md
• Commits are signed per the DCO using --signoff

[codecov]

Codecov Report

Merging #4023 (0e6bb6e) into main (bb0c98f) will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #4023      +/-   ##
==========================================
+ Coverage   66.35%   66.39%   +0.03%     
==========================================
  Files        3229     3229              
  Lines       62138    62293     +155     
  Branches     9619     9669      +50     
==========================================
+ Hits        41233    41359     +126     
- Misses      18584    18600      +16     
- Partials     2321     2334      +13     

Flag	Coverage Δ
Linux	66.33% <ø> (+0.03%)	⬆️
Windows	66.33% <ø> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 7 files with indirect coverage changes

[manasvinibs]

There is a cypress error while generating the data (without-security) for timeline visualization using 2.6 and 2.7 artifacts.

Generating BWC test data without security
       adds Timeline visualization:
     CypressError: Timed out retrying after 60050ms: `cy.click()` failed because this element is `disabled`:

`<button class="euiButton euiButton--primary euiButton--small euiButton-isDisabled" disabled="" type="button" data-test-subj="visualizeSaveButton">...</button>`

Fix this problem, or use `{force: true}` to disable error checking.

https://on.cypress.io/element-cannot-be-interacted-with
      at $Cy.ensureNotDisabled (http://localhost:5601/__cypress/runner/cypress_runner.js:163868:76)
      at runAllChecks (http://localhost:5601/__cypress/runner/cypress_runner.js:150539:14)
      at retryActionability (http://localhost:5601/__cypress/runner/cypress_runner.js:150616:16)
      at tryCatcher (http://localhost:5601/__cypress/runner/cypress_runner.js:13022:23)
      at Function.Promise.attempt.Promise.try (http://localhost:5601/__cypress/runner/cypress_runner.js:10296:29)
      at whenStable (http://localhost:5601/__cypress/runner/cypress_runner.js:168808:63)
      at http://localhost:5601/__cypress/runner/cypress_runner.js:168305:14
      at tryCatcher (http://localhost:5601/__cypress/runner/cypress_runner.js:13022:23)
      at Promise._settlePromiseFromHandler (http://localhost:5601/__cypress/runner/cypress_runner.js:10957:31)
      at Promise._settlePromise (http://localhost:5601/__cypress/runner/cypress_runner.js:11014:18)
      at Promise._settlePromise0 (http://localhost:5601/__cypress/runner/cypress_runner.js:11059:10)
      at Promise._settlePromises (http://localhost:5601/__cypress/runner/cypress_runner.js:11139:18)
      at Promise._fulfill (http://localhost:5601/__cypress/runner/cypress_runner.js:11083:18)
      at http://localhost:5601/__cypress/runner/cypress_runner.js:12697:46
  From Your Spec Code:
      at Context.eval (http://localhost:5601/__cypress/tests?p=cypress/integration/without-security/helpers/generate_data.js:150:54)




  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        4                                                                                │
  │ Passing:      3                                                                                │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  1                                                                                │
  │ Video:        true                                                                             │
  │ Duration:     1 minute, 27 seconds                                                             │
  │ Spec Ran:     without-security/helpers/generate_data.js                                        │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


  (Screenshots)

  -  /home/ubuntu/OpenSearch-Dashboards/bwc_tmp/test/cypress/screenshots/without-secu     (1280x720)
     rity/helpers/generate_data.js/Generating BWC test data without security -- adds                
     Timeline visualization (failed).png                                                         

[ananzh]

LGTM

[manasvinibs]

In this PR #2720 we recently introduced a change where the default expression for timeline changed from .es() to .opensearch() , so now we have one more step in the workflow where we need to click update button when we create timeline vis before we save. So this line is to get the update button click through.

[joshuarrrr]

sorry I missed this! thanks for the update!

[ananzh]

@manasvinibs I think bwc tests still fail due to Unable to find an artifact with the name: linux-x64-3.0.0 as shown here https://github.com/opensearch-project/OpenSearch-Dashboards/actions/runs/5160037751/jobs/9410249664

@kavilla mentioned that bwc test utilizes the build output artifact from build-min-artifact-tests task and that artifact only gets hosted by github for 1 day (intentionally). So this PR should be re-checked after we have a PR to fix it.

If we need to merge it early, I am okay with manual tests and screenshots posted here.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-4023-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 93d543e868416d41daa122eb987e5721cb89aac6
# Push it to GitHub
git push --set-upstream origin backport/backport-4023-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4023-to-2.x.