Skip to content

Commit

Permalink
[BWC] updates to BWC test scripts (opensearch-project#1190)
Browse files Browse the repository at this point in the history
Allows the calling script with file location or url to enable
the CI/CD pipeline to pass URLs.

Some minor clean ups related to linter and pass by CSV for params.

Finally, update to add releases parameter which enables us to test
version mismatch if/when we allow the out of the box experience of
OpenSearch Dashboards to work with previous versions of OpenSearch.

Signed-off-by: Kawika Avilla <[email protected]>
  • Loading branch information
kavilla committed Mar 9, 2022
1 parent 00e0693 commit eb990c8
Show file tree
Hide file tree
Showing 44 changed files with 1,126 additions and 50 deletions.
7 changes: 7 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -757,5 +757,12 @@ module.exports = {
],
},
},
{
files: ['cypress/**/*.js'],
rules: {
'import/no-unresolved': 'off',
'no-undef': 'off',
},
},
],
};
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ trash
/built_assets
target
/build
/bwc_tmp
.jruby
.idea
*.iml
Expand Down
17 changes: 17 additions & 0 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,24 @@ To debug functional tests:
Say that you would want to debug a test in CI group 1, you can run the following command in your environment:
`node --debug-brk --inspect scripts/functional_tests.js --config test/functional/config.js --include ciGroup1 --debug`

<<<<<<< HEAD
This will print of an address, to which you could open your chrome browser on your instance and navigate to `chrome://inspect/#devices` and inspect the functional test runner `scripts/functional_tests.js`.
=======
This will print off an address, to which you could open your chrome browser on your instance and navigate to `chrome://inspect/#devices` and inspect the functional test runner `scripts/functional_tests.js`.

### Backwards Compatibility tests
To run all the backwards compatibility tests on OpenSearch Dashboards without security:

`yarn test:bwc -o [test path to opensearch.tar.gz] -d [test path to opensearch-dashboards.tar.gz]`

To run all the backwards compatibility tests on OpenSearch Dashboards with security, pass the security parameter to the test:

`yarn test:bwc -o [test path to opensearch.tar.gz] -d [test path to opensearch-dashboards.tar.gz] -s true`

To run specific versions' backwards compatibility tests, pass the versions to the test:

`yarn test:bwc -o [test path to opensearch.tar.gz] -d [test path to opensearch-dashboards.tar.gz] -v "[test versions]"`
>>>>>>> 6960f9c538... [BWC] updates to BWC test scripts (#1190)
### Additional checks
Make sure you run lint checker before submitting a pull request. To run lint checker:
Expand Down
86 changes: 36 additions & 50 deletions bwctest.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -2,42 +2,52 @@

set -e

# TODO: Update to include all known BWC of data
DEFAULT_VERSIONS="osd-1.1.0"

function usage() {
echo ""
echo "This script is used to run bwc tests on a remote OpenSearch/Dashboards cluster."
echo "--------------------------------------------------------------------------"
echo "Usage: $0 [args]"
echo ""
echo "Required arguments:"
echo "None"
echo -e "-d DASHBOARDS\t, Specify the url of the build/dist of OpenSearch Dashboards"
echo ""
echo "Optional arguments:"
echo -e "-a BIND_ADDRESS\t, defaults to localhost | 127.0.0.1, can be changed to any IP or domain name for the cluster location."
echo -e "-o OPENSEARCH\t, Specify the url of the build/dist of OpenSearch"
echo -e "-b BIND_ADDRESS\t, defaults to localhost | 127.0.0.1, can be changed to any IP or domain name for the cluster location."
echo -e "-p BIND_PORT\t, defaults to 9200 or 5601 depends on OpenSearch or Dashboards, can be changed to any port for the cluster location."
echo -e "-b BUNDLED_OSD\t(true | false), defaults to true. Specify the usage of bundled Dashboards or not."
echo -e "-s SECURITY_ENABLED\t(true | false), defaults to true. Specify the OpenSearch/Dashboards have security enabled or not."
echo -e "-c CREDENTIAL\t(usename:password), no defaults, effective when SECURITY_ENABLED=true."
echo -e "-h\tPrint this message."
echo "--------------------------------------------------------------------------"
}

while getopts ":ha:p:b:c:" arg; do
while getopts ":h:b:p:s:c:o:d:" arg; do
case $arg in
h)
usage
exit 1
;;
a)
b)
BIND_ADDRESS=$OPTARG
;;
p)
BIND_PORT=$OPTARG
;;
b)
BUNDLED_OSD=$OPTARG
s)
SECURITY_ENABLED=$OPTARG
;;
c)
CREDENTIAL=$OPTARG
;;
o)
OPENSEARCH=$OPTARG
;;
d)
DASHBOARDS=$OPTARG
;;
:)
echo "-${OPTARG} requires an argument"
usage
Expand All @@ -50,50 +60,26 @@ while getopts ":ha:p:b:c:" arg; do
esac
done

[ -z "$BIND_ADDRESS" ] && BIND_ADDRESS="localhost"
[ -z "$BIND_PORT" ] && BIND_PORT="5601"
[ -z "$SECURITY_ENABLED" ] && SECURITY_ENABLED="false"
[ -z "$CREDENTIAL" ] && CREDENTIAL="admin:admin"

if [ -z "$BIND_ADDRESS" ]
then
BIND_ADDRESS="localhost"
fi

if [ -z "$BIND_PORT" ]
then
BIND_PORT="5601"
fi

if [ -z "$BUNDLED_OSD" ]
then
BUNDLED_OSD="true"
fi

if [ -z "$CREDENTIAL" ]
then
CREDENTIAL="admin:admin"
USERNAME=`echo $CREDENTIAL | awk -F ':' '{print $1}'`
PASSWORD=`echo $CREDENTIAL | awk -F ':' '{print $2}'`
fi

cwd=$(pwd)
dir="bwc-tmp"
if [ -d "$dir" ]; then
rm -rf "$dir"
echo "bwc-tmp exists and needs to be removed"
fi

mkdir "$dir"
git clone https://github.com/opensearch-project/opensearch-dashboards-functional-test "$dir"
rm -rf "$dir/cypress"
cp -r cypress "$dir"
cd "$dir"

npm install
# If no OpenSearch build was passed then this constructs the version
if [ -z "$OPENSEARCH" ]; then
IFS='/' read -ra SLASH_ARR <<< "$DASHBOARDS"
# Expected to be opensearch-x.y.z-platform-arch.tar.gz
TARBALL="${SLASH_ARR[12]}"
IFS='-' read -ra DASH_ARR <<< "$TARBALL"
# Expected to be arch.tar.gz
DOTS="${DASH_ARR[4]}"
IFS='.' read -ra DOTS_ARR <<< "$DOTS"

VERSION="${DASH_ARR[2]}"
PLATFORM="${DASH_ARR[3]}"
ARCH="${DOTS_ARR[0]}"

if [ $BUNDLED_OSD = "true" ]
then
echo "run security enabled tests"
npx cypress run --spec "$cwd/bwc-tmp/cypress/integration/bundled-osd/*.js"
else
npx cypress run --spec "$cwd/bwc-tmp/cypress/integration/osd/*.js"
OPENSEARCH="https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/$VERSION/latest/$PLATFORM/$ARCH/dist/opensearch/opensearch-$VERSION-$PLATFORM-$ARCH.tar.gz"
fi

rm -rf "$cwd/$dir"
./scripts/bwctest_osd.sh -b $BIND_ADDRESS -p $BIND_PORT -s $SECURITY_ENABLED -c $CREDENTIAL -o $OPENSEARCH -d $DASHBOARDS -v $DEFAULT_VERSIONS
41 changes: 41 additions & 0 deletions cypress/integration/with-security/check_advanced_settings.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

import {
MiscUtils,
LoginPage,
} from '@opensearch-dashboards-test/opensearch-dashboards-test-library';

const miscUtils = new MiscUtils(cy);
const loginPage = new LoginPage(cy);

describe('verify the advanced settings are saved', () => {
beforeEach(() => {
miscUtils.visitPage('app/management/opensearch-dashboards/settings');
loginPage.enterUserName('admin');
loginPage.enterPassword('admin');
loginPage.submit();
});

it('the dark mode is on', () => {
cy.get('[data-test-subj="advancedSetting-editField-theme:darkMode"]')
.invoke('attr', 'aria-checked')
.should('eq', 'true');
});

it('the Timeline default columns field is set to 4', () => {
cy.get('[data-test-subj="advancedSetting-editField-timeline:default_columns"]').should(
'have.value',
4
);
});

it('the Timeline Maximum buckets field is set to 4', () => {
cy.get('[data-test-subj="advancedSetting-editField-timeline:max_buckets"]').should(
'have.value',
4
);
});
});
25 changes: 25 additions & 0 deletions cypress/integration/with-security/check_default_page.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

import {
MiscUtils,
LoginPage,
} from '@opensearch-dashboards-test/opensearch-dashboards-test-library';

const miscUtils = new MiscUtils(cy);
const loginPage = new LoginPage(cy);

describe('verify default landing page work for bwc', () => {
beforeEach(() => {
miscUtils.visitPage('');
loginPage.enterUserName('admin');
loginPage.enterPassword('admin');
loginPage.submit();
});

it('the overview page is set as the default landing page', () => {
cy.url().should('include', '/app/opensearch_dashboards_overview#/');
});
});
82 changes: 82 additions & 0 deletions cypress/integration/with-security/check_filter_and_query.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

import {
MiscUtils,
CommonUI,
LoginPage,
} from '@opensearch-dashboards-test/opensearch-dashboards-test-library';

const miscUtils = new MiscUtils(cy);
const commonUI = new CommonUI(cy);
const loginPage = new LoginPage(cy);

describe('check dashboards filter and query', () => {
beforeEach(() => {
miscUtils.visitPage('app/dashboards#');
loginPage.enterUserName('admin');
loginPage.enterPassword('admin');
loginPage.submit();
});

afterEach(() => {
cy.clearCookies();
});

it('tenant-switch-modal page should show and be clicked', () => {
cy.get('[data-test-subj="tenant-switch-modal"]');
cy.get('[data-test-subj="confirm"]').click();
});

describe('osx filter and query should work in [Logs] Web Traffic dashboards', () => {
beforeEach(() => {
cy.get('[data-test-subj="dashboardListingTitleLink-[Logs]-Web-Traffic"]').click();
cy.get('[data-test-subj="breadcrumb last"]')
.invoke('attr', 'title')
.should('eq', '[Logs] Web Traffic');
});

it('osx filter and query should exist and be named correctly', () => {
cy.get('[data-test-subj="saved-query-management-popover-button"]').click();
cy.get('[data-test-subj="saved-query-management-popover"]')
.find('[class="osdSavedQueryListItem__labelText"]')
.should('have.text', 'test-query')
.click();
cy.get('[data-test-subj="queryInput"]').should('have.text', 'resp=200');
cy.get(
'[data-test-subj="filter filter-enabled filter-key-machine.os filter-value-osx filter-unpinned "]'
)
.should('have.text', 'osx filter')
.click();
cy.get('[data-test-subj="editFilter"]').click();
cy.get('[data-test-subj="filterFieldSuggestionList"]')
.find('[data-test-subj="comboBoxInput"]')
.should('have.text', 'machine.os');
cy.get('[data-test-subj="filterOperatorList"]')
.find('[data-test-subj="comboBoxInput"]')
.should('have.text', 'is');
cy.get('[data-test-subj="filterParams"]').find('input').should('have.value', 'osx');
});

it('osx filter and query should function correctly', () => {
cy.get('[data-test-subj="saved-query-management-popover-button"]').click();
cy.get('[data-test-subj="saved-query-management-popover"]')
.find('[class="osdSavedQueryListItem__labelText"]')
.should('have.text', 'test-query')
.click();
commonUI.setDateRange('Dec 1, 2021 @ 00:00:00.000', 'Jan 1, 2021 @ 00:00:00.000');

//[Logs] vistor chart should show osx 100%
cy.get('[data-title="[Logs] Visitors by OS"]')
.find('[class="label"]')
.should('have.text', 'osx (100%)');

//[Logs] Response chart should show 200 label
cy.get('[data-title="[Logs] Response Codes Over Time + Annotations"]')
.find('[title="200"]')
.should('have.text', '200');
});
});
});
Loading

0 comments on commit eb990c8

Please sign in to comment.