test: Add initial automated UI smoke tests (#4393)

[keithchong]

Signed-off-by: Keith Chong [email protected]

#4393

All changes in this PR are in a new folder argoproj/argo-cd/ui-test, which parallels argoproj/argo-cd/test. This is an 'independent' package that resides in argo-cd.

To try it out, ensure you have the changes to the following PR (or update to master since it is merged):
#4683

You can also simply get the contents of the ui-test folder on its own and put it anywhere on your file system.

Then, go to the ui-test folder and run (you need to have node.js installed):

1. npm install
2. npm compile
3. Adjust the env var values in the file ui-test/.env (eg. Change IS_HEADLESS=true if you want to see the test run in the browser)
4. npm test

Checklist:

• Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
• The title of the PR states what changed and the related issues number (used for the release note).
• I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
• Does this PR require documentation updates?
• I've updated documentation as required by this PR.
• Optional. My organization is added to USERS.md.
• I've signed the CLA and my build is green (troubleshooting builds).

[alexmt]

Looking, sorry for the delay.

[wtam2018]

@keithchong I ran into some issue. What do I miss? Thanks.

[wtam@localhost ui-test]$ npm test

> [email protected] pretest /home/wtam/go/src/github.com/argoproj/argo-cd/ui-test
> cp .env out/.env


> [email protected] test /home/wtam/go/src/github.com/argoproj/argo-cd/ui-test
> node out/test001.js

internal/modules/cjs/loader.js:834
  throw err;
  ^

Error: Cannot find module '/home/wtam/go/src/github.com/argoproj/argo-cd/ui-test/out/test001.js'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:831:15)
    at Function.Module._load (internal/modules/cjs/loader.js:687:27)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
    at internal/main/run_main_module.js:17:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}
npm ERR! Test failed.  See above for more details.

[keithchong]

@wtam2018 , just wanted to confirm and rule it out, but did you run npm install and npm compile ?

[alexmt]

Sorry for the ridiculously long delay. Had to prioritize 1.8 bug fixing. Added few minor comments and one main proposal to use Page Object pattern for tests implementation.

Let me know what you think please. Happy to work together on initial implementation in your fork.

[alexmt]

Since we are using yarn everywhere else, can you please generate yarn.lock instead?

[alexmt]

For consistency with backend e2e tests please rename this folder to test/smoke.

[keithchong]

The ui-test folder is at the 'top level', as a sibling to the ui folder because both are Node projects. Moving this to test/smoke will mean these tests will be 'mixed in' with the go test cases at a lower level. Should we keep those tests homogeneous?

[alexmt]

I'm a big fan of page object pattern. The idea is to group page-specific methods into a class that represents a page and class instance can be created by navigating to the page so that test look as following:

// initializes web driver and returns instance of `Navigation` class that holds`WebDriver` instance
const navigation = utils.init();
// navigates to `/applications` page, waits for navigation to complete and returns `ApplicationsList` class
const appListPage = navigation.goToAppList();
// clicks new app buttons and returns `NewApplicationDialog` that holds link to sliding panel element
const newAppDialog = await appListPage.newAppButtonClick();
// locates app name element and sends 'myApp' keys
await newAppDialog.setAppName('myApp');
...

ApplicationsList:

export class ApplicationsList {
  panel: WebElement;
  public async setAppName(val: string) {
    findUiElement(Const.CREATE_APPLICATION_FIELD_APP_NAME);
    await appNameField.sendKeys(appName);
  }
}

This approach was very helpful in one of my previous projects and helped us to effectively structure test utility functions.

[keithchong]

Hi @alexmt, what's the naming convention for file names in ArgoCD? For example,

argo-cd/ui/src/app/shared/components/version-info/version-info-panel.tsx

Line 13 in d2295d4

export class VersionPanel extends React.Component<VersionPanelProps, {copyState: CopyState}> {

, the file name is version-info-panel.tsx and the class name defined is VersionPanel. Although, not all files contain classes is probably one reason why VersionPanel.tsx is not used (I'm thinking Java, of course).

I'll try to follow the same structure as in the actual code, eg. I'll create the applications-list folder that contains the applications-list.ts file.

[keithchong]

The latest changes now include the Page object design pattern. I've added some extras features:

• The "create new app, sync, delete app" test (test001.ts) will now work for any number of existing applications already created, as long as the new application name is unique. This takes advantage of the change made in test: Allow individual application tiles to be selectable (#4841) #4851
  • API methods to access all three buttons on any specific app in the Application Tile view
  • API methods to access/test/wait for the status (healthy, sync'ed, operation states) on any app in the Application Tile view
• Other Page/Component-based APIs added (although not used yet). eg. clickPromptCancel() in PopupManager. So this base framework that we have can be expanded as more tests are added
• Support added to navigate to any of the four navigation bar buttons. See Navigation. See new test,test002.ts
• Set the same prettier settings (.prettierrc) as in ArgoCD

All tests should now start with something like:

const navigation = await UiTestUtilities.init();
    try {
        const appsList: ApplicationsList = await navigation.clickApplicationsNavBarButton();

and end with navigation.quit()

To run the default test (this will change when we use mocha)

• yarn install (if you haven't done so)
• yarn run compile
• yarn run test

I still have one outstanding todo w.r.t. the review comments, if my counter-argument is given a thumbs-down:

• Have to rename/move the ui-test direct to test/smoke

Some outstanding things to consider in future PRs:

• Compile step will put the output javascript in the out folder. Debugging the test case might be tricky since the line numbers are for the formatted javascript, instead of the actual typescript source. Need to resolve this.
• Some new CSS/XPath locators should be replaced by unique ids
• Need to decide on a naming convention for test cases. Currently there are two, test001.ts and test002.ts
• Use mocha to run a suite a tests
• etc. (see more in Automated smoke test that mimics user behavior UI and CLI #4393)

[keithchong]

This PR depends on:
#5074 (this has been merged as of this comment/post)
argoproj/argo-ui#86

[keithchong]

Hi @alexmt , is this ok to be merged? The latest changes have to be reviewed.

[alexmt]

Thank you @keithchong ! LGTM

Created followup ticket #6847 to work on Docker file to package tests dependencies.