Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automation to Run CLI e2e tests on various different OS #25955

Closed
akashkeshari opened this issue Mar 28, 2023 · 1 comment · Fixed by #26782
Closed

Automation to Run CLI e2e tests on various different OS #25955

akashkeshari opened this issue Mar 28, 2023 · 1 comment · Fixed by #26782
Assignees
Labels
Azure CLI Team The command of the issue is owned by Azure CLI team feature-request Test Framework

Comments

@akashkeshari
Copy link

Related command
azdev test azext_connectedk8s.{function/command_to_test} --discover --live

Is your feature request related to a problem? Please describe.
Our team owns the connectedk8s CLI extension.
There are two problems which we want to highlight regarding cli e2e testing framework:

  1. One of our customers recently reported an issue when running the az connectedk8s commands on a Cent OS 7 VM. The 'az connectedk8s connect' command was failing with 'Import Error' since we had introduced circular imports in the 1.3.9 version of the connectedk8s CLI. However, this issue did not repro on Ubuntu & Windows and hence we could not catch this during e2e tests
  2. The CLI extension testing pipeline runs all of the tests only for Linux (not sure which one). We claim that the connectedk8s CLI will work on any Linux, Windows & Mac distros. However, we end up doing manual testing for on windows and mac before releasing the connectedk8s CLI code changes.

Describe the solution you'd like
CLI e2e test pipeline provides a way to run automation tests on various OS. We are specifically looking out for options to run the e2e tests on Windows, Ubuntu, Cent OS 7 & Mac OS for now. This would help us save testing time/cost(solve problem #2) and also prevent bugs from being released for certain types of OS distros (as explained in problem #1 above)

Describe alternatives you've considered
The only alternative is to do manual testing locally with Windows, Mac, Cent OS 7, and the testing pipeline can be used to test on Linux. This is neither a wholesome experience to the developer and nor the adoption of testing pipeline that easy when manual steps are required

Additional context

@wangzelin007 wangzelin007 self-assigned this Mar 28, 2023
@yonzhan yonzhan added Test Framework feature-request Azure CLI Team The command of the issue is owned by Azure CLI team labels Mar 28, 2023
@yonzhan yonzhan added this to the April 2023 (2023-04-25) milestone Mar 28, 2023
@yonzhan
Copy link
Collaborator

yonzhan commented Mar 28, 2023

different OS support

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Azure CLI Team The command of the issue is owned by Azure CLI team feature-request Test Framework
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants