Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add base structure for testing of statuses from real cloud #334

Merged

Conversation

rgildein
Copy link
Contributor

@rgildein rgildein commented Mar 26, 2024

With these changes, we can simply add any result form real cloud to tests/unit/sample_plans directory in yaml file formatted like this:

plan: |
  Upgrade plan defined as string

applications:
  keystone:
    can_upgrade_to: ussuri/stable
    charm: keystone
    channel: ussuri/stable
    config:
      openstack-origin:
        value: distro
      action-managed-upgrade:
        value: true
    origin: ch
    series: focal
    subordinate_to: []
    workload_version: 17.0.1
    units:
      keystone/0:
        name: keystone/0
        machine: '0'
        workload_version: 17.0.1
        os_version: ussuri
    machines:
      '0':
        id: '0'
        apps: !!python/tuple ['keystone-ldap']
        az: az-0

My only two concerns are:
- usage of pytest-subtests, which result could be strange if test failed, as you can see in current CI After discussion I choose to got with only fixture returning dictionary with file as key, and tuple value with model and exp_plan. The model.get_applications returns applications defined in file.
- these test should be more part of functional tests instead of unit tests, or somehow exclude them from coverage report (e.g. 100% coverage is achieved without these tests) After discussions we agreed that new tox env should be created, so I created mocked-plans. I also add new jobs to run this environment to our GitHub workflow.

@rgildein rgildein added the enhancement New feature or request label Mar 26, 2024
@rgildein rgildein self-assigned this Mar 26, 2024
@rgildein rgildein requested a review from a team as a code owner March 26, 2024 17:35
@rgildein rgildein requested review from Pjack, aieri, agileshaw and jneo8 March 26, 2024 17:35
@rgildein rgildein force-pushed the tests/BSENG-2204/add-testing-structure branch from f244ef3 to 78cc044 Compare March 27, 2024 13:39
@rgildein rgildein marked this pull request as draft March 27, 2024 16:15
Add mocked-plans tox environment and job in GitHub workflows.
I also remove usage of boots-actions, since it's old and not
maintained.
@rgildein rgildein marked this pull request as ready for review March 28, 2024 12:47
@rgildein
Copy link
Contributor Author

@gabrielcocenza Thanks for doing #341. I merged, so I can use it here.

@rgildein rgildein force-pushed the tests/BSENG-2204/add-testing-structure branch from 5380e2d to 7998cf5 Compare March 28, 2024 13:36
Copy link
Contributor

@agileshaw agileshaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a partial review since I haven't finish going through all the changes yet.

pyproject.toml Outdated Show resolved Hide resolved
tests/unit/utils.py Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
tests/unit/apps_planning/test_apps_plan.py Outdated Show resolved Hide resolved
tests/unit/utils.py Outdated Show resolved Hide resolved
tests/unit/apps_planning/test_apps_plan.py Outdated Show resolved Hide resolved
tests/unit/sample_plans/base.yaml Outdated Show resolved Hide resolved
Copy link
Contributor

@agileshaw agileshaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good with some minor changes needed

tests/mocked_plans/utils.py Outdated Show resolved Hide resolved
pyproject.toml Show resolved Hide resolved
tests/mocked_plans/test_base_plan.py Outdated Show resolved Hide resolved
tests/unit/utils.py Show resolved Hide resolved
tests/mocked_plans/conftest.py Outdated Show resolved Hide resolved
@rgildein rgildein requested a review from agileshaw April 3, 2024 21:34
agileshaw
agileshaw previously approved these changes Apr 3, 2024
Copy link
Contributor

@agileshaw agileshaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

Copy link
Contributor

@aieri aieri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, I really like this work!

I have only left some non-blocking suggestions.

Additionally and as a sidenote, I don't like that we're tightly coupling what a plan is to its string representation; we should be able to change how a plan is represented without having to alter the tests that verify that it is correct. But of course this cannot be solved in this PR.

tests/mocked_plans/test_base_plan.py Show resolved Hide resolved
tests/mocked_plans/utils.py Show resolved Hide resolved
@rgildein rgildein dismissed gabrielcocenza’s stale review April 4, 2024 08:33

I want to merge this to ublock another tasks.

@rgildein rgildein merged commit 45950d0 into canonical:main Apr 4, 2024
5 checks passed
@rgildein rgildein deleted the tests/BSENG-2204/add-testing-structure branch April 4, 2024 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants