Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test that old zebrad.toml files still work #4625

Closed
3 tasks
teor2345 opened this issue Jun 15, 2022 · 5 comments · Fixed by #4676
Closed
3 tasks

Test that old zebrad.toml files still work #4625

teor2345 opened this issue Jun 15, 2022 · 5 comments · Fixed by #4676
Assignees
Labels
C-testing Category: These are tests I-usability Zebra is hard to understand or use

Comments

@teor2345
Copy link
Contributor

teor2345 commented Jun 15, 2022

Motivation

We want to make sure that we don't accidentally break the config format.

Designs

  • Commit a copy of the zebrad generate output to a test folder in the Zebra git repository
  • Add an acceptance test that makes sure the config is up to date in the Zebra git repository
  • Add an acceptance test that launches zebrad start using the config from the Zebra git repository

We already have a test that launches using a freshly generated config.

@teor2345 teor2345 added S-needs-triage Status: A bug report needs triage P-Medium ⚡ I-usability Zebra is hard to understand or use C-testing Category: These are tests labels Jun 15, 2022
@oxarbitrage
Copy link
Contributor

I am unsure if we should use cargo insta, snapshots and how.

1- If you make a snapshot of the config, you have to remove the header meta data to convert it to a real config file that zebra can use to start.
2- If the config changes, the snapshot of the config will need to be updated. Then when zebra is started it will use a new config and not an old one.

Here is what i think we should do:

  • Create a file with the current config and store it in a test folder.
  • Create a test that start zebra with that config.

If there are breaking changes to the config in the future the old config will fail to start zebra, we will then have to update the stored old config file to make the test pass. That way we will know before merging and can notify users (I think is the only thing we can do if the new config has breaking changes and we still want to go ahead with them).

Let me know how do you see it, or if we want to use snapshots please explain the workflow a bit more. Thanks.

@teor2345
Copy link
Contributor Author

Here is what i think we should do:

  • Create a file with the current config and store it in a test folder.
  • Create a test that start zebra with that config.

This seems better to me, I'll edit the ticket.

@teor2345
Copy link
Contributor Author

We also need to make sure that new config fields are added to the test config.
If we don't, we could make a breaking change to a config that was added recently, and the test wouldn't pick it up.

@oxarbitrage
Copy link
Contributor

Store a copy of the zebrad generate output in a test folder in git

Can you clarify the git part please ? Thanks!

@teor2345
Copy link
Contributor Author

Store a copy of the zebrad generate output in a test folder in git

Can you clarify the git part please ? Thanks!

I meant "in the Zebra git repository", I've added that to the ticket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-testing Category: These are tests I-usability Zebra is hard to understand or use
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants