Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test framework: Initial impl of exhaustive state verification with rollback #1713

Merged
merged 19 commits into from
Mar 6, 2023

Conversation

dcorral
Copy link
Contributor

@dcorral dcorral commented Jan 27, 2023

What kind of PR is this?:

/kind fix

What this PR does / why we need it:

Adds capture_rollback_verify decorator to DefiTestFramework.

Any member function inside a given DefiTestFramework child class can now use @DefiTestFramework.capture_rollback_verify decorator. This decorator will capture the state of the chain, run a rollback and verify the data is properly restored after the function execution.

Usage example

from test_framework.test_framework import DefiTestFramework

class ExampleTest(DefiTestFramework):
    
    def test_scenario(self):
        # test logic   

    @DefiTestFramework.capture_rollback_verify
    def test_scenario_with_rollback(self):
        self.test_scenario()

    def run_test(self):
        self.test_scenario()
        self.test_scenario_with_rollback()

if __name__ == '__main__':
    ExampleTest().main()

WARNING: This decorator when used in large tests can become very expensive. The following commands are run to gather the data to be verified on rollback:

[
    self.nodes[0].logaccountbalances(),
    self.nodes[0].logstoredinterests(),
    self.nodes[0].listvaults(),
    self.nodes[0].listtokens(),
    self.nodes[0].listgovs(),
    self.nodes[0].listmasternodes(),
    self.nodes[0].listaccounthistory(),
    self.nodes[0].getburninfo(),
    self.nodes[0].getloaninfo(),
    self.nodes[0].listanchors(),
    self.nodes[0].listgovproposals(),
    self.nodes[0].listburnhistory(),
    self.nodes[0].listcommunitybalances()
]

@prasannavl
Copy link
Member

prasannavl commented Mar 3, 2023

Could you please update the description to reflect the changes, and also add some in code comments as doc on the function on the final state that you expect before it's usable - along with a note that it's currently an expensive one to be called and that use caution if used before the final state impl is finished.

Happy to merge it after that to prevent it from becoming stale.

@dcorral
Copy link
Contributor Author

dcorral commented Mar 3, 2023

Waiting on fix to #1788 responsible for consortium test fail

@prasannavl prasannavl changed the title Test framework rollback decorator Test framework: Initial impl of exhaustive state capture verification with rollback Mar 6, 2023
@prasannavl prasannavl changed the title Test framework: Initial impl of exhaustive state capture verification with rollback Test framework: Initial impl of exhaustive state verification with rollback Mar 6, 2023
@prasannavl prasannavl merged commit 532eafe into master Mar 6, 2023
@prasannavl prasannavl deleted the test/rollback-decorator branch March 6, 2023 08:05
@dcorral dcorral mentioned this pull request Mar 7, 2023
40 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants