Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvement: Add a possibility to do good JSON assertions #279

Open
14 tasks
wurzelkuchen opened this issue Jul 31, 2024 · 1 comment · Fixed by #294
Open
14 tasks

Improvement: Add a possibility to do good JSON assertions #279

wurzelkuchen opened this issue Jul 31, 2024 · 1 comment · Fixed by #294
Assignees
Labels
codeReadyForMerge The issue was implemented and the branch is ready to be reviewed and merged into develop docsReadyForMerge Documentation was created for this issue in a separate branch which is ready to merge to develop
Milestone

Comments

@wurzelkuchen
Copy link
Contributor

wurzelkuchen commented Jul 31, 2024

Quick Description

In several projects we check for JSON data (REST, Google Analytics, etc.) These checks often result in a big unreadable text blob as an error message.
We can do better. We should add a helper class to assert JSON Strings. If an assertion fails however we need a good message and overview on the JSON data inside our allure report.

User Story

As a test report reader,
I want to see directly what is wrong with the JSON data, compared to the expected test data
so that I don't waste time fiddling around with the data.

Upfront research

  • Check if there's a well maintained and useful json compare library for Java
  • Search for a good client side JSON diff library in JS
  • The JSON Diff Library should be hosted in a CDN

Functional Requirements

  • Add a helper class for JSON assertion
  • Add Methods for:
  • JSONAssert.assertEquals(actualData,expectedData)
  • JSONAssert.assertHas(actualData,jsonPath)
  • JSONAssert.assertEquals(data,jsonPath)
  • JSONAssert.assertMatch(data,jsonPath,regExp) (Updated 2024-08-21)
  • think about what additional assertion methods could be useful
  • Give every assertion method the possibility to add a custom message
  • For the (non custom) error messages, keep in mind that they should work with and without allure
  • If the assertion fails, add a nice json diff view to the allure report before throwing an AssertionError.
  • For the diff view we should do a two column view marking the differences with colors.
  • Add support for different Comparison modes (Strict, StrictOrder, Lenient, nonExtensible) (Updated 2024-08-21)

Note: for the methods with jsonPath be reminded that those can have simple fields as well as complex json data as a result.

Documentation Requirements

  • Add a documentation page
  • Mention the allure diff view on the reports page of the wiki

Test Requirements

  • Write sufficient unit tests
  • Do a smoke test using the feature with an external library like the neodymium example (check if the allure view works)
@wurzelkuchen wurzelkuchen added docsNeeded The documenttion needs to be checked if there are changes needed and if so, adaptions should be made codeImplementationNeeded This issue requires a change in the code of Neodymium labels Jul 31, 2024
@wurzelkuchen wurzelkuchen added this to the v5.1.0 milestone Jul 31, 2024
@wurzelkuchen wurzelkuchen changed the title Add a possibility to do good JSON assertions Improvement: Add a possibility to do good JSON assertions Jul 31, 2024
kqmpetenz pushed a commit that referenced this issue Aug 15, 2024
kqmpetenz pushed a commit that referenced this issue Aug 15, 2024
kqmpetenz pushed a commit that referenced this issue Aug 15, 2024
kqmpetenz pushed a commit that referenced this issue Aug 20, 2024
kqmpetenz pushed a commit that referenced this issue Aug 20, 2024
@wurzelkuchen
Copy link
Contributor Author

Update: after internal discussion the requirements were adapted. We want to support different JSON compare modes, as we need those in different customer projects

kqmpetenz pushed a commit that referenced this issue Sep 9, 2024
@kqmpetenz kqmpetenz added docsReadyForMerge Documentation was created for this issue in a separate branch which is ready to merge to develop codeReadyForMerge The issue was implemented and the branch is ready to be reviewed and merged into develop and removed docsNeeded The documenttion needs to be checked if there are changes needed and if so, adaptions should be made codeImplementationNeeded This issue requires a change in the code of Neodymium labels Sep 10, 2024
@kqmpetenz kqmpetenz linked a pull request Sep 10, 2024 that will close this issue
@kqmpetenz kqmpetenz assigned wurzelkuchen and unassigned kqmpetenz Sep 10, 2024
wurzelkuchen added a commit that referenced this issue Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
codeReadyForMerge The issue was implemented and the branch is ready to be reviewed and merged into develop docsReadyForMerge Documentation was created for this issue in a separate branch which is ready to merge to develop
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants