Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata API: Tests (de)serialization with invalid arguments #1434

Closed
3 tasks
MVrachev opened this issue Jun 3, 2021 · 2 comments · Fixed by #1460
Closed
3 tasks

Metadata API: Tests (de)serialization with invalid arguments #1434

MVrachev opened this issue Jun 3, 2021 · 2 comments · Fixed by #1460
Assignees
Labels
Milestone

Comments

@MVrachev
Copy link
Collaborator

MVrachev commented Jun 3, 2021

Description of issue or feature request:
I started creating issues documenting some research I do for how we use our metadata attributes.
For example #1418, #1419 and #1420.
The end goal of this research is to decide how we are going to validate each of them, make the necessary changes and add tests ensuring the validation works.

Together with @jku we discussed that to achieve the last step - adding tests ensuring the validation works
it will be better if we wait for #1391 to be resolved.
That way we would come up with a better-refined approach of running a test case against multiple inputs.

Also, I wanted to point out that there was a suggestion to use a fuzzing tool for this.
Maybe it's worth exploring.

We will want to test cases when our attributes are:

  • empty values
  • values from the wrong type
  • non existent - they don't appear in the input dictionary when calling from_dict()
@MVrachev MVrachev changed the title Metadata API: Test (de)serialization with invalid arguments Metadata API: Tests (de)serialization with invalid arguments Jun 3, 2021
@MVrachev MVrachev added backlog Issues to address with priority for current development goals and removed backlog Issues to address with priority for current development goals labels Jun 3, 2021
@sechkova sechkova added this to the weeks24-25 milestone Jun 9, 2021
@MVrachev
Copy link
Collaborator Author

I created a prototype of how those tests could look like in this branch: https://github.com/MVrachev/tuf/tree/invalid-attributes.
@jku what do you think?

@jku
Copy link
Member

jku commented Jun 16, 2021

I don't think you get any value from introducing the dict-in-dict structure for the data?

You could have these in the same file as the valid test cases, using the same DataSet data structure, using the same decorator. I think having the valid and invalid data for a class (like Key) near each other has value: if the file becomes too big then lets e.g. start splitting Signed-derivates to another file (but I don't think we're there yet).

@MVrachev MVrachev self-assigned this Jun 22, 2021
@sechkova sechkova modified the milestones: weeks24-25, weeks26-27 Jun 23, 2021
@joshuagl joshuagl modified the milestones: weeks26-27, Sprint 4 Jul 7, 2021
@joshuagl joshuagl removed the backlog Issues to address with priority for current development goals label Jul 7, 2021
@jku jku closed this as completed in #1460 Jul 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants