Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use JSON schemas for JSON CRAC import #1175

Merged
merged 70 commits into from
Nov 13, 2024
Merged

Use JSON schemas for JSON CRAC import #1175

merged 70 commits into from
Nov 13, 2024

Conversation

bqth29
Copy link
Collaborator

@bqth29 bqth29 commented Oct 18, 2024

Please check if the PR fulfills these requirements

  • The commit message follows our guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)

Does this PR already have an issue describing the problem?
Fixes #1165
Fixes #1171
Fixes #1074

What kind of change does this PR introduce?

What is the current behavior?

What is the new behavior (if this is a feature change)?

Does this PR introduce a breaking change or deprecate an API?

  • Yes
  • No

If yes, please check if the following requirements are fulfilled

  • The Breaking Change or Deprecated label has been added
  • The migration steps are described in the following section

What changes might users need to make in their application due to this PR? (migration steps)

Other information:
This PR is the first step of a broader work of implementing a thorough and coherent Creation Context for JSON CRAC as presented in #1083 and #1098.

When importing a JSON CRAC file, there are two possible types of errors that can occur:

  1. stuctural errors because of missing mandatory fields, unknown fields or incoherent fields with respect to a fixed version that must lead the importer to crash
  2. business errors because of data that make no sense in a business point of view even though their aspect seems legit on a structural point of view (ex: a curative FlowCNEC without a contingency)

The latter are the types of erros that must be added to a Creation Context. Thanks to the JSON schemas, the JSON CRAC deserializer is called only is we are sure that the input file matches an accepted pattern for a JSON CRAC file. Then, we can simply handle minor business errors in the deserializer and add them to the Creation Context.

This is better than what is currently done because a structural error in the provided JSON files leads the importer to crash even though the JSON importer asserted that the format was right. This prevents for instance users to define another type of JSON Crac.

bqth29 added 21 commits October 15, 2024 15:46
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
@bqth29 bqth29 added feature New feature or request cleaning This issue or pull request only concerns improving the overall state of the code experiment This subject is experimental and needs to be discussed thoroughly first labels Oct 18, 2024
@bqth29 bqth29 marked this pull request as ready for review October 18, 2024 14:56
@bqth29 bqth29 requested a review from pet-mit October 18, 2024 14:56
bqth29 and others added 15 commits November 5, 2024 16:18
data/crac/crac-io/crac-io-json/pom.xml Outdated Show resolved Hide resolved
@@ -47,26 +49,19 @@ public boolean exists(String filename, InputStream inputStream) {
return false;
}
try {
Map<Pair<Integer, Integer>, List<String>> validationErrorsPerVersion = new HashMap<>();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discussed with @phiedw
let's try reading the "version" field in the crac and then fetching the corresponding json schema. this has 3 advantages :

  • you won't need to fetch all schema files anymore
  • we do one validation per file
  • the error messages will be specific for the crac version indicated by the user

To read the crac version, either do it manually by parsing the json first before validating it, or create a new validation schema that only read the version, and call it before the other version-specific schema

pom.xml Outdated Show resolved Hide resolved
bqth29 and others added 4 commits November 7, 2024 16:27
@bqth29 bqth29 requested a review from pet-mit November 8, 2024 08:57
Signed-off-by: Thomas Bouquet <[email protected]>
Signed-off-by: Thomas Bouquet <[email protected]>
@bqth29 bqth29 requested a review from pet-mit November 13, 2024 13:39
@pet-mit pet-mit merged commit c2da6cd into main Nov 13, 2024
10 checks passed
@pet-mit pet-mit deleted the feature/json-crac-schema branch November 13, 2024 14:15
bqth29 added a commit that referenced this pull request Nov 13, 2024
bqth29 added a commit that referenced this pull request Nov 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cleaning This issue or pull request only concerns improving the overall state of the code experiment This subject is experimental and needs to be discussed thoroughly first feature New feature or request PR: waiting-for-review This PR is waiting to be reviewed
Projects
None yet
2 participants