Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specifying catala options in Clerk and backend testing #192

Closed
adelaett opened this issue Feb 8, 2022 · 3 comments
Closed

Specifying catala options in Clerk and backend testing #192

adelaett opened this issue Feb 8, 2022 · 3 comments
Labels
🏗️ build system Build system or Makefile ✨ enhancement New feature or request
Milestone

Comments

@adelaett
Copy link
Collaborator

adelaett commented Feb 8, 2022

Some features are missing inside clerk

  • The possibility to write tests cases corresponding to specific catala's compiler arguments. For instance --optimize or --avoid_exceptions modify the compilation results, hence failing the test. Adding other outputs formats (for instance .catala_en.avoid_exception.ml) could solve the problem, but other solution might be better.
  • To check if a compilation pass is correct, one way is to check whenever it type-check correctly inside ocaml. To do so, we need at least to import catala.runtime. One solution could be to generate dune files.
@adelaett adelaett added ✨ enhancement New feature or request 🏗️ build system Build system or Makefile labels Feb 8, 2022
@denismerigoux denismerigoux added this to the Catala v1.0 milestone Feb 16, 2022
@denismerigoux
Copy link
Contributor

Maybe @EmileRolley you could tackle this in #186?

@EmileRolley
Copy link
Collaborator

I think it will preferable to implement this in a separate PR, indeed, I will run out of time. Consequently, I prefer to finish #186 by providing the minimal features first.

@denismerigoux denismerigoux changed the title Improving clerk Specifying catala options in Clerk and backend testing Jul 8, 2022
@denismerigoux
Copy link
Contributor

denismerigoux commented Jul 8, 2022

For the specification of Catala options in Clerk, here is the solution we've chosen. Suppose you have a file A.catala_en that you want to test using several unit tests that act on one or more scope of A.catala_en, with different combinations of catala compiler options to use.

Inside A.catala_en, you can specify all those unit tests using a little test specification DSL that will be interpreted by Clerk:

```catala-test { id="with_optim" } 
catala Interpret -s A  --optimize
```

Using that declaration, Clerk called a first time with -r will create an expected output file whose name is the heading name (with a suffix if there are several tests inside one heading). Then it will run Catala with the specified options and fill the output file with the output of the command (stdout and stderr mixed).

Next time, calling Clerk without -r will parse again the test declaration in the file to test to recreate the Catala command to run, then it will run the Catala command and diff its output against the expected output file created earlier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏗️ build system Build system or Makefile ✨ enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants