exercise management utility

[ZapAnton]

In order to automate some exercise management routines (for instance generating new exercise, updating the test suite to the latest version from the problem-specifications or modifying the contents of the config.json file), an utility was developed for the Rust track. Would this track benefit much, if a similar utility was developed here?

[arcuru]

TL;DR - Maybe? But for the time-investment doc updates to explain step-by-step how to some of those things are probably a better use of your time.

Functionality to modify the config.json is definitely something that should be added centrally for all the tracks. If you're willing, I'd suggest trying to add whatever they've done to https://github.com/exercism/configlet. That would benefit everyone and help more than just us.

We don't auto-generate the test suites, so that part is not useful (I haven't thought about doing that at all, but if Rust is somehow doing it we might be able to...I'd be happy to learn if that's possible?)

We don't even have the versions of the test suites logged anywhere AFAIK, getting the test suite versions in comments in the test files would help with knowing when we needed to update.. exercism/exercism#4390 may come online that will help us update the test suites when they get changed, but it's not implemented yet.

[ZapAnton]

We don't auto-generate the test suites, so that part is not useful (I haven't thought about doing that at all, but if Rust is somehow doing it we might be able to...I'd be happy to learn if that's possible?)

If I am not mistaken every exercise on this track is designed as follows:

• README.md (generated by configlet)
• makefile (common across the whole track)
• src directory (rarely present)
• test directory, contains the test suite and Unity files

So what I thought about, when I proposed an auto-generation utility, is a program that creates an exercise directory, copies the common files (makefile and Unity), runs configlet to generate README and, finally, fetch the canonical-data.json and generate the test suite from it. Usually the test suite includes the Unity boilerplate and the number of test_case_canonical_description functions, so should not be that hard to generate for the utility. The auto-generated contents of the test functions could be a call to the canonical property function that is bound to the test case.

We don't even have the versions of the test suites logged anywhere AFAIK, getting the test suite versions in comments in the test files would help with knowing when we needed to update

I propose a more primitive design: Fetch the canonical-data.json and compare it with the existing test suite. If the test suite does not contain the test case funtion, that follows the test_case_canonical_description naming rule, generate this function with some marker (for instance //NEW comment). If the test suite does contain such a function, generate the copy of the function with a different marker (for instance //UPDATED comment). Then the maintainer will manually resolve which funtions they need to delete and which to leave.

[arcuru]

Oh I see now, you would mainly focus on generating the individual test cases and not necessarily the body of the test. That could be useful...and it could definitely help check on some of our test suite requirements...

So I'll say this, that would be pretty useful for us, though it would be more useful to build something as generic as you can to share among all the tracks. Maybe let each track extend a single tool with their specific needs, so it would be easier for us to just write a hook for the C track test specifics and share the framework with everyone else. Extending the Rust tool to support that perhaps?

A more generic tool with the least amount of track specific stuff would be a lot more broadly useful, but I understand if you don't want to take that direction.

My guess is that you would spend more time writing this tool than this track alone would collectively save with it in the near future. But if you want to do this go for it.

I would focus on the easy win stuff first. The exercise setup is easy to make with a bash script even, and starting to just validate some of the things in our test suites would be useful and get you started on the parser. For example, every test case we have is a non static function and has a single matching call to RUN_TEST inside of main (we've missed this before), and all other functions are static.

(Also by the way, the C++ track is close enough to this one that a tool like this would only require minor changes. In fact it would probably be a lot simpler.)