Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bringing your track in line with the latest changes to Problem Specifications #916

Merged
merged 8 commits into from
Nov 15, 2020

Conversation

ErikSchierboom
Copy link
Member

We're in the process of re-opening the Problem Specifications repo, as discussed in this issue.
This PR adds .meta/tests.toml files for all exercises for which canonical data is defined in the Problem Specifications repo.
We'll now discuss why we're making this change.

Keeping track of implemented tests

While most of the changes to the Problem Specifications repo are specific to that repo, there is one track-specific change:

If a track implements an exercise for which test data exists, the exercise must contain a .meta/tests.toml file.
The goal of the tests.toml file is to keep track of which tests are implemented by the exercise.
Tests in this file are identified by their UUID and each test has a boolean value that indicates if it is implemented by that exercise.
A tests.toml file for a track's two-fer exercise looks like this:

[canonical-tests]
# no name given
"19709124-b82e-4e86-a722-9e5c5ebf3952" = true
# a name given
"3451eebd-123f-4256-b667-7b109affce32" = true
# another name given
"653611c6-be9f-4935-ab42-978e25fe9a10" = false

In this case, the track has chosen to implement two of the three available tests.
If a track uses a test generator to generate an exercise's test suite, it must use the contents of the tests.toml file to determine which tests to include in the generated test suite.

Tooling

To make it easy to keep the tests.toml up to date, tracks can use the canonical_data_syncer application.
This application is a small, standalone binary that will compare the tests specified in the tests.toml files against the tests that are defined in the exercise's canonical data.
It then interactively gives the maintainer the option to include or exclude test cases that are currently missing, updating the tests.toml file accordingly.
To use the canonical data syncer tool, tracks should copying the fetch-canonical_data_syncer and/or fetch-canonical_data_syncer.ps1 scripts into their repository.
Then, running either of these scripts will download the latest version of the tool to the track's bin directory.
The tool can be run using ./bin/canonical_data_syncer or .\bin\canonical_data_syncer.exe, depending on your operating system.

Changes

In this PR, we're adding meta/tests.toml files for all the exercises for which canonical data is defined.
We've initially marked all tests as included, which means that we'll assume that your track has implemented those tests.
If there isn't anything obviously wrong with the PR, I would suggest to merge this PR first, and then later on update the meta/tests.toml files according to your track's actual implementation.

@petertseng
Copy link
Member

todo: process these entries

$ ruby up-to-date/hs.rb | grep '<'                                  (10-11 22:05)
           palindrome-products: 1.1.0.4 < 1.2.0
                  phone-number: 1.6.1.8 < 1.7.0
                      list-ops: 2.4.0.8 < 2.4.1
                  grade-school: 1.0.0.4 < 1.0.1
                       anagram: 1.4.0.8 < 1.5.0
                  queen-attack: 2.2.0.7 < 2.3.0
                     allergies: 1.2.0.7 < 2.0.0
                 simple-cipher: 1.2.0.5 < 2.0.0
                           bob: 1.4.0.10 < 1.6.0

@petertseng
Copy link
Member

exercises for which no change is needed, and the reason why:

palindrome-products: we're using Nothing to signify are no palindromes. upstream 1.1.0 -> 1.2.0 exercism/problem-specifications#1430
phone number: we're using Nothing to signify invalid phone number. upstream 1.6.1 -> 1.7.0 exercism/problem-specifications#1423
list-ops: we already use a correct description. upstream 2.4.0 -> 2.4.1 exercism/problem-specifications#1611
grade-school: README change is already covered in #917. upstream 1.0.0 -> 1.0.1 exercism/problem-specifications#1624
simple-cipher: we weren't depending on the specific key in the JSON file. upstream 1.2.0 -> 2.0.0 exercism/problem-specifications#1462

exercises where a change is needed:
anagram: two new cases. upstream 1.4.0 -> 1.5.0 exercism/problem-specifications#1552
queen-attack: test case value changed. upstream 2.2.0 -> 2.3.0 exercism/problem-specifications#1504
allergies: test strategy changed. upstream 1.2.0 -> 2.0.0 exercism/problem-specifications#1494
bob: test case value changed. upstream 1.4.0 -> 1.5.0 exercism/problem-specifications#1631 1.5.0 -> 1.6.0 exercism/problem-specifications#1632

@petertseng
Copy link
Member

for tests that we intend to never add, we should leave them in the tests.toml but mark them as false.

for tests that we intend to add but haven't gotten around to yet, we should remove them from tests.toml so that tools that detect whether a test is present in tests.toml will properly detect that it isn't there and will ask us to explicitly include or exclude it

Copy link
Member

@petertseng petertseng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With these commits I have added (I suggest to squash when merging), all .meta/tests.toml files now correctly reflect the state of this track, so I can safely approve this.

For CI to pass, it is necessary for a maintainer other than myself to approve #917 , at which point this PR needs to be rebased on master.

@sshine
Copy link
Contributor

sshine commented Nov 14, 2020

I've approved and merged #917.

I've also rebased this PR onto master.

@petertseng petertseng force-pushed the pre-populate-data-for-exercises branch from 8dd3ef7 to 5a0fd58 Compare November 15, 2020 07:51
@petertseng
Copy link
Member

rebasing on #929

@petertseng petertseng merged commit 28aa243 into master Nov 15, 2020
@petertseng petertseng deleted the pre-populate-data-for-exercises branch November 15, 2020 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants