Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Impliment new template based generator #1730

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
77a991b
Impliment basic prototype as proof of concept
meatball133 Nov 9, 2024
039911c
Add ci and more tests
meatball133 Nov 9, 2024
8e3afbe
Update gemfile.lock to include toml-rb
meatball133 Nov 9, 2024
d278534
Merge branch 'main' into add-new-generator
meatball133 Nov 9, 2024
5c27b41
Fix rubocop and fix ci
meatball133 Nov 9, 2024
b1f131f
Format files
meatball133 Nov 9, 2024
7004988
EOL for every line in text file
kotp Nov 9, 2024
067df32
Generator now executable and changes based on feedback
meatball133 Nov 9, 2024
653f629
Fix interpreter name
kotp Nov 9, 2024
4960f96
Merge branch 'main' into add-new-generator
meatball133 Nov 11, 2024
39bf79b
Add missing name key in ci file
meatball133 Nov 11, 2024
7fb52c5
Fix execution path of ci scripts
meatball133 Nov 11, 2024
e1c69f4
Remove Crystal image refernce and fixes to ci
meatball133 Nov 11, 2024
79b00d1
Bump rubocop version and add missing actions checkout
meatball133 Nov 11, 2024
f806548
Test adding bundle install
meatball133 Nov 11, 2024
bbad039
Test uppdating gemfile
meatball133 Nov 11, 2024
b6d2e9e
Change to using `bundle exec`
meatball133 Nov 11, 2024
98d267d
Make the generate script use the same rubocop config as the repo
meatball133 Nov 11, 2024
68d4671
Test rollback to rubocop 1.50
meatball133 Nov 11, 2024
76504b5
Update readme to reflect recent changes
meatball133 Nov 11, 2024
00af4d0
Split utils methods into its own module
meatball133 Nov 15, 2024
492b44b
Breakout helper method and exception class
kotp Nov 16, 2024
f40f4c4
Verify now creates a file in exercise directory to get same formattin…
meatball133 Nov 17, 2024
769dee3
Changes based on feedback and add more tasks to rakefile
meatball133 Dec 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/workflows/generator-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: GeneratorTests

on:
workflow_dispatch:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
test-generator-templates:
name: Check Generator Templates
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Set up Ruby
uses: ruby/setup-ruby@52753b7da854d5c07df37391a986c76ab4615999
with:
ruby-version: "3.3"
bundler-cache: true
- name: Verify templates
run: bundle exec ./bin/generate --verify
test-generator:
name: Test Generator
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
- name: Set up Ruby
uses: ruby/setup-ruby@52753b7da854d5c07df37391a986c76ab4615999
with:
ruby-version: "3.3"
bundler-cache: true
- name: Run tests
run: bundle exec rake test:generator
1 change: 1 addition & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ source 'https://rubygems.org'
gem 'base64'
gem 'minitest'
gem 'rake'
gem 'toml-rb', require: false
gem 'mocha', require: false
gem 'rubocop', '~> 1.50.0', require: false
gem 'rubocop-minitest', require: false
Expand Down
19 changes: 12 additions & 7 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,20 @@ GEM
specs:
ast (2.4.2)
base64 (0.2.0)
citrus (3.0.2)
docile (1.4.0)
json (2.7.2)
json (2.8.1)
minitest (5.22.3)
mocha (2.1.0)
ruby2_keywords (>= 0.0.5)
parallel (1.24.0)
parser (3.3.0.5)
parallel (1.26.3)
parser (3.3.6.0)
ast (~> 2.4.1)
racc
racc (1.7.3)
rainbow (3.1.1)
rake (13.2.1)
regexp_parser (2.9.0)
regexp_parser (2.9.2)
rexml (3.3.9)
rubocop (1.50.2)
json (~> 2.3)
Expand All @@ -27,8 +28,8 @@ GEM
rubocop-ast (>= 1.28.0, < 2.0)
ruby-progressbar (~> 1.7)
unicode-display_width (>= 2.4.0, < 3.0)
rubocop-ast (1.31.2)
parser (>= 3.3.0.4)
rubocop-ast (1.34.1)
parser (>= 3.3.1.0)
rubocop-minitest (0.34.5)
rubocop (>= 1.39, < 2.0)
rubocop-ast (>= 1.30.0, < 2.0)
Expand All @@ -42,7 +43,10 @@ GEM
simplecov_json_formatter (~> 0.1)
simplecov-html (0.12.3)
simplecov_json_formatter (0.1.4)
unicode-display_width (2.5.0)
toml-rb (3.0.1)
citrus (~> 3.0, > 3.0)
racc (~> 1.7)
unicode-display_width (2.6.0)

PLATFORMS
ruby
Expand All @@ -57,6 +61,7 @@ DEPENDENCIES
rubocop-minitest
rubocop-rake
simplecov
toml-rb

BUNDLED WITH
2.5.7
20 changes: 20 additions & 0 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,21 @@ task :rubocop do
system('rubocop --display-cop-names')
end

desc "Run generator for specefic exercise"
task :generate, [:exercise] do |_t, argumments|
system("./bin/generate --exercise #{argumments[:exercise]}")
end

desc "Run generator for all exercises"
task :generate_all do
system("./bin/generate --all")
end

desc "Verify templates for all exercises"
task :verify do
system("./bin/generate --verify")
end

namespace :test do
flags = ARGV.drop_while { |e| e != '--' }.drop(1).join(' ')

Expand All @@ -25,5 +40,10 @@ namespace :test do
task.pattern = 'test/**/*_test.rb'
end

Rake::TestTask.new :generator do |task|
task.options = flags
task.pattern = 'generatorv2/test/**/*_test.rb'
end

ExerciseTestTasks.new options: flags
end
58 changes: 58 additions & 0 deletions bin/generate
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#!/usr/bin/env ruby
require 'optparse'
require 'tempfile'
require_relative '../generatorv2/lib/generator'

# Helper methods
def exercises
Dir.entries('./exercises/practice')
.select { |file| File.directory? File.join('./exercises/practice', file) }
end

class VerificationError < StandardError
MESSAGE = 'The result generated for %<exercise>s, does not match the current file'

def initialize(message = MESSAGE)
super
end
end

# Parsing Code
parser = OptionParser.new

parser.on('-v', '--version', 'Print the version') do
puts '0.1.0'
end

parser.on('-h', '--help', 'Prints help') do
puts parser
end

parser.on('-a', '--all', 'Generate all exercises') do
exercises.each do |exercise|
if File.exist?("./exercises/practice/#{exercise}/.meta/test_template.erb")
Generator.new(exercise).generate
end
end
end

parser.on('--verify', 'Verify all exercises') do
exercises.each do |exercise|
if File.exist?("./exercises/practice/#{exercise}/.meta/test_template.erb")
current_code = File.read("./exercises/practice/#{exercise}/#{exercise}_test.rb")
f = File.new("./exercises/practice/#{exercise}/temp_test.rb", 'w+')
Generator.new(exercise).generate(f.path)
generated_code = f.read
File.delete(f.path)
fail VerificationError unless current_code == generated_code
end
rescue VerificationError => e
stderr.puts e.message % {exercise:}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
stderr.puts e.message % {exercise:}
STDERR.puts e.message % {exercise:}

My mistake, not sure that this is under test, and so I missed the case of this constant.

end
end

parser.on('-e', '--exercise EXERCISE', 'The exercise to generate') do |exercise|
Generator.new(exercise).generate
end

parser.parse!
13 changes: 13 additions & 0 deletions exercises/practice/acronym/.meta/test_template.erb
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
require 'minitest/autorun'
require_relative 'acronym'

class AcronymTest < Minitest::Test
<% json["cases"].each do |cases| %>
def test_<%= underscore(cases["description"]) %>
<%= status() %>
assert_equal '<%= cases["expected"] %>', <%= camel_case(json["exercise"]) %>.<%= underscore(cases["property"]) %>('<%= cases["input"]["phrase"] %>')
end
<% end %>
end


15 changes: 7 additions & 8 deletions exercises/practice/acronym/acronym_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,37 +4,36 @@
class AcronymTest < Minitest::Test
def test_basic
# skip
assert_equal "PNG", Acronym.abbreviate('Portable Network Graphics')
assert_equal 'PNG', Acronym.abbreviate('Portable Network Graphics')
end

def test_lowercase_words
skip
assert_equal "ROR", Acronym.abbreviate('Ruby on Rails')
assert_equal 'ROR', Acronym.abbreviate('Ruby on Rails')
end

def test_punctuation
skip
assert_equal "FIFO", Acronym.abbreviate('First In, First Out')
assert_equal 'FIFO', Acronym.abbreviate('First In, First Out')
end

def test_all_caps_word
skip
assert_equal "GIMP", Acronym.abbreviate('GNU Image Manipulation Program')
assert_equal 'GIMP', Acronym.abbreviate('GNU Image Manipulation Program')
end

def test_punctuation_without_whitespace
skip
assert_equal "CMOS", Acronym.abbreviate('Complementary metal-oxide semiconductor')
assert_equal 'CMOS', Acronym.abbreviate('Complementary metal-oxide semiconductor')
end

def test_very_long_abbreviation
skip
assert_equal "ROTFLSHTMDCOALM",
Acronym.abbreviate('Rolling On The Floor Laughing So Hard That My Dogs Came Over And Licked Me')
assert_equal 'ROTFLSHTMDCOALM', Acronym.abbreviate('Rolling On The Floor Laughing So Hard That My Dogs Came Over And Licked Me')
end

def test_consecutive_delimiters
skip
assert_equal "SIMUFTA", Acronym.abbreviate('Something - I made up from thin air')
assert_equal 'SIMUFTA', Acronym.abbreviate('Something - I made up from thin air')
end
end
134 changes: 134 additions & 0 deletions generatorv2/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Generator

Last Updated: 2024/11/9

The generator is a powerful tool that can be used to generate tests for exercises based on the canonical data.
The generator is written in Ruby and is located in the `bin` directory.

## How to use the generator

### Things to do before running the generator

meatball133 marked this conversation as resolved.
Show resolved Hide resolved
Run `bundle install` to install the required libraries.
Before running the generator you have to make sure a couple of files are in place.

1. `tests.toml` file

It is located under the `.meta` folder for each exercise.
The toml file is used to configure which exercises are generated and which are not.
Since the generator grabs all the data from the canonical data, so does this enable new tests that won't automatically be merged in.
Instead so does new tests have to be added to the toml file before they show up in the test file.

If there is a test that isn't needed or something that doesn't fit Ruby you can remove it from the toml file.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If there is a test that isn't needed or something that doesn't fit Ruby you can remove it from the toml file.
If there is a test that isn't needed or something that doesn't fit Ruby you can remove it from the configuration file.

In case we move from TOML to another type of configuration file, this reference will not need to change in the documentation.

By writing after the test name `include = false` and it will be skipped when generating the test file.

2. `config.json` file, located in the root of the track

The generator makes sure that the exercise is in the config.json so you need to add it there before running the generator.

#### Things to note

The script which grabs info from the toml file is quite sensitive, writing the toml file in an incorrect way can brick the generator.

Here are some examples of how you should **NOT** work with the toml file.

Make sure that the UUID is the only thing inside of `[UUID]`, if there is, for example, an extra space so would that break it.
Here is an example

```toml
# This would break it since it is an extra space between UUID and `]`
[1e22cceb-c5e4-4562-9afe-aef07ad1eaf4 ]
# This would break it since it is an extra space between UUID and `[`
[ 1e22cceb-c5e4-4562-9afe-aef07ad1eaf4]
```

The script won't care if you write `include = true` since if it sees the UUID it will always take it as long as `include = false` is not written.
The script will not work if anything is misspelled, although the part which gets `include = false` doesn't care if it gets an extra space or not.

**NOTE:**
You are also **NOT** allowed to write `include = false` more than once after each UUID.
Since that can lead to errors in the generator.

Bad way:

```toml
[1e22cceb-c5e4-4562-9afe-aef07ad1eaf4]
description = "basic"
include = false
include = false
```

Good way:

```toml
[1e22cceb-c5e4-4562-9afe-aef07ad1eaf4]
description = "basic"
include = false
```

### Template

The generator uses a template file to generate the test file.
The template is located under the `.meta` for each exercise.

This template has to be manually written for each exercise.
The goal although is to make it so that you only have to write the template once and then it will be able to be used to generate new tests.

The template file is written in [Embedded Ruby(ERB)][erb].
ERB enables you to write Ruby code inside of the template file.
It also means that the templates can be highly customizable since you can write any Ruby code you want.

When writing the template file, it is recommended to look at already existing template files to get a better understanding of how it works.
The template is getting a slightly modified version of the canonical data, so you can check out the [canonical data][canonical data] to see the data structure.
The modification is that the cases which are not included in the toml file will be removed from the data structure.

When writing the template so is it a special tool that can help with giving `# skip` and `skip` tags for tests.
You simply have to call the `status` method.
It will return either `# skip` or `skip` depending on if it is the first test case or not.

Here is an example:

```
<%= status()%>
<%= status()%>
<%= status()%>
```

result:

```
# skip
skip
skip
```

### The Test Generator

If all the earlier steps are done so can you run the generator.
To run the generator you need to have a working Ruby installation and installed all gems in the Gemfile.
The generator is located in the `bin` directory and is called `generator.rb`.

To run the generator so do you have to be in the root directory and run the following command:

```shell
bundle exec ./bin/generate -e <exercise_slug>
```

Where `<exercise_slug>` is the same name as the slug name which is located in the `config.json` file.

For more commands so can you run the following command:

```shell
bundle exec ./bin/generate --help
```

### Errors and warnings

The generator will give you errors and warnings if something is wrong.
That includes if the exercise is not in the `config.json` file, if the exercise is not in the toml file, or if the template file is missing.
Also if it has a problem getting the `canonical-data.json` file so will it give you an error.
The generator also uses a formatter which will give you errors if the generated file is not formatted correctly.
The file will still be generated even if formatter gives errors, therefore can you check the file and see what is wrong and fix it in the template.

[erb]: https://docs.ruby-lang.org/en/master/ERB.html
[canonical data]: https://github.com/exercism/problem-specifications
Loading