Skip to content

Commit

Permalink
Generate unique items for arrays with the "uniqueItems" property
Browse files Browse the repository at this point in the history
This re-calculates the values of an array if the "uniqueItems" property
is set and a non-unique item has been generated. This has been to help
resolve flaky tests that expect a random array and occasionally fail due
to the generation of an array with collisions.

As there is a risk that this can produce an eternal loop I've put in a
check that if there are very number of attempts to generate the unique
value then it should abort. This does produce a risk of failure if there
is a low amount of variance in the allowed variables.

This is to resolve issues such as:

```
GovukSchemas::InvalidContentGenerated: An invalid content item was generated.

This probably means there's a bug in the generator that causes it to output
invalid values. Below you'll find the generated payload, the validation errors
and the schema that was used.

...

Validation errors:
--------------------------

[
  {
    "schema": "869a7d30-f94e-5044-84e7-dc3c01c8f1ba#",
    "fragment": "#/details/featured_attachments",
    "message": "The property '#/details/featured_attachments' contained duplicated array values in schema 869a7d30-f94e-5044-84e7-dc3c01c8f1ba#",
    "failed_attribute": "UniqueItems"
  },
  {
    "schema": "869a7d30-f94e-5044-84e7-dc3c01c8f1ba",
    "fragment": "#/details/featured_attachments",
    "message": "The property '#/details/featured_attachments' contained duplicated array values in schema 869a7d30-f94e-5044-84e7-dc3c01c8f1ba",
    "failed_attribute": "UniqueItems"
  }
]
```
  • Loading branch information
kevindew committed Mar 8, 2021
1 parent 8c2b10b commit 33180c9
Show file tree
Hide file tree
Showing 2 changed files with 56 additions and 2 deletions.
19 changes: 17 additions & 2 deletions lib/govuk_schemas/random_schema_generator.rb
Original file line number Diff line number Diff line change
Expand Up @@ -116,12 +116,27 @@ def generate_random_object(subschema)
def generate_random_array(props)
min = props["minItems"] || 0
max = props["maxItems"] || 10
unique = props["uniqueItems"] == true
num_items = @random.rand(min..max)
items = []
attempts = 0
max_attempts = num_items * 100

num_items.times.map do
until items.length == num_items
# sometimes arrays don't have `items` specified, not sure if this is a bug
generate_value(props["items"] || {})
new_value = generate_value(props["items"] || {})

if unique && items.include?(new_value)
attempts += 1
raise "Failed to create a unique array item after #{max_attempts} attempts" if attempts >= max_attempts
next
end

attempts = 0
items << new_value
end

items
end

def generate_random_string(props)
Expand Down
39 changes: 39 additions & 0 deletions spec/lib/random_schema_generator_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -110,5 +110,44 @@
expect(generator.payload["my_enum"]).to eq("a")
expect(generator.payload.keys).to include("my_field")
end

describe "unique arrays" do
it "handles arrays that require unique items" do
schema = {
"type" => "array",
"items" => { "type" => "string" },
"uniqueItems" => true,
"minItems" => 5,
"maxItems" => 5,
}

generator = GovukSchemas::RandomSchemaGenerator.new(schema: schema)

# These stubs are to ensure determinism in the random array value
# generation.
allow(generator).to receive(:generate_value).and_call_original
allow(generator)
.to receive(:generate_value)
.with({ "type" => "string" })
.and_return(*%w[a a b b c c d d e e])

expect(generator.payload).to match_array(%w[a b c d e])
end

it "raises an error for a situation where it can't generate a random array" do
schema = {
"type" => "array",
"items" => { "enum" => %w[a b] },
"uniqueItems" => true,
"minItems" => 3,
"maxItems" => 3,
}

generator = GovukSchemas::RandomSchemaGenerator.new(schema: schema)

expect { generator.payload }
.to raise_error "Failed to create a unique array item after 300 attempts"
end
end
end
end

0 comments on commit 33180c9

Please sign in to comment.