Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix stale schema #6286

Merged
merged 2 commits into from
Jan 3, 2025
Merged

Conversation

ulidtko
Copy link
Contributor

@ulidtko ulidtko commented Nov 23, 2024

Hi!

Two brief commits here, fixing issues in product-schema.json for validating the frontmatters.

See commit messages 👀

Copy link

welcome bot commented Nov 23, 2024

Thank you for opening this pull request 👍. If you are not familiar with the project, please check out our Contributing Guidelines and our Guiding Principles. Also take a look at our Hacking Guide if you intend to work on site internals.

@BiNZGi BiNZGi added the product-updates Product content updates that are not fixes nor releases updates label Dec 9, 2024
@@ -472,7 +488,7 @@
"title": "Latest release date",
"description": "The date of the latest release.",
"type": "string",
"format": "date-time"
"format": "date"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commit d28c0b2 👀

Comment on lines +74 to +87
"type": "object",
"required": ["methods"],
"properties": {
"cumulative": {
"title": "Mark auto-update as being cumulative (optional, default = false)",
"description": "When true, data won't be deleted before fetching new data.\nActivating cumulative updates is not recommended for most products, but could be useful for products that:\n - have a long history of releases that is long to fetch,\n - have a history of releases that is not available anymore.",
"type": "boolean"
},
"methods": {
"title": "Auto-update configs",
"description": "See https://github.com/endoflife-date/endoflife.date/wiki/Automation",
"type": "array",
"items": {
"$ref": "#/$defs/automethod"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commit 4961fb8 👀

repairs product-schema.json staleness introduced in 1312fab — the actual schema has changed [method1, method2]{"cumulative": bool, "methods": [method1, method2]}.

the large oneOf switch was factored out into $defs below.

@captn3m0
Copy link
Member

I haven't worked a lot with the schema, happy to merge once I confirm the validation matches existing products.

See https://json-schema.org/understanding-json-schema/reference/string#dates-and-times

Pretty much all products specify already YYYY-MM-DD a.k.a. json-schema format: 'date'.

Not 'date-time'.

Additionally, the releaseDate key is specified as #/$defs/boolOrDate, which besides the
bool variant, accepts json-schema string format: 'date'.
After commit 1312fab, the actual
schema has changed; but product-schema.json was left stale.

Fix that by factoring out the auto.methods enum into $defs, and
repairing the nesting structure in the schema.
@ulidtko ulidtko force-pushed the fix/product-schema.json branch from 4961fb8 to 3f6a83d Compare December 26, 2024 14:35
@ulidtko
Copy link
Contributor Author

ulidtko commented Dec 26, 2024

I followed the guidance in CONTRIBUTING.md:

## ✅ Validating your changes
If you're using an IDE like `vscode` or `vim` (or any other IDE that supports JSON schema validation),
you can use [this jsonschema](./product-schema.json) to validate the new product.

On CLI... it's quite fiddly, I tried a few stock validators, none seem to work well with yaml frontmatters embedded in markdown.

Perhaps a little script can be written...

Yup, here's one ✍️
#!/usr/bin/env ruby

require('yaml')
require('json_schemer') #-- gem install json_schemer

#-- load product schema
product_schema = JSONSchemer.schema(
  JSON.load File.new("product-schema.json"),
  format: true,
  formats: {
    'date' => JSONSchemer::Format::DATE
  }
)
unless product_schema.valid_schema?
    puts "Warning: schema itself is not valid"
end

#-- Monkey-patch workaround, for an issue with YAML dates
#-- https://github.com/davishmcclurg/json_schemer/issues/203
class UnparsedDateMonkeyPatch
  def strptime strscalar, fmt, calendar
    strscalar #-- return input untouched, don't parse nothing.
  end
end
module YAML
  class ClassLoader
    class Restricted
      def find klassname
        return UnparsedDateMonkeyPatch.new if klassname == 'Date'
        super
      end
    end
  end
end

#-- iterate md frontmatters, validate, output errors if any
Dir.glob('products/*.md').select{|f| File.file? f}.each{|productmd|
  frontmatter = YAML.load_file(productmd, permitted_classes: [Date])

  product_schema.validate(frontmatter).each {|validation_result|
    printf "%s: %s\n", productmd, validation_result['error']
  }
}

@captn3m0 do you want this in-repo? I hadn't committed, because wasn't sure how to declare dependency on json_schemer. It doesn't belong in the main Gemfile, does it?..


Then, using the script, we can see there's been ever-increasing divergence of schema VS the product files. That's to be expected, because there's no enforcement of this validation in CI.

My fixes here improve the number of products files that pass validation:

  • from 43/349 (306 with issues) on master
  • to 221/349 (128 with issues) on this branch, freshly rebased.

Copy link
Member

@marcwrobel marcwrobel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ulidtko for this contribution

@marcwrobel marcwrobel merged commit 0eee4e8 into endoflife-date:master Jan 3, 2025
5 checks passed
@ulidtko ulidtko deleted the fix/product-schema.json branch January 3, 2025 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
product-updates Product content updates that are not fixes nor releases updates
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants