Expose product level information in the API #2062

marcwrobel · 2022-12-15T14:08:41Z

Currently no product-level information is available from the API. Urls such as https://endoflife.date/api/python.json only contains product releases.

We need an endpoint for exposing product level information.

marcwrobel · 2022-12-15T14:13:46Z

We have to decide first what the endpoint will be.

The most obvious solution would be to expose those informations from the existing /api/<product>.json. But if we do that it require a breaking change in this endpoint response schema (releases would have to be moved under a key such as releases).

We could also introduce a new endpoint without introducing a breaking change. This endpoint could be /api/<product>-info.json or /api/<product>/info.json.

I think the best this to do would be to reuse the existing /api/<product>.json despite the breaking change.

captn3m0 · 2022-12-15T15:01:45Z

I’d be okay with the breaking change as well. We should probably have an API Changelog Issue that users can subscribe to and get notified of any changes. We have explicitly kept it in Alpha to allow for such breaking changes, so let’s do it. On phone right now, but there’s a API Website Parity discussion that has some of this.

marcwrobel · 2022-12-16T08:44:01Z

What do you mean by an "API Website Parity discussion" ?

hugovk · 2022-12-16T10:03:31Z

This "API<>Website Parity" discussion: #759

hugovk · 2022-12-16T10:11:38Z

I'm fine with breaking changes, as it's explicitly in alpha.

It would be great to have a PR with a deploy preview PR kept open for some time first, so that I can update the https://github.com/hugovk/norwegianblue/ client to work with the new API.

Then I can also release at the same time as the PR is merged (or maybe release first to support both versions).

Do we have some idea of the usage of the API? Can we get a list of user agents from other clients, to try and give them advance warning?

captn3m0 · 2022-12-16T10:54:35Z

Netlify gave us route level Analytics information while we were paying for it, and the last report only had a single API endpoint mentioned: https://github.com/endoflife-date/endoflife.date/wiki/Netlify-Analytics-2022-01

It was also almost an year ago for the last report. I could probably get another report, but it won't tell us much beyond route-level data, no user-agents or IP addresses.

Maybe we can put a banner on top of the website that warns about the impending change in the API? Seems like an overkill for now, while we're still in Alpha though. Just pinning an issue and mentioning it in the README should be fine I think.

marcwrobel · 2022-12-16T12:58:42Z

It would be great to have a PR with a deploy preview PR kept open for some time first, so that I can update the https://github.com/hugovk/norwegianblue/ client to work with the new API.

I think we can have both API running side-by-side on https://endoflife.date/ by combining this issue with #2066.

endoflife.date API is just a script generating JSON files. So if we create a new script generating new API files in another directory (such as api/v1) and leave the old script like it is now, everything will be fine. The impacts of this are :

the current create-json-files.rb have to be duplicated. But this does not increase maintenant because we will not have to update it anymore.
Versioning API will make it much easier to communicate about in case of breaking changes. Communication could be done by adopting semver for the API and using GitHub releases to create changelogs.
With this approach we may be able to know for sure if the old API is still used using Netlify Analytics (I am not sure, must be confirmed).
It will increase the site build time, but I do not think it will be significant enough to be a blocker (I did not notice a build time increase in [Feature] Add support for ICS Files #755).
API documentation (https://endoflife.date/docs/api) may be an issue, but if it is let's say the documentation is only for the last version.

Overall this approach will make it easier for endoflife API client to migrate, because both version will be available at the same time on https://endoflife.date/. Note that by this change I do not mean that the v1 API must be marked stabled: it will remain in beta until #759 issues are resolved. And the old script will have to be deleted after some time (how much will have to be decided later).

I also think adding a banner is an overkill because it's still in beta (and usages of the API seems low, at least it was one year ago). But I like you idea of creating and endoflife page of endoflife API ;).

hugovk · 2022-12-16T15:08:33Z

Only 34 results for https://endoflife.date/api on GitHub:

https://cs.github.com/?scopeName=All+repos&scope=&q=https%3A%2F%2Fendoflife.date%2Fapi

marcwrobel · 2022-12-16T15:40:51Z

For those who don't yet have access to the new GitHub code search feature : https://github.com/search?q=https%3A%2F%2Fendoflife.date%2Fapi&type=code.

captn3m0 · 2022-12-17T07:36:40Z

Read through all of the links, and added the relevant ones here: https://github.com/endoflife-date/endoflife.date/wiki/Known-Users

I didn't add non-critical users (such as generating READMEs, or dotfiles, or one-time scripts etc).

marcwrobel · 2022-12-17T08:51:06Z

Still a work in progress, but I started something. Example: https://deploy-preview-2080--endoflife-date.netlify.app/api/v1/python.json.

marcwrobel · 2022-12-17T16:28:54Z

@captn3m0, do you know if /api/<product>/<release>.json files are used (or can you see it using Netlify anlytics) ? I cannot think of a scenario where it could be useful because all the cycles information is already in the /api/<product>.json endpoint.

Another problem I saw in the current API : some field have multiple types (such as lts which is either a date or a boolean). Many languages does not easily support that, and I think it makes the API more difficult to understand.

captn3m0 · 2022-12-17T17:26:35Z

We’d added it to make it easier for consumption, couldn’t find usage in the wild and I don’t think it will show up in Netlify Analytics either (it shows too 100 pages or so iirc). Okay with dropping it in the v1 release, will enable Netlify Analytics and try to get some numbers.

captn3m0 · 2022-12-19T08:36:59Z

As expected, Netlify Analytics are useless.

The API shows up a few times on 404 URLs. Reached out to netlify support for Log Drains functionality, since that might work better for us.

Those links are not used on the website, but will soon be used in the API (#2062).

Compared the the old v0 API : - v1 API is now generated using a Jekyll Generator (see https://jekyllrb.com/docs/plugins/generators/). - JSON files has been moved to the v1 subdirectory (#2066). This makes it easier to implement non-backward-compatible changes in API. - The all.json, that returned a simple array of product names in v0, now returns a JSON document in v1. The array of products can be found under the 'products' key. This enables us to add endoflife.date-level data (such as the number of products). - The "products" array in all.json now also contains JSON document instead of simple strings. This will make it easier in the future to expose new data (such as tags for products). - Product level information are now exposed in the API (#2062) at the "product" endpoint. Instead of a JSON array in v0, v1 returns a JSON document. The release cycles array can be found under the 'releases' key. - Cycles data now always contains all the release cycles key, even if they are null. - Individual cycles JSON files has been removed (we thought it was not used/useful enough to be kept).

Those links are not used on the website, but will soon be used in the API (#2062).

Those links are not used on the website, but will soon be used in the API (see #2062).

Compared the the old v0 API : - v1 API is now generated using a Jekyll Generator (see https://jekyllrb.com/docs/plugins/generators/). - JSON files has been moved to the v1 subdirectory (#2066). This makes it easier to implement non-backward-compatible changes in API. - The all.json, that returned a simple array of product names in v0, now returns a JSON document in v1. The array of products can be found under the 'products' key. This enables us to add endoflife.date-level data (such as the number of products). - The "products" array in all.json now also contains JSON document instead of simple strings. This will make it easier in the future to expose new data (such as tags for products). - Product level information are now exposed in the API (#2062) at the "product" endpoint. Instead of a JSON array in v0, v1 returns a JSON document. The release cycles array can be found under the 'releases' key. - Cycles data now always contains all the release cycles key, even if they are null. - Individual cycles JSON files has been removed (we thought it was not used/useful enough to be kept).

This is a major rework of the API with a lot of breaking changes. See CHANGELOG_API.md for more information. Note that we thought of disabling API generation in development (using JEKYLL_ENV like the Jekyll Feed plugin - see https://github.com/jekyll/jekyll-feed/blob/master/lib/jekyll-feed/generator.rb#L145), but it was finally reverted. It does not work well with Netlify preview, and generate production URL (i.e. https://endoflife.date URLs) in development which makes it difficult to use.

marcwrobel added enhancement New feature or request api Changes that relates to API labels Dec 15, 2022

marcwrobel mentioned this issue Dec 15, 2022

Display the last modified date of products pages #2061

Merged

captn3m0 mentioned this issue Dec 16, 2022

Product Harmonization #2036

Merged

marcwrobel self-assigned this Dec 17, 2022

marcwrobel linked a pull request Dec 17, 2022 that will close this issue

Endoflife api/v1 #2080

Open

marcwrobel added a commit that referenced this issue Dec 19, 2022

[Android] Fix links for EOLed releases (#39)

78aaa70

Those links are not used on the website, but will soon be used in the API (#2062).

marcwrobel mentioned this issue Dec 19, 2022

[Android] Fix links for EOLed releases #2098

Merged

captn3m0 mentioned this issue Dec 20, 2022

Project Roadmap #2108

Open

marcwrobel added a commit that referenced this issue Dec 26, 2022

[Android] Fix links for EOLed releases (#39)

074f158

Those links are not used on the website, but will soon be used in the API (#2062).

marcwrobel added a commit that referenced this issue Dec 26, 2022

[Android] Fix links for EOLed releases (#39, #2098)

56996f5

Those links are not used on the website, but will soon be used in the API (see #2062).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expose product level information in the API #2062

Expose product level information in the API #2062

marcwrobel commented Dec 15, 2022

marcwrobel commented Dec 15, 2022

captn3m0 commented Dec 15, 2022 via email

marcwrobel commented Dec 16, 2022

hugovk commented Dec 16, 2022

hugovk commented Dec 16, 2022

captn3m0 commented Dec 16, 2022

marcwrobel commented Dec 16, 2022 •

edited

Loading

hugovk commented Dec 16, 2022

marcwrobel commented Dec 16, 2022

captn3m0 commented Dec 17, 2022

marcwrobel commented Dec 17, 2022

marcwrobel commented Dec 17, 2022 •

edited

Loading

captn3m0 commented Dec 17, 2022 via email

captn3m0 commented Dec 19, 2022

Expose product level information in the API #2062

Expose product level information in the API #2062

Comments

marcwrobel commented Dec 15, 2022

marcwrobel commented Dec 15, 2022

captn3m0 commented Dec 15, 2022 via email

marcwrobel commented Dec 16, 2022

hugovk commented Dec 16, 2022

hugovk commented Dec 16, 2022

captn3m0 commented Dec 16, 2022

marcwrobel commented Dec 16, 2022 • edited Loading

hugovk commented Dec 16, 2022

marcwrobel commented Dec 16, 2022

captn3m0 commented Dec 17, 2022

marcwrobel commented Dec 17, 2022

marcwrobel commented Dec 17, 2022 • edited Loading

captn3m0 commented Dec 17, 2022 via email

captn3m0 commented Dec 19, 2022

marcwrobel commented Dec 16, 2022 •

edited

Loading

marcwrobel commented Dec 17, 2022 •

edited

Loading