Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Use lychee as a link checker. #1328

Conversation

ptondereau
Copy link
Contributor

@ptondereau ptondereau commented Sep 21, 2022

@ptondereau ptondereau force-pushed the chore/change-link-checker-xtask branch from 43378c4 to 523523a Compare September 22, 2022 09:36
@ptondereau
Copy link
Contributor Author

@EverlastingBugstopper lychee seems to find less links than the previous tool (see Lint action).
I'll check later

@ptondereau ptondereau force-pushed the chore/change-link-checker-xtask branch 2 times, most recently from 09872c7 to 3051361 Compare September 29, 2022 08:46
@ptondereau
Copy link
Contributor Author

@EverlastingBugstopper found out that some links are deduplicated and that's why there are fewer links to check.

Before:

https://docs.netlify.com/routing/redirects/
https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
https://apollographql.com/docs/rover
./ARCHITECTURE.md
https://github.com/apollographql/rover/issues/new/choose
https://docs.rs/clap/latest/clap/
https://docs.rs/clap
https://docs.rs/graphql_client/latest/graphql_client/
https://www.apollographql.com
https://www.rust-lang.org/
https://www.rust-lang.org/tools/install
https://doc.rust-lang.org/cargo/index.html
https://doc.rust-lang.org/stable/rustc/platform-support.html
https://code.visualstudio.com/
https://rust-analyzer.github.io/manual.html
https://rust-analyzer.github.io/manual.html#installation
https://github.com/apollographql/rover/issues
https://github.com/apollographql/rover/issues/new/choose
https://github.com/apollographql/rover/blob/main/CHANGELOG.md
https://github.com/apollographql/gatsby-theme-apollo/tree/master/packages/gatsby-theme-apollo-docs
http://localhost:8000
https://github.com/apollographql/gatsby-theme-apollo/tree/master/packages/gatsby-theme-apollo-docs#sidebarcategories
https://github.com/apollographql/gatsby-theme-apollo/tree/master/packages/gatsby-theme-apollo-docs#creating-pages
https://github.com/apollographql/rover/blob/main/ARCHITECTURE.md
https://github.com/apollographql/.github/blob/main/CODE_OF_CONDUCT.md
https://app.circleci.com/pipelines/github/apollographql/rover?branch=main
https://github.com/apollographql/rover/releases/latest
https://go.apollo.dev/r/docs
https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
https://github.com/apollographql/rover/tree/main/crates/houston
https://github.com/apollographql/rover/tree/main/crates/robot-panic
https://github.com/rust-cli/human-panic
https://github.com/apollographql/rover/tree/main/crates/rover-client
https://github.com/apollographql/rover/tree/main/crates/sputnik
https://github.com/apollographql/rover/tree/main/crates/timber
https://curl.se/download.html
https://github.com/apollographql/rover/issues/537
https://docs.npmjs.com/cli/v6/using-npm/scripts
https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally
https://github.com/apollographql/rover/releases
https://github.com/rust-lang/cargo
https://go.apollo.dev/r/contributing
https://circleci.com/gh/apollographql/rover.svg?style=svg
https://shields.io/github/downloads/apollographql/rover/total.svg
https://github.com/apollographql/rover/milestones
./Cargo.toml
./docs/source/getting-started.md
./docs/source/ci-cd.md
./README.md#command-line-options
https://github.com/apollographql/rover/releases
./.circleci/config.yml
https://stackoverflow.com/questions/5480258/how-to-delete-a-remote-tag
https://github.com/matklad/cargo-xtask
../.github/workflows
https://github.com/apollographql/supergraph-demo
https://docs.volta.sh/guide/getting-started
../RELEASE_CHECKLIST.md

After:

https://github.com/apollographql/rover/issues/new/choose
https://github.com/apollographql/rover/issues
https://www.rust-lang.org/tools/install
https://www.rust-lang.org/
https://www.apollographql.com/
https://doc.rust-lang.org/stable/rustc/platform-support.html
https://code.visualstudio.com/
https://doc.rust-lang.org/cargo/index.html
https://rust-analyzer.github.io/manual.html
https://github.com/apollographql/rover/blob/main/CHANGELOG.md
https://www.rust-lang.org/tools/install
https://github.com/apollographql/gatsby-theme-apollo/tree/master/packages/gatsby-theme-apollo-docs#sidebarcategories
https://github.com/apollographql/rover/blob/main/ARCHITECTURE.md
https://github.com/apollographql/gatsby-theme-apollo/tree/master/packages/gatsby-theme-apollo-docs
https://rust-analyzer.github.io/manual.html#installation
https://github.com/apollographql/.github/blob/main/CODE_OF_CONDUCT.md
https://github.com/apollographql/gatsby-theme-apollo/tree/master/packages/gatsby-theme-apollo-docs#creating-pages
https://docs.netlify.com/routing/redirects/
https://github.com/apollographql/rover/issues/new/choose
https://docs.rs/clap/latest/clap/
https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
https://docs.rs/clap
https://docs.rs/graphql_client/latest/graphql_client/
https://apollographql.com/docs/rover
file:///home/circleci/project/ARCHITECTURE.md
https://github.com/apollographql/rover/releases
https://github.com/rust-cli/human-panic
https://github.com/apollographql/rover/releases/latest
https://shields.io/github/downloads/apollographql/rover/total.svg
https://github.com/apollographql/rover/tree/main/crates/sputnik
https://github.com/rust-lang/cargo
https://circleci.com/gh/apollographql/rover.svg?style=svg
https://github.com/apollographql/rover/tree/main/crates/houston
https://go.apollo.dev/r/docs
https://docs.npmjs.com/cli/v6/using-npm/scripts
https://go.apollo.dev/r/contributing
https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally
https://app.circleci.com/pipelines/github/apollographql/rover?branch=main
https://curl.se/download.html
https://github.com/apollographql/rover/tree/main/crates/timber
https://github.com/apollographql/rover/tree/main/crates/robot-panic
https://github.com/apollographql/rover/issues/537
https://github.com/apollographql/rover/tree/main/crates/rover-client
https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
https://github.com/apollographql/rover/releases
file:///home/circleci/project/README.md
https://stackoverflow.com/questions/5480258/how-to-delete-a-remote-tag
file:///home/circleci/project/docs/source/getting-started.md
file:///home/circleci/project/Cargo.toml
file:///home/circleci/project/docs/source/ci-cd.md
https://github.com/apollographql/rover/milestones
file:///home/circleci/project/.circleci/config.yml
https://github.com/apollographql/supergraph-demo
file:///home/circleci/project/RELEASE_CHECKLIST.md
https://github.com/matklad/cargo-xtask
file:///home/circleci/project/.github/workflows
https://docs.volta.sh/guide/getting-started

@ptondereau ptondereau changed the title WIP - chore: Use lychee as a link checker. chore: Use lychee as a link checker. Sep 29, 2022
@ptondereau ptondereau marked this pull request as ready for review September 29, 2022 09:13
Copy link
Contributor

@EverlastingBugstopper EverlastingBugstopper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking really good! would you mind also removing all of the existing code for the link checker? it should be the package.json with markdown-link-checker, in addition to the mlc_config.json file.

xtask/src/tools/lychee.rs Outdated Show resolved Hide resolved
xtask/src/tools/lychee.rs Show resolved Hide resolved
@ptondereau
Copy link
Contributor Author

ptondereau commented Sep 29, 2022

looking really good! would you mind also removing all of the existing code for the link checker? it should be the package.json with markdown-link-checker, in addition to the mlc_config.json file.

Yes, I've forgotten to remove mlc_config.json file, but markdown-link-checker is fetched on-the-fly through npm exec. Removing it from the npm runner is enough.

@ptondereau ptondereau force-pushed the chore/change-link-checker-xtask branch from 22f31de to ca9499d Compare September 29, 2022 19:46
@EverlastingBugstopper
Copy link
Contributor

OK and one more thing. Can we collect all failures into a Vec and display them at the bottom, grouped up together?

@ptondereau ptondereau force-pushed the chore/change-link-checker-xtask branch from f6371c5 to 87d6f29 Compare September 29, 2022 20:29
@ptondereau
Copy link
Contributor Author

ptondereau commented Sep 29, 2022

OK and one more thing. Can we collect all failures into a Vec and display them at the bottom, grouped up together?

It was on my final_V2_real_final todo list haha

I have to mention that due to the openssl-sys crate, I have to require explicitly OpenSSL for Windows distribution:
I've just added VCPKG to the installation process on CI for Windows only. It slows down the CI workflow but I don't know how far I can disable a transitive dependency in Cargo

@EverlastingBugstopper
Copy link
Contributor

hmmm... i wonder if that might have been a transitive CI error? can you try removing it and pushing once more? it may have just been a bit flaky. if it still fails we can keep it around for sure 👍🏼

Cargo.toml Outdated Show resolved Hide resolved
@EverlastingBugstopper
Copy link
Contributor

EverlastingBugstopper commented Sep 30, 2022

boo! hhooonesstlly look. we don't even run lints in windows in CI. want to just nest the link checker in a #[cfg(unix)] block and call it a day? we don't really have windows contributors either afaik.

you can then add lychee-lib as an optional dependency that only comes in when the OS isn't windows i think, something like:

[target.'cfg(unix)'.dependencies]
lychee-lib = "x.x"

more docs on this here: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#platform-specific-dependencies

you can then have two separate functions, one for #[cfg(unix)] and one for #[cfg(not(unix))] and one will pull in the lychee dep and one will just print "skipping the link checker"

@ptondereau ptondereau force-pushed the chore/change-link-checker-xtask branch 3 times, most recently from f5781b4 to 15055c3 Compare October 1, 2022 12:36
@ptondereau ptondereau force-pushed the chore/change-link-checker-xtask branch from 15055c3 to 27bcd0a Compare October 1, 2022 12:42
@ptondereau
Copy link
Contributor Author

boo! hhooonesstlly look. we don't even run lints in windows in CI. want to just nest the link checker in a #[cfg(unix)] block and call it a day? we don't really have windows contributors either afaik.

you can then add lychee-lib as an optional dependency that only comes in when the OS isn't windows i think, something like:


[target.'cfg(unix)'.dependencies]

lychee-lib = "x.x"

more docs on this here: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#platform-specific-dependencies

you can then have two separate functions, one for #[cfg(unix)] and one for #[cfg(not(unix))] and one will pull in the lychee dep and one will just print "skipping the link checker"

I decided to exclude only Windows and kept Unix + macOS. Should I update the contribution documentation too?

@EverlastingBugstopper
Copy link
Contributor

nah - i don't think so! this works great 👍👍

@ptondereau ptondereau requested review from EverlastingBugstopper and removed request for lrlna October 3, 2022 15:32
@ptondereau
Copy link
Contributor Author

@EverlastingBugstopper I think CircleCI's last build crashed. Do you mind rerunning, please?

@EverlastingBugstopper
Copy link
Contributor

CI passing, merging now 😄

@EverlastingBugstopper EverlastingBugstopper merged commit 803d45f into apollographql:main Oct 4, 2022
@ptondereau ptondereau deleted the chore/change-link-checker-xtask branch October 5, 2022 05:47
EverlastingBugstopper added a commit that referenced this pull request Nov 14, 2022
# [0.10.0] - 2022-11-10

> Important: 1 potentially breaking change below, indicated by **❗
BREAKING ❗**

## ❗ BREAKING ❗

- **Fix implementation of `--header` argument - @EverlastingBugstopper,
#1369 fixes #1365**

This change tightens up usage of the `--header` argument used for
`introspect` commands by disallowing previously valid (but undocumented)
usage like this: `--header "Header-1: value" "Header-2: value"`. After
this change, you _must_ conform to what we have in the documentation,
which indicates separate instances of the `--header` argument for each
header, like so: `--header "Header-1: value" --header "Header-2:
value"`.

## 🚀 Features

- **Provide prebuilt binaries for ARM devices - @EverlastingBugstopper,
#1356 fixes #582**

As of this release, [`rover.apollo.dev`](https://rover.apollo.dev)
delivers prebuilt binaries from our GitHub release for ARM devices. Most
notably this means that Docker on M1 devices should work out of the box.
You should be able to replace any custom builds in your tooling pipeline
with a call to the [official curl
installer](https://www.apollographql.com/docs/rover/getting-started/#linux--macos-installer).

- **Report downstream check task results - @sachindshinde, #1385**

When running `rover subgraph check` commands, if the proposed schema
would cause downstream failures (i.e. with contracts), those failures
are now reported in the check response.

- **Faster `rover supergraph compose` - @EverlastingBugstopper, #1392
fixes #992**

Rover now resolves all subgraph schemas in parallel when running `rover
supergraph compose` on a `supergraph.yaml` file. This should improve the
speed to compose large supergraphs significantly. This change also
drastically improves error handling by reporting _all_ issues with
resolving subgraph schemas (and informing you which schema(s) failed to
resolve) rather than exiting on the first failed schema resolution.

- **Add `--polling-interval` to `rover dev` - @patrick91, #1377 fixes
#1221**

You can now set `--polling-interval` when running `rover dev` to change
the frequency of introspection poll requests for subgraphs that don't
provide the schema from the file system with the `--schema` argument.

- **Adds `--skip-update-check` to skip the once-per-day update check -
@Tsing, #1396 fixes #1394**

Once per day, Rover checks if there is a new version available for
update and notifies the user if there is. There is now a flag you can
pass to disable this check: `--skip-update-check`.

- **Respect the `NO_COLOR` environment variable - @chnn, #1360**

`rover` will not use color in any output when executed with the
`NO_COLOR` environment variable set to `true`.

## 🛠 Maintenance

- **Updates from clap v3 to clap v4 - @EverlatingBugstopper, #1404 fixes
#1400**

This release updated the command line argument parsing library to major
version 4. There should be no noticeable compatibility issues with this
update, only lighter binaries. The look and feel of the main `rover
--help` output has changed to a neutral color palette along with this
change.

- **Updates Rust to 1.65.0 - @EverlastingBugstopper, #1399**

- **Updates node.js to v18 - @renovate, #1389**

- **Updates node dev-dependencies - @renovate, #1204 and zs#1398**

- **Remove dependency on the `saucer` crate - @EverlastingBugstopper,
#1402**

- **Updates `introspector-gadget` to 0.2.0 - @EverlastingBugstopper,
#1386**

- **Only cache dependencies in CI, not whole `/target` -
@EverlastingBugstopper, #1387**

- **Use `engine@main` instead of `engine@current` to fetch the API
schema - @EverlastingBugstopper, #1368**

- **Use `lychee` as a link checker instead of npm - @ptondereau, #1328
fixes #1306**

We now use a Rust-based link checker to check the links in the Rover
repository instead of a node-based link checker (that was much more
flaky).

- **Describe latest federation versions in
`./latest_plugin_versions.json` - @EverlastingBugstopper, #1363**

When you run `rover supergraph compose`, the latest version of
composition is automatically downloaded to your machine, these latest
version numbers are now stored in `./latest_plugin_versions.json` in the
Rover repo.

- **Rename `apollo-` headers to `apollographql-` headers - @jsegaran,
#1411**

- **Update npm to v9 - @renovate, #1412**

## 📚 Documentation

- **Update studio algolia key to graphos - @trevorblades, #1384**

- **Fix some broken links - @StephenBarlow, #1376**

- **Fix a typo in the migration guide instructing the use of `check`
instead of `publish` - @EverlastingBugstopper, #1364 fixes #1361**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants