chore: Bugifx update all package repository fields

[Graham42]

Description

Looking at some gatsby packages on npm they are missing any link to the github repo source so it was difficult to find where the source code was.

This PR fixes all the repository fields for packages and add a script to be able to do this easily again in the future. Or if we want to make a sweeping change can update the script instead of manually updating each package.

Something I'm not sure about: I had to update the lerna config to include the themes directory. I think this should be ok but not sure what else this might affect.

Related Issues

Related: #12755

[Graham42]

bootstrap is failing because the check-versions script is failing

$ babel-node scripts/check-versions.js
gatsby-starter-blog-theme:
  Depends on "gatsby-theme-blog@^1.0.0"
  instead of "[email protected]".

gatsby-starter-notes-theme:
  Depends on "gatsby-theme-notes@^1.0.0"
  instead of "[email protected]".

gatsby-starter-theme:
  Depends on "gatsby-theme-blog@^1.0.0"
  instead of "[email protected]".

  Depends on "gatsby-theme-notes@^1.0.0"
  instead of "[email protected]".

It appears there are 1.0.0 versions of some packages published but that doesn't match the version in the source?

[pieh]

bootstrap is failing because the check-versions script is failing

Those are caused by changes you did to package.json -> workspaces and fact that we didn't update deps in themes/* - this is change that we will likely make, but it affect how themes and starters are handled, so we are not confident in making this change yet

[Graham42]

Ok, I've dropped the commit that updated the package json worskpaces field.

[wardpeet]

Thank you!

I'm wondering why the following syntax is not correct?

"repository": {
  "type": "git",
  "url": "https://github.com/gatsbyjs/gatsby/tree/master/packages/gatsby-codemods"
}

Official npm docs say it's okay https://docs.npmjs.com/files/package.json#repository

[Graham42]

Yes that would also be fine, I chose the style that most commonly already exists in this repo. I don't think it makes any significant difference.

[Graham42]

Made a tweak to the script to not insert repository as the last key in package.json which makes merge conflicts more likely, and rebased on the master branch.

[wardpeet]

npm has support for monorepos now so I would suggest moving to this syntax

"repository": {
  "type" : "git",
  "url" : "https://github.com/gatsbyjs/gatsby.git",
  "directory": "packages/gatsby"
}

Sorry for the back and forth

[Graham42]

I tried that syntax and it doesn't work for the npm page link to GitHub. Until it does, I would recommend the current format. See my comment here: ea1d7d1#diff-5f64c3c255be83cf700e83e922506fa3R22

[wardpeet]

Thank you! I didn't go through the code yet 🙊. PR looks great! I'm loving the node script so we can change all our packages at any time 🤗

[moonmeister]

So I've noticed that most of the change logs are broken. I think this is due to Lerna prefixing urls with the "repository" url. This means "tree/master/packages/..." get's added to every link to issues, prs, commits, etc...and breaks the url. I'm guessing the new format (#15477 (comment)) mentioned by @wardpeet would fix this, but I get it may have other drawbacks.

[moonmeister]

Okay, did some more reading, I now have some thoughts:

@Graham42 How does the monorepo syntax not work for the nom page link? (my assumption is you mean it doesn't link you to the sub-directory) The more I think about this the repository link should be the root...and not include the folder path. NPM makes it clear that the 'repository" field is not for human consumption but for computers.

The URL should be a publicly available (perhaps read-only) url that can be handed directly to a VCS program without any modification. It should not be a url to an html project page that you put in your browser. It’s for computers.

The "homepage" field already exists in the package.json to get the user into the sub-directory(maybe your script can make sure all packages have this field). I think the repository url should be the new syntax.

"repository": {
  "type" : "git",
  "url" : "https://github.com/gatsbyjs/gatsby.git",
  "directory": "packages/gatsby"
}

Let me know if you have any questions. The full docs are here: https://docs.npmjs.com/files/package.json

[Graham42]

Correct, I was meaning the sub-directory.

I missed the homepage field, that looks like a better fit, thanks! I'll update the script.

[Graham42]

Updated script to use the monorepo format, and to also update the homepage field.

After having to update this PR multiple times I've realize that the repo would easily drift after this fix so I've refactored the script a bit to work as a check that runs as part of the circle ci lint stage.

Not sure if this might also fix #12755 now?

[moonmeister]

Yes, it would close #12755

[wardpeet]

Let's merge and see what it does 😄

Thanks for fixing! 🏅

[gatsbot]

Holy buckets, @Graham42 — we just merged your PR to Gatsby! 💪💜

Gatsby is built by awesome people like you. Let us say “thanks” in two ways:

1. We’d like to send you some Gatsby swag. As a token of our appreciation, you can go to the Gatsby Swag Store and log in with your GitHub account to get a coupon code good for one free piece of swag. We’ve got Gatsby t-shirts, stickers, hats, scrunchies, and much more. (You can also unlock even more free swag with 5 contributions — wink wink nudge nudge.) See gatsby.dev/swag for details.
2. We just invited you to join the Gatsby organization on GitHub. This will add you to our team of maintainers. Accept the invite by visiting https://github.com/orgs/gatsbyjs/invitation. By joining the team, you’ll be able to label issues, review pull requests, and merge approved pull requests.

If there’s anything we can do to help, please don’t hesitate to reach out to us: tweet at @gatsbyjs and we’ll come a-runnin’.

Thanks again!