Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: sort when syncing by parent #973

Merged
merged 6 commits into from
Feb 15, 2024
Merged

Conversation

kellyjosephprice
Copy link
Contributor

@kellyjosephprice kellyjosephprice commented Feb 7, 2024

🚥 Resolves RM-8336

🧰 Changes

When syncing docs, it sorts them by their parents!

Users reported issues where if a parent document was in a directory, it could error out because it would get synced after one of its children.

🧬 QA & Testing

Not sure 😬.

@kellyjosephprice kellyjosephprice changed the base branch from next to tests/mock-git-cmds February 7, 2024 21:19
@kellyjosephprice kellyjosephprice changed the title feat/sort sync heirarchy feat: sort when syncing by parent Feb 7, 2024
@kellyjosephprice kellyjosephprice requested review from erunion and kanadgupta and removed request for erunion February 7, 2024 21:26
@kellyjosephprice kellyjosephprice marked this pull request as ready for review February 7, 2024 21:27
Copy link
Member

@kanadgupta kanadgupta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh wow this is a very sophisticated approach, love it! i have some feedback + questions below, happy to pair on any of this!

src/lib/syncDocsPath.ts Outdated Show resolved Hide resolved
src/lib/syncDocsPath.ts Outdated Show resolved Hide resolved
Comment on lines 138 to 142
const doc = frontMatter(fsSync.readFileSync(filePath, 'utf8'));
const slug = path.basename(filePath).replace(extensionRegexp, '');
const parentDocSlug = doc.data.parentDocSlug;

return { filePath, slug, parentDocSlug };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have a readDoc utility that should take care of this, mind using that instead? also it appears we're reading each file twice now, any way we could optimize that1?

Footnotes

  1. just flagging it so we're aware, but i don't think it's a real issue!


const dependencies = Object.values(filesBySlug).reduce(
(edges, obj) => {
if (obj.parentDocSlug) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i know we have users that sync only parts of their docs using rdme, i think we should probably also ensure that the parent file actually exists

(also let me know if i'm misunderstanding how this works!)

Suggested change
if (obj.parentDocSlug) {
if (obj.parentDocSlug && filesBySlug[obj.parentDocSlug]) {

@@ -127,6 +130,40 @@ async function pushDoc(
});
}

function sortFiles(files: string[], { allowedFileExtensions }: { allowedFileExtensions: string[] }): string[] {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function appears to use parentDocSlug, is there any way we add basic support for parentDoc? i know it uses object IDs so there's no way for us to build a graph, but at the very least we could make it so docs that have a parentDoc attribute get synced after ones that lack one?

@erunion erunion removed their request for review February 13, 2024 18:37
Copy link
Member

@kanadgupta kanadgupta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Comment on lines -33 to +36
const { content, data, hash, slug } = readDoc(filePath);
const { content, data, filePath, hash, slug } = fileData;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this refactor (i.e., decoupling the readDoc so we can read the contents prior to sorting) is very clever!

Base automatically changed from tests/mock-git-cmds to next February 15, 2024 00:38
@kellyjosephprice kellyjosephprice merged commit 9795840 into next Feb 15, 2024
8 checks passed
@kellyjosephprice kellyjosephprice deleted the feat/sort-sync-heirarchy branch February 15, 2024 00:56
kanadgupta pushed a commit that referenced this pull request Feb 15, 2024
# [9.0.0-next.11](v9.0.0-next.10...v9.0.0-next.11) (2024-02-15)

### Features

* sort when syncing by parent ([#973](#973)) ([9795840](9795840))

[skip ci]
kanadgupta pushed a commit that referenced this pull request Dec 6, 2024
# [9.0.0](v8.6.6...v9.0.0) (2024-12-06)

* feat!: deprecation notices for non-readme-refactored commands ([#1099](#1099)) ([732e32b](732e32b))
* feat!: drop node 18 ([#1085](#1085)) ([ebc2ac0](ebc2ac0))
* feat!: final v9 touchups ([#1106](#1106)) ([15447c5](15447c5)), closes [#1107](#1107)
* feat!: oclif (take 2) ([#1068](#1068)) ([5e05f93](5e05f93)), closes [#962](#962) [#1067](#1067) [/github.com/readmeio/rdme/blob/d01d76fe3c2e4a98b4f5c415be03e589fbe3b56e/.releaserc.yml#L30](https://github.com//github.com/readmeio/rdme/blob/d01d76fe3c2e4a98b4f5c415be03e589fbe3b56e/.releaserc.yml/issues/L30) [#1067](#1067)
* feat!: remove `validate`, deprecate/hide `open` ([#1072](#1072)) ([f1b46f6](f1b46f6))
* feat!: switch topic separator to space ([#1100](#1100)) ([13eb8ab](13eb8ab))

### Bug Fixes

* add better debugging for fetch failures ([a17f8da](a17f8da))
* add GITHUB_TOKEN ([e106e10](e106e10))
* attempt to use semantic-release/github instead of `gh` ([331d28b](331d28b))
* build `dist-gha/` files with every release ([f42392b](f42392b))
* bump semantic-release versions ([21efc66](21efc66))
* debug properly ([#1078](#1078)) ([5de0a4f](5de0a4f))
* **deps:** bump more deps ([#1042](#1042)) ([786a000](786a000))
* empty commit to trigger build ([51c1566](51c1566))
* empty commit to trigger build ([76efb7e](76efb7e))
* empty commit to trigger release ([3604f2a](3604f2a))
* fixed debug script ([#908](#908)) ([82c3541](82c3541))
* hide `ExperimentalWarning` ([#901](#901)) ([f9b5679](f9b5679))
* **npm:** tweak npm lifecycle scripts ([#1073](#1073)) ([35534db](35534db))
* oclif-friendly error logging ([#1080](#1080)) ([73c5f4c](73c5f4c))
* **openapi:** add spinner catch statement ([#961](#961)) ([4669b29](4669b29))
* **release:** track `dist-gha` assets properly ([b0934cb](b0934cb))
* **release:** track changes to commands doc directory ([78b6554](78b6554))
* swap out ts-node for tsx ([16f64b3](16f64b3))
* Update LICENSE for 2024 ([#963](#963)) ([23cf44b](23cf44b))
* use newest import attributes syntax ([#993](#993)) ([d76c0ae](d76c0ae)), closes [/github.com/nodejs/node/blob/main/doc/changelogs/CHANGELOG_V20.md#2023-11-22-version-20100](https://github.com//github.com/nodejs/node/blob/main/doc/changelogs/CHANGELOG_V20.md/issues/2023-11-22-version-20100)
* **versions/update:** various edge cases ([#965](#965)) ([580e307](580e307))

### chore

* type fixes for tests ([#903](#903)) ([d423b4c](d423b4c))

### Code Refactoring

* remove `oas`, `swagger`, `docs:edit` ([#902](#902)) ([9107d15](9107d15))

### Features

* add `@oclif/plugin-not-found`, other cleanups ([#1074](#1074)) ([6529b03](6529b03))
* add `title` flag to `openapi:convert` ([#1071](#1071)) ([1d71f3f](1d71f3f)), closes [#1068](#1068)
* convert to ESM (breaking change) ([#856](#856)) ([84b8571](84b8571)), closes [/www.stefanjudis.com/snippets/how-to-import-json-files-in-es-modules-node-js/#option-1](https://github.com//www.stefanjudis.com/snippets/how-to-import-json-files-in-es-modules-node-js//issues/option-1) [1#L228](https://github.com/1/issues/L228)
* **fetch:** use undici + `ProxyAgent`  ([#1000](#1000)) ([9da7132](9da7132)), closes [#999](#999)
* **openapi/convert:** support openapi in convert command ([#941](#941)) ([a33bbeb](a33bbeb))
* revamped `oclif`-powered docs ([#1081](#1081)) ([8a2833b](8a2833b))
* sort when syncing by parent ([#973](#973)) ([9795840](9795840))
* support `README_API_KEY` env var ([#950](#950)) ([bcf3f18](bcf3f18))
* support node 18 and up ([#900](#900)) ([a217904](a217904))
* **test:** moving unit tests over to vitest ([#857](#857)) ([36d561b](36d561b)), closes [#870](#870)
* use `actions/checkout@v4` everywhere ([#895](#895)) ([d30b71c](d30b71c))
* **versions:** flag parity with API, copy fixes ([#906](#906)) ([d424d9f](d424d9f))

### BREAKING CHANGES

* `rdme openapi` is deprecated and will be replaced in
`rdme@10` by a command with a simpler flag setup based on community
feedback.
* deprecates commands that are not supported in ReadMe
Refactored. For more information, please visit our migration guide:
https://github.com/readmeio/rdme/tree/v9/documentation/migration-guide.md

## 🧬 QA & Testing

Does the copy in these deprecation warnings make sense to you? Note that
the links will be broken for now since we haven't tagged a proper v9
release yet, but that will be fixed once this release is out!
* the topic separator (i.e., what separates a command
from its subcommand) has changed from a colon to a space by default. For
example, `rdme openapi:validate` is now `rdme openapi validate`. The
colon topic separator will continue to be supported so there is no need
to change any existing commands, but all documentation and help screens
will reflect the space topic separator.

## 🧬 QA & Testing

Do tests still pass?
* dropping support for Node.js v18. The minimum required
Node.js version is v20.10.0.
* `rdme validate` has been removed, use `rdme
openapi:validate` instead.
* `rdme open` is now deprecated

also updates our docs accordingly
* the `rdme` GitHub Actions is now a [the `node20` JavaScript action](https://docs.github.com/en/actions/sharing-automations/creating-actions/metadata-syntax-for-github-actions#runs-for-javascript-actions) rather than [a Docker container action](https://docs.github.com/en/actions/sharing-automations/creating-actions/metadata-syntax-for-github-actions#runs-for-docker-container-actions).
* `rdme` is now powered by [`oclif`](https://oclif.io/). The formatting and content of certain error messages and outputs may have changed. Please continue to only utilize exit codes to determine command successes/failures.

completed tasks:

- [x] a handful of tests are still failing, hoping that
oclif/test#652 gets merged in)
- [x] github actions dry runs are failing (but i got them working in
* **versions:** this flips the `isPublic` flag to `hidden`.
* removes several deprecated commands

* chore: knip cleanup

* chore: type fixes for tests
* removes several deprecated commands
* Node.js >= 18 required

[skip ci]
@kanadgupta
Copy link
Member

🎉 This PR is included in version 9.0.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants