Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show individual doc differences for rendered documentation preview on PRs #2647

Merged
merged 28 commits into from
Aug 19, 2023

Conversation

AetherUnbound
Copy link
Collaborator

@AetherUnbound AetherUnbound commented Jul 14, 2023

Fixes

Fixes #2646 by @AetherUnbound

Description

This PR adds some new logic to the emit docs step to determine which files have been added or changed, and notes them in the docs preview. The easiest way I found to accomplish this was with the diff CLI tool, but the output of that tool is not necessarily machine-readable and thus more complicated than a series of bash commands inside a GitHub Actions' run block. I instead put all the logic inside a python file which is called by the workflow. This emits a comment (preserved for posterity) like the one below:

image

I knowingly did not add tests for this script - we don't yet have tests (but want to add them as part of #1916), and setting up testing for the automation scripts would be its own bolus of work. I didn't want to include that in this PR, so we can tackle that as part of #1916 down the line.

Testing Instructions

Check the emit docs comment!

Checklist

  • My pull request has a descriptive title (not a vague title likeUpdate index.md).
  • My pull request targets the default branch of the repository (main) or a parent feature branch.
  • My commit messages follow best practices.
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.
  • I ran the DAG documentation generator (if applicable).

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@github-actions github-actions bot added 🧱 stack: documentation Related to Sphinx documentation 🧱 stack: mgmt Related to repo management and automations labels Jul 14, 2023
@openverse-bot openverse-bot added 🟩 priority: low Low priority and doesn't need to be rushed 🌟 goal: addition Addition of new feature 🤖 aspect: dx Concerns developers' experience with the codebase labels Jul 14, 2023
@github-actions
Copy link

github-actions bot commented Jul 14, 2023

Full-stack documentation: https://docs.openverse.org/_preview/2647

Please note that GitHub pages takes a little time to deploy newly pushed code, if the links above don't work or you see old versions, wait 5 minutes and try again.

You can check the GitHub pages deployment action list to see the current status of the deployments.

Changed files 🔄:

@AetherUnbound AetherUnbound force-pushed the feature/individual-doc-differences branch from 0b41ead to d969c79 Compare July 15, 2023 00:02
@AetherUnbound AetherUnbound force-pushed the feature/individual-doc-differences branch 3 times, most recently from 47998bb to f77076d Compare July 28, 2023 23:49
@AetherUnbound AetherUnbound marked this pull request as ready for review July 29, 2023 00:07
@AetherUnbound AetherUnbound requested a review from a team as a code owner July 29, 2023 00:07
Copy link
Contributor

@obulat obulat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work! A lot of tricky edge-cases, too :) This will review the friction from reviewing the docs changes.

I wonder if we could add a change to a storybook story here to see if the link is generated. I've added a patch that updates the story for VLicenseElements, removing the deprecated default parameters and replacing them with args.

Remove_deprecated_defaults.patch

"CNAME",
"*.pickle",
"*.doctree",
"*.inv",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"*.inv",
"*.inv",
"*.txt",
"*.map",

These two also need to be excluded based on the comment to this PR. I'm not so sure about the txt, though, would there ever be txt files added that we would like to review?

Copy link
Collaborator

@sarayourfriend sarayourfriend Aug 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be easier to only include relevant files (is it just HTML?), instead of playing whack-a-mole with file types?

https://stackoverflow.com/a/10132592

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sarayourfriend I've tried a few different attempts based off this approach (example) but for some reason diff is still comparing files it should be excluding 😕 it's not ideal, but in the interest of not spending hours on this I'll add the additional exclusions and keep the current approach.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've also added some handling in 8e5ff36 to ensure we only display valid HTML pages in the diff.

Copy link
Member

@dhruvkb dhruvkb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition! Maybe in the future we can ship this as a standalone action, I imagine it could be useful for many other projects.

@zackkrida
Copy link
Member

@dhruvkb to your point, there seem to be a few existing actions which allow for listing changed files, for example: https://github.com/marketplace/actions/changed-files

@AetherUnbound's custom approach is still better, I reckon, since it does the link formatting we need.

@sarayourfriend
Copy link
Collaborator

sarayourfriend commented Aug 7, 2023

The action we currently use to determine where changes have happened in the repo can also provide the list of changed files: https://github.com/dorny/paths-filter#usage

If we add the list_files option, we could output that from the get-changes action, then we could get the files from outputs.documentation_files. Probably easier than parsing shell output. Update: Actually, I misunderstood how this PR worked before. It is getting a list of the files changed on the gh-pages branch, not by the PR itself. In other words, we need to get a diff of the output, not the input, I guess because something could generate pages that don't correspond 1-to-1 with a markdown file. dorny/paths-filter could potentially still be used to identify the changed output, just not with our get-changes action. In all likelihood not worth it because the diff implementation in this PR is ultimately fine anyway.

@AetherUnbound AetherUnbound force-pushed the feature/individual-doc-differences branch from 2b515d2 to ed74c93 Compare August 18, 2023 18:09
@AetherUnbound AetherUnbound requested a review from a team as a code owner August 18, 2023 18:09
@github-actions github-actions bot added the 🧱 stack: frontend Related to the Nuxt frontend label Aug 18, 2023
@github-actions
Copy link

github-actions bot commented Aug 18, 2023

Size Change: 0 B

Total Size: 877 kB

ℹ️ View Unchanged
Filename Size
./frontend/.nuxt/dist/client/238.js 272 B
./frontend/.nuxt/dist/client/238.modern.js 277 B
./frontend/.nuxt/dist/client/239.js 1.85 kB
./frontend/.nuxt/dist/client/app.js 122 kB
./frontend/.nuxt/dist/client/app.modern.js 113 kB
./frontend/.nuxt/dist/client/commons/app.js 90.5 kB
./frontend/.nuxt/dist/client/commons/app.modern.js 81.1 kB
./frontend/.nuxt/dist/client/commons/components/v-external-search-form/components/v-external-source-list/components/v-no-results/f323c204.js 5.18 kB
./frontend/.nuxt/dist/client/commons/components/v-external-search-form/components/v-external-source-list/components/v-no-results/f323c204.modern.js 5.62 kB
./frontend/.nuxt/dist/client/components/loading-icon.js 728 B
./frontend/.nuxt/dist/client/components/loading-icon.modern.js 734 B
./frontend/.nuxt/dist/client/components/table-sort-icon.js 515 B
./frontend/.nuxt/dist/client/components/table-sort-icon.modern.js 519 B
./frontend/.nuxt/dist/client/components/v-all-results-grid.js 6.62 kB
./frontend/.nuxt/dist/client/components/v-all-results-grid.modern.js 6.46 kB
./frontend/.nuxt/dist/client/components/v-audio-collection.js 4.38 kB
./frontend/.nuxt/dist/client/components/v-audio-collection.modern.js 4.26 kB
./frontend/.nuxt/dist/client/components/v-audio-details.js 1.79 kB
./frontend/.nuxt/dist/client/components/v-audio-details.modern.js 1.77 kB
./frontend/.nuxt/dist/client/components/v-audio-list.js 1.32 kB
./frontend/.nuxt/dist/client/components/v-audio-list.modern.js 1.31 kB
./frontend/.nuxt/dist/client/components/v-audio-result.js 1 kB
./frontend/.nuxt/dist/client/components/v-audio-result.modern.js 995 B
./frontend/.nuxt/dist/client/components/v-audio-track-skeleton.js 958 B
./frontend/.nuxt/dist/client/components/v-audio-track-skeleton.modern.js 961 B
./frontend/.nuxt/dist/client/components/v-audio-track.js 5.67 kB
./frontend/.nuxt/dist/client/components/v-audio-track.modern.js 5.64 kB
./frontend/.nuxt/dist/client/components/v-back-to-search-results-link.js 634 B
./frontend/.nuxt/dist/client/components/v-back-to-search-results-link.modern.js 640 B
./frontend/.nuxt/dist/client/components/v-bone.js 632 B
./frontend/.nuxt/dist/client/components/v-bone.modern.js 636 B
./frontend/.nuxt/dist/client/components/v-box-layout.js 1.33 kB
./frontend/.nuxt/dist/client/components/v-box-layout.modern.js 1.33 kB
./frontend/.nuxt/dist/client/components/v-content-link.js 1.06 kB
./frontend/.nuxt/dist/client/components/v-content-link.modern.js 1.06 kB
./frontend/.nuxt/dist/client/components/v-content-page.js 531 B
./frontend/.nuxt/dist/client/components/v-content-page.modern.js 535 B
./frontend/.nuxt/dist/client/components/v-content-report-button.js 492 B
./frontend/.nuxt/dist/client/components/v-content-report-button.modern.js 497 B
./frontend/.nuxt/dist/client/components/v-content-report-form.js 3.33 kB
./frontend/.nuxt/dist/client/components/v-content-report-form.modern.js 3.21 kB
./frontend/.nuxt/dist/client/components/v-content-report-popover.js 3.79 kB
./frontend/.nuxt/dist/client/components/v-content-report-popover.modern.js 3.67 kB
./frontend/.nuxt/dist/client/components/v-copy-button.js 3.99 kB
./frontend/.nuxt/dist/client/components/v-copy-button.modern.js 4 kB
./frontend/.nuxt/dist/client/components/v-copy-license.js 2.27 kB
./frontend/.nuxt/dist/client/components/v-copy-license.modern.js 2.25 kB
./frontend/.nuxt/dist/client/components/v-dmca-notice.js 793 B
./frontend/.nuxt/dist/client/components/v-dmca-notice.modern.js 798 B
./frontend/.nuxt/dist/client/components/v-error-image.js 2.41 kB
./frontend/.nuxt/dist/client/components/v-error-image.modern.js 2.38 kB
./frontend/.nuxt/dist/client/components/v-error-section.js 372 B
./frontend/.nuxt/dist/client/components/v-error-section.modern.js 375 B
./frontend/.nuxt/dist/client/components/v-external-search-form.js 3.76 kB
./frontend/.nuxt/dist/client/components/v-external-search-form.modern.js 3.12 kB
./frontend/.nuxt/dist/client/components/v-external-source-list.js 2.71 kB
./frontend/.nuxt/dist/client/components/v-external-source-list.modern.js 2.08 kB
./frontend/.nuxt/dist/client/components/v-full-layout.js 1.57 kB
./frontend/.nuxt/dist/client/components/v-full-layout.modern.js 1.57 kB
./frontend/.nuxt/dist/client/components/v-grid-skeleton.js 1.52 kB
./frontend/.nuxt/dist/client/components/v-grid-skeleton.modern.js 1.53 kB
./frontend/.nuxt/dist/client/components/v-hide-button.js 812 B
./frontend/.nuxt/dist/client/components/v-hide-button.modern.js 813 B
./frontend/.nuxt/dist/client/components/v-home-gallery.js 4.28 kB
./frontend/.nuxt/dist/client/components/v-home-gallery.modern.js 4.26 kB
./frontend/.nuxt/dist/client/components/v-homepage-content.js 1.8 kB
./frontend/.nuxt/dist/client/components/v-homepage-content.modern.js 1.77 kB
./frontend/.nuxt/dist/client/components/v-icon-button.js 523 B
./frontend/.nuxt/dist/client/components/v-icon-button.modern.js 528 B
./frontend/.nuxt/dist/client/components/v-image-cell.js 1.95 kB
./frontend/.nuxt/dist/client/components/v-image-cell.modern.js 1.94 kB
./frontend/.nuxt/dist/client/components/v-image-details.js 1.44 kB
./frontend/.nuxt/dist/client/components/v-image-details.modern.js 1.43 kB
./frontend/.nuxt/dist/client/components/v-image-grid.js 4.41 kB
./frontend/.nuxt/dist/client/components/v-image-grid.modern.js 4.29 kB
./frontend/.nuxt/dist/client/components/v-license-tab-panel.js 641 B
./frontend/.nuxt/dist/client/components/v-license-tab-panel.modern.js 650 B
./frontend/.nuxt/dist/client/components/v-load-more.js 1.18 kB
./frontend/.nuxt/dist/client/components/v-load-more.modern.js 1.07 kB
./frontend/.nuxt/dist/client/components/v-media-license.js 931 B
./frontend/.nuxt/dist/client/components/v-media-license.modern.js 939 B
./frontend/.nuxt/dist/client/components/v-media-reuse.js 2.93 kB
./frontend/.nuxt/dist/client/components/v-media-reuse.modern.js 2.91 kB
./frontend/.nuxt/dist/client/components/v-media-tag.js 416 B
./frontend/.nuxt/dist/client/components/v-media-tag.modern.js 421 B
./frontend/.nuxt/dist/client/components/v-modal.js 1 kB
./frontend/.nuxt/dist/client/components/v-modal.modern.js 996 B
./frontend/.nuxt/dist/client/components/v-no-results.js 2.66 kB
./frontend/.nuxt/dist/client/components/v-no-results.modern.js 2.03 kB
./frontend/.nuxt/dist/client/components/v-radio.js 1 kB
./frontend/.nuxt/dist/client/components/v-radio.modern.js 1.01 kB
./frontend/.nuxt/dist/client/components/v-related-audio.js 823 B
./frontend/.nuxt/dist/client/components/v-related-audio.modern.js 742 B
./frontend/.nuxt/dist/client/components/v-related-audio/pages/search/audio.js 4.38 kB
./frontend/.nuxt/dist/client/components/v-related-audio/pages/search/audio.modern.js 4.26 kB
./frontend/.nuxt/dist/client/components/v-related-images.js 805 B
./frontend/.nuxt/dist/client/components/v-related-images.modern.js 719 B
./frontend/.nuxt/dist/client/components/v-report-desc-form.js 976 B
./frontend/.nuxt/dist/client/components/v-report-desc-form.modern.js 980 B
./frontend/.nuxt/dist/client/components/v-row-layout.js 1.72 kB
./frontend/.nuxt/dist/client/components/v-row-layout.modern.js 1.73 kB
./frontend/.nuxt/dist/client/components/v-safety-wall.js 1.32 kB
./frontend/.nuxt/dist/client/components/v-safety-wall.modern.js 1.32 kB
./frontend/.nuxt/dist/client/components/v-scroll-button.js 891 B
./frontend/.nuxt/dist/client/components/v-scroll-button.modern.js 892 B
./frontend/.nuxt/dist/client/components/v-search-grid.js 6.83 kB
./frontend/.nuxt/dist/client/components/v-search-grid.modern.js 6.18 kB
./frontend/.nuxt/dist/client/components/v-search-results-title.js 616 B
./frontend/.nuxt/dist/client/components/v-search-results-title.modern.js 621 B
./frontend/.nuxt/dist/client/components/v-server-timeout.js 301 B
./frontend/.nuxt/dist/client/components/v-server-timeout.modern.js 306 B
./frontend/.nuxt/dist/client/components/v-single-result-controls.js 1.36 kB
./frontend/.nuxt/dist/client/components/v-single-result-controls.modern.js 1.36 kB
./frontend/.nuxt/dist/client/components/v-sketch-fab-viewer.js 1.01 kB
./frontend/.nuxt/dist/client/components/v-sketch-fab-viewer.modern.js 915 B
./frontend/.nuxt/dist/client/components/v-snackbar.js 1.07 kB
./frontend/.nuxt/dist/client/components/v-snackbar.modern.js 1.07 kB
./frontend/.nuxt/dist/client/components/v-sources-table.js 14.4 kB
./frontend/.nuxt/dist/client/components/v-sources-table.modern.js 14.4 kB
./frontend/.nuxt/dist/client/components/v-warning-suppressor.js 307 B
./frontend/.nuxt/dist/client/components/v-warning-suppressor.modern.js 311 B
./frontend/.nuxt/dist/client/pages/about.js 1.42 kB
./frontend/.nuxt/dist/client/pages/about.modern.js 1.42 kB
./frontend/.nuxt/dist/client/pages/audio/_id/index.js 10.7 kB
./frontend/.nuxt/dist/client/pages/audio/_id/index.modern.js 10.4 kB
./frontend/.nuxt/dist/client/pages/feedback.js 1.36 kB
./frontend/.nuxt/dist/client/pages/feedback.modern.js 1.36 kB
./frontend/.nuxt/dist/client/pages/image/_id/index.js 8.22 kB
./frontend/.nuxt/dist/client/pages/image/_id/index.modern.js 8.04 kB
./frontend/.nuxt/dist/client/pages/image/_id/report.js 4.89 kB
./frontend/.nuxt/dist/client/pages/image/_id/report.modern.js 4.67 kB
./frontend/.nuxt/dist/client/pages/index.js 6.33 kB
./frontend/.nuxt/dist/client/pages/index.modern.js 6.3 kB
./frontend/.nuxt/dist/client/pages/preferences.js 1.32 kB
./frontend/.nuxt/dist/client/pages/preferences.modern.js 1.31 kB
./frontend/.nuxt/dist/client/pages/privacy.js 1.26 kB
./frontend/.nuxt/dist/client/pages/privacy.modern.js 1.26 kB
./frontend/.nuxt/dist/client/pages/search-help.js 1.62 kB
./frontend/.nuxt/dist/client/pages/search-help.modern.js 1.61 kB
./frontend/.nuxt/dist/client/pages/search.js 2.27 kB
./frontend/.nuxt/dist/client/pages/search.modern.js 2.09 kB
./frontend/.nuxt/dist/client/pages/search/audio.js 503 B
./frontend/.nuxt/dist/client/pages/search/audio.modern.js 506 B
./frontend/.nuxt/dist/client/pages/search/image.js 461 B
./frontend/.nuxt/dist/client/pages/search/image.modern.js 462 B
./frontend/.nuxt/dist/client/pages/search/index.js 315 B
./frontend/.nuxt/dist/client/pages/search/index.modern.js 319 B
./frontend/.nuxt/dist/client/pages/search/model-3d.js 242 B
./frontend/.nuxt/dist/client/pages/search/model-3d.modern.js 246 B
./frontend/.nuxt/dist/client/pages/search/video.js 240 B
./frontend/.nuxt/dist/client/pages/search/video.modern.js 244 B
./frontend/.nuxt/dist/client/pages/sources.js 1.54 kB
./frontend/.nuxt/dist/client/pages/sources.modern.js 1.54 kB
./frontend/.nuxt/dist/client/runtime.js 2.77 kB
./frontend/.nuxt/dist/client/runtime.modern.js 2.77 kB
./frontend/.nuxt/dist/client/vendors/app.js 68.3 kB
./frontend/.nuxt/dist/client/vendors/app.modern.js 68.1 kB

compressed-size-action

@AetherUnbound AetherUnbound force-pushed the feature/individual-doc-differences branch from 876b38e to 8e5ff36 Compare August 18, 2023 19:55
@AetherUnbound
Copy link
Collaborator Author

Pushing one last commit to remove the example file, then merging this!
image

@AetherUnbound AetherUnbound merged commit 23d3972 into main Aug 19, 2023
@AetherUnbound AetherUnbound deleted the feature/individual-doc-differences branch August 19, 2023 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖 aspect: dx Concerns developers' experience with the codebase 🌟 goal: addition Addition of new feature 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: documentation Related to Sphinx documentation 🧱 stack: frontend Related to the Nuxt frontend 🧱 stack: mgmt Related to repo management and automations
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Add a list of (changed) rendered files to the "docs emitted" PR comment
6 participants