Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Site Editor: Create auto-drafts for modified files only #26383

Merged
merged 3 commits into from
Nov 16, 2020

Conversation

david-szabo97
Copy link
Member

@david-szabo97 david-szabo97 commented Oct 22, 2020

Description

Only run synchronization logic for modified files rather than all file-based template and template part files.

Fixes #25868

How has this been tested?

  • Edit a theme block template file
  • Load site editor
  • Confirm wp_template is created or updated
  • You might need to reload the site editor once

Types of changes

Code quality & Performance

Checklist:

  • My code is tested.
  • My code follows the WordPress code style.
  • My code follows the accessibility standards.
  • My code has proper inline documentation.
  • I've included developer documentation if appropriate.
  • I've updated all React Native files affected by any refactorings/renamings in this PR.

@david-szabo97 david-szabo97 self-assigned this Oct 22, 2020
@david-szabo97 david-szabo97 added the [Status] In Progress Tracking issues with work in progress label Oct 22, 2020
@github-actions
Copy link

github-actions bot commented Oct 22, 2020

Size Change: +56 B (0%)

Total Size: 1.19 MB

Filename Size Change
build/date/index.js 11.2 kB +1 B
build/edit-site/index.js 23.3 kB +55 B (0%)
ℹ️ View Unchanged
Filename Size Change
build/a11y/index.js 1.14 kB 0 B
build/annotations/index.js 3.77 kB 0 B
build/api-fetch/index.js 3.42 kB 0 B
build/autop/index.js 2.84 kB 0 B
build/blob/index.js 664 B 0 B
build/block-directory/index.js 8.71 kB 0 B
build/block-directory/style-rtl.css 943 B 0 B
build/block-directory/style.css 942 B 0 B
build/block-editor/index.js 133 kB 0 B
build/block-editor/style-rtl.css 11.3 kB 0 B
build/block-editor/style.css 11.3 kB 0 B
build/block-library/editor-rtl.css 8.91 kB 0 B
build/block-library/editor.css 8.92 kB 0 B
build/block-library/index.js 147 kB 0 B
build/block-library/style-rtl.css 8.1 kB 0 B
build/block-library/style.css 8.1 kB 0 B
build/block-library/theme-rtl.css 792 B 0 B
build/block-library/theme.css 793 B 0 B
build/block-serialization-default-parser/index.js 1.87 kB 0 B
build/block-serialization-spec-parser/index.js 3.06 kB 0 B
build/blocks/index.js 48 kB 0 B
build/components/index.js 171 kB 0 B
build/components/style-rtl.css 15.3 kB 0 B
build/components/style.css 15.3 kB 0 B
build/compose/index.js 9.9 kB 0 B
build/core-data/index.js 14.8 kB 0 B
build/data-controls/index.js 821 B 0 B
build/data/index.js 8.74 kB 0 B
build/deprecated/index.js 768 B 0 B
build/dom-ready/index.js 571 B 0 B
build/dom/index.js 4.92 kB 0 B
build/edit-navigation/index.js 11.1 kB 0 B
build/edit-navigation/style-rtl.css 881 B 0 B
build/edit-navigation/style.css 885 B 0 B
build/edit-post/index.js 306 kB 0 B
build/edit-post/style-rtl.css 6.51 kB 0 B
build/edit-post/style.css 6.49 kB 0 B
build/edit-site/style-rtl.css 3.98 kB 0 B
build/edit-site/style.css 3.98 kB 0 B
build/edit-widgets/index.js 26.3 kB 0 B
build/edit-widgets/style-rtl.css 3.16 kB 0 B
build/edit-widgets/style.css 3.16 kB 0 B
build/editor/editor-styles-rtl.css 476 B 0 B
build/editor/editor-styles.css 478 B 0 B
build/editor/index.js 42.5 kB 0 B
build/editor/style-rtl.css 3.85 kB 0 B
build/editor/style.css 3.85 kB 0 B
build/element/index.js 4.62 kB 0 B
build/escape-html/index.js 735 B 0 B
build/format-library/index.js 6.86 kB 0 B
build/format-library/style-rtl.css 547 B 0 B
build/format-library/style.css 548 B 0 B
build/hooks/index.js 2.16 kB 0 B
build/html-entities/index.js 623 B 0 B
build/i18n/index.js 3.57 kB 0 B
build/is-shallow-equal/index.js 698 B 0 B
build/keyboard-shortcuts/index.js 2.52 kB 0 B
build/keycodes/index.js 1.94 kB 0 B
build/list-reusable-blocks/index.js 3.1 kB 0 B
build/list-reusable-blocks/style-rtl.css 476 B 0 B
build/list-reusable-blocks/style.css 476 B 0 B
build/media-utils/index.js 5.32 kB 0 B
build/notices/index.js 1.79 kB 0 B
build/nux/index.js 3.4 kB 0 B
build/nux/style-rtl.css 671 B 0 B
build/nux/style.css 668 B 0 B
build/plugins/index.js 2.56 kB 0 B
build/primitives/index.js 1.43 kB 0 B
build/priority-queue/index.js 790 B 0 B
build/redux-routine/index.js 2.84 kB 0 B
build/reusable-blocks/index.js 3.05 kB 0 B
build/rich-text/index.js 13.3 kB 0 B
build/server-side-render/index.js 2.77 kB 0 B
build/shortcode/index.js 1.69 kB 0 B
build/token-list/index.js 1.27 kB 0 B
build/url/index.js 4.05 kB 0 B
build/viewport/index.js 1.84 kB 0 B
build/warning/index.js 1.14 kB 0 B
build/wordcount/index.js 1.22 kB 0 B

compressed-size-action

@david-szabo97 david-szabo97 marked this pull request as ready for review October 28, 2020 13:08
@david-szabo97 david-szabo97 changed the title [WIP] Run template reconciliation when theme version changes Run template reconciliation when theme version changes Oct 28, 2020
Copy link
Contributor

@Addison-Stavlo Addison-Stavlo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its nice to see more eyes in this template (and part) auto-drafting arena. It feels a bit all over the place atm (not this PR, just whats in place in general) and seems like it needs to be rethought a bit.

I see its still In Progress but a few Qs and thoughts:

lib/templates.php Outdated Show resolved Hide resolved
lib/templates.php Outdated Show resolved Hide resolved
lib/templates.php Outdated Show resolved Hide resolved
lib/templates.php Outdated Show resolved Hide resolved
@david-szabo97 david-szabo97 removed the [Status] In Progress Tracking issues with work in progress label Nov 3, 2020
@david-szabo97 david-szabo97 changed the title Run template reconciliation when theme version changes Site Editor: Run template reconciliation when theme version changes Nov 3, 2020
@Addison-Stavlo
Copy link
Contributor

May also be interested in #26650 - @youknowriad is also looking into the template part auto drafting stuff currently.

@youknowriad
Copy link
Contributor

It seems we should hold on this one for a bit until #26650 lands and simplifies all this logic.

@carlomanf
Copy link

It seems we should hold on this one for a bit until #26650 lands and simplifies all this logic.

@youknowriad that one appears to be about template parts, but this one is about templates, or do you anticipate extending that one to cover templates as well?

@youknowriad
Copy link
Contributor

that one appears to be about template parts, but this one is about templates, or do you anticipate extending that one to cover templates as well?

Oh right, I missed that part. I'm thinking of taking a look at templates though but I don't know if there will be a lot of changes there. So actually, feel free to proceed here and I'll just check whatever state we'll have.

@youknowriad
Copy link
Contributor

Actually, I have now updated the PR to also introduce the same "sync" function for templates.

@aristath
Copy link
Member

aristath commented Nov 5, 2020

Just curious.... Why not just store the filemtime() as post-meta in the template/template-part in the db, and then reconciliate if the current file's filemtime is newer than the stored one?

@noahtallen
Copy link
Member

Why not just store the filemtime() as post-meta in the template/template-part in the db, and then reconciliate if the current file's filemtime is newer than the stored one

Interesting! I wonder if this would work well with distributed databases/servers 🤔 But it does seem useful for development

@carlomanf
Copy link

Just curious.... Why not just store the filemtime() as post-meta in the template/template-part in the db, and then reconciliate if the current file's filemtime is newer than the stored one?

Is there any particular advantage to that over the current approach? It would be easily break if the db were manually modified and it would also complicate changing themes. The current approach by @david-szabo97 seems to work quite well.

@aristath
Copy link
Member

aristath commented Nov 6, 2020

After thinking about this some more, using post-meta would probably be bad for performance... The suggested implementation with the separate option containing versions of all templates feels a bit weird (why save a post's attribute - like its version - in an option instead of the post itself), but at the same time it will perform better.
However, using filemtime instead of the theme's version as the template-version would make sense as it can accommodate not only theme-updates, but also manual template edits etc with ease 🤔

@carlomanf
Copy link

The suggested implementation with the separate option containing versions of all templates feels a bit weird (why save a post's attribute - like its version - in an option instead of the post itself)

It's not doing that. It's saving the version of the theme, not the version of the post. All theme-related data is stored in the options table. Doesn't seem weird to me.

However, using filemtime instead of the theme's version as the template-version would make sense as it can accommodate not only theme-updates, but also manual template edits etc with ease 🤔

True, but would anyone ever need to edit an FSE template directly?

@david-szabo97
Copy link
Member Author

The new option is a single query, whereas checking the auto-drafts would require a single query for each template, plus we don't need the whole WP_Post object in this case.

@aristath
Copy link
Member

The new option is a single query, whereas checking the auto-drafts would require a single query for each template, plus we don't need the whole WP_Post object in this case.

Fair point. The option will indeed be more performant.

Copy link
Member

@noahtallen noahtallen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This overall looks awesome, thanks for working on this!

Unfortunately, I am noticing that some template parts are not loading in the editor. They are loading on the front end of the site with the correct template, but they don't load in the site editor after a few loads.

Screen Shot 2020-11-12 at 4 18 41 PM

Here are the details:

  1. twenty twentyone blocks does work. This is loaded as normal via wp-env.
  2. I have the Automattic/themes mapped to wp-content/themes/themes.
  3. Neither seedlet blocks or ibis (both from the above repo) will load template parts in the site editor.

I investigated a bit, and it is resolving the correct directory for the template parts:

$theme_slug = get_stylesheet();
// $theme_slug = "themes/seedlet-blocks"

$theme = wp_get_theme( $theme_slug );

$theme_dir = $theme->get_stylesheet_directory();
// $theme_dir = "/var/www/html/wp-content/themes/themes/seedlet-blocks"

And that directory exists and contains the "block-template-parts" directory. So it seems like the code should be working. I really can't tell why twenty twenty one blocks is working but not the other two.

lib/templates-sync.php Outdated Show resolved Hide resolved
lib/templates-sync.php Show resolved Hide resolved
@david-szabo97
Copy link
Member Author

Unfortunately, I am noticing that some template parts are not loading in the editor. They are loading on the front end of the site with the correct template, but they don't load in the site editor after a few loads.

Thanks for testing with other themes, I totally forgot to do that 😄 This commit should fix the bug. Can you give it a check?

Copy link
Member

@noahtallen noahtallen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's weird, for some reason it's still not working. Specifically, the index template won't load its template parts. The singular template does, and I can select the template parts from the sidebar. Edit: actually, the singular template won't either after cleaning the database.

Screen Shot 2020-11-13 at 2 57 57 PM

lib/templates-sync.php Show resolved Hide resolved
lib/templates-sync.php Outdated Show resolved Hide resolved
@youknowriad
Copy link
Contributor

how ready is this PR? It would be good to include in the RC release today.

@david-szabo97
Copy link
Member Author

Fixed the errors reported by Noah above. Did a few tests and seems to work fine. If you could do some tests and report back that would be great. I'm trying to get E2E tests to pass.

@youknowriad
Copy link
Contributor

@david-szabo97 There's a lot of e2e test failures on master now which I'm working on. Most of the failures in the PR should be unrelated but I'll you know once I make progress so you can rebase.

@jorgefilipecosta jorgefilipecosta force-pushed the update/site-editor-template-reconcillation branch from 4b19a47 to a522a6b Compare November 16, 2020 15:21
@Copons
Copy link
Contributor

Copons commented Nov 16, 2020

I've stumbled upon a potential issue, that might or might not be a big deal, depending on how "in depth" a site is used.

Recently I've been working a lot with templates and template parts, and to keep things safe while switching branches, I got used to delete all wp_template and wp_template_part posts all the times.

As it turned out, I've tested this branch, then changed to work on something else, then deleted all templates and parts, and switched back on this branch.
What happened was that no templates and parts were created.

This is because the time check options were created during my original test.
When I came back to this branch without any templates and parts posts, $last_check was set to a timestamp more recent than the theme files modification date (as I normally don't wipe the theme).

For new themes activations, this PR relies on the absence of time check options for the given theme, defaulting to 0, which is safely older than anything:

$last_check = isset( $last_checks[ $theme_slug ] ) ? $last_checks[ $theme_slug ] : 0;

This was also confirmed by the fact that once I deleted the time check options too, the template and parts were created successfully.

In other words: if a user deletes all wp_template (and _part) posts, they also need to delete the gutenberg_last_synchronize_theme_{ type }_checks options, otherwise the site won't create the auto-drafts anymore, leaving the site broken.


I fully recognize that this is not a normal flow, but I also can't help but wonder if the time check options might unintentionally hinder what is supposed to be a relatively safe operation, without possibility of recover (unless one is aware of the options).

This will be aggravated if we end up displaying auto-draft in the templates and parts wp-admin lists (as proposed in #26636). (As a side note: the intention with showing auto-drafts there is to help with the development, rather than being a production feature)

Am I overestimating this potential issue?

@aristath
Copy link
Member

I think it's an edge case... but perhaps we could delete the stored timestamps if the file doesn't exist? 🤔

@Copons
Copy link
Contributor

Copons commented Nov 16, 2020

I think it's an edge case... but perhaps we could delete the stored timestamps if the file doesn't exist? 🤔

@aristath Unfortunately it's the other way around 😅

The files are there, untouched.
What I deleted are the templates and parts posts!

Basically we would need to:

  • Is the file more recent than the last check?
    • Yes: update the template post. 👍
    • No. Is there a template post in the DB?
      • Yes: good to go! 👍
      • No: create a template post for that file.

Which... I'm pretty sure it would remove all benefits of this PR. 😄

@Copons
Copy link
Contributor

Copons commented Nov 16, 2020

With the 6e8bb9d change, the issue seems fixed.

Just for additional context, I have found a definitely not edge case that would have triggered this issue (which, again, is now fixed 🙂):

  • Let's assume we have a clean site with twenty-twentyone-blocks.
  • Tun the templates sync by opening the front end or the Site Editor.
  • Open the Site Editor: it should be on the Front Page template.
  • Customize the template and save.
  • Open the Templates list (/wp-admin/edit.php?post_type=wp_template): there should be a Front Page template.
  • Trash it.
  • Open the Site Editor again: now it's on the Index template (which is the fallback if Front Page is missing).
  • Look in the sidebar: no Front Page template anymore.

This was caused by the same issue: we only compared the sync date with the file date, but didn't consider that a template might be deleted.

6e8bb9d should fix most use cases. I guess power users should be also able to delete a site option, if needs arise. 🤔

@jorgefilipecosta jorgefilipecosta merged commit 0e6da94 into master Nov 16, 2020
@jorgefilipecosta jorgefilipecosta deleted the update/site-editor-template-reconcillation branch November 16, 2020 18:18
@github-actions github-actions bot added this to the Gutenberg 9.4 milestone Nov 16, 2020
* @param WP_Post $post WP_Post instance of the deleted post.
*/
function gutenberg_clear_synchronize_last_checks_after_delete( $postid, $post ) {
if ( 'wp_template' !== $post->post_type || 'wp_template_part' !== $post->post_type ) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be 'wp_template' === $post->post_type?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have a PR to fix the issues introduced here so I can backport them to 9.4?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started one here #27068.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Type] Performance Related to performance efforts
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Site Editor: Templates Reconciliation Performance
9 participants