Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix REST API auto-draft creation #26650

Merged
merged 13 commits into from
Nov 9, 2020

Conversation

youknowriad
Copy link
Contributor

@youknowriad youknowriad commented Nov 3, 2020

As you can see on the template-part e2e test failures that happen from time to time, creating and fetching drafts right now is broken and suffering a race condition. It's very hard to localize properly because it's very hard to reason about the flows to create those drafts right now. I think it's time to step back and think about how we want template parts flows to work.

This PR starts with the following assumption: The site editor always fetches template parts using REST API.
Based on this premise, why not just create these auto drafts on all GET requests to template parts? It's important to note that it's close to how it works currently but right now, it's hidden behind a number of function calls and is not consistent.

This will probably break some minor flows like site export but I believe we can fix these differently.

I also looked at the code for initializing templates... there's a lot of unneeded code IMO that is just creating hidden bugs.

What do you think? What am I missing?

lib/template-loader.php Outdated Show resolved Hide resolved
@github-actions
Copy link

github-actions bot commented Nov 3, 2020

Size Change: -629 B (0%)

Total Size: 1.21 MB

Filename Size Change
build/api-fetch/index.js 3.45 kB -2 B (0%)
build/block-directory/index.js 8.72 kB -2 B (0%)
build/block-editor/index.js 133 kB +11 B (0%)
build/block-library/editor-rtl.css 8.96 kB -57 B (0%)
build/block-library/editor.css 8.96 kB -53 B (0%)
build/block-library/index.js 147 kB +204 B (0%)
build/block-library/style-rtl.css 8.03 kB +66 B (0%)
build/block-library/style.css 8.03 kB +68 B (0%)
build/components/index.js 172 kB -95 B (0%)
build/components/style-rtl.css 15.3 kB +23 B (0%)
build/components/style.css 15.3 kB +25 B (0%)
build/compose/index.js 9.8 kB -2 B (0%)
build/data-controls/index.js 771 B -1 B
build/data/index.js 8.8 kB +29 B (0%)
build/deprecated/index.js 769 B +1 B
build/edit-navigation/index.js 11.2 kB +4 B (0%)
build/edit-post/index.js 306 kB -2 B (0%)
build/edit-site/index.js 22.6 kB +53 B (0%)
build/edit-widgets/index.js 26.3 kB +2 B (0%)
build/editor/index.js 42.8 kB +1 B
build/element/index.js 4.65 kB -1 B
build/format-library/index.js 6.63 kB -1.07 kB (16%) 👏
build/hooks/index.js 2.16 kB -1 B
build/keyboard-shortcuts/index.js 2.52 kB +2 B (0%)
build/keycodes/index.js 1.94 kB -1 B
build/list-reusable-blocks/index.js 3.11 kB +1 B
build/media-utils/index.js 5.34 kB +1 B
build/notices/index.js 1.78 kB -1 B
build/nux/index.js 3.42 kB +2 B (0%)
build/plugins/index.js 2.56 kB -2 B (0%)
build/redux-routine/index.js 2.85 kB +1 B
build/rich-text/index.js 13.4 kB +166 B (1%)
build/server-side-render/index.js 2.77 kB +3 B (0%)
build/token-list/index.js 1.27 kB -1 B
build/viewport/index.js 1.84 kB -1 B
build/warning/index.js 1.14 kB +1 B
ℹ️ View Unchanged
Filename Size Change
build/a11y/index.js 1.14 kB 0 B
build/annotations/index.js 3.78 kB 0 B
build/autop/index.js 2.84 kB 0 B
build/blob/index.js 665 B 0 B
build/block-directory/style-rtl.css 943 B 0 B
build/block-directory/style.css 942 B 0 B
build/block-editor/style-rtl.css 11.2 kB 0 B
build/block-editor/style.css 11.2 kB 0 B
build/block-library/theme-rtl.css 792 B 0 B
build/block-library/theme.css 793 B 0 B
build/block-serialization-default-parser/index.js 1.88 kB 0 B
build/block-serialization-spec-parser/index.js 3.1 kB 0 B
build/blocks/index.js 48.1 kB 0 B
build/core-data/index.js 12.5 kB 0 B
build/date/index.js 31.8 kB 0 B
build/dom-ready/index.js 571 B 0 B
build/dom/index.js 4.46 kB 0 B
build/edit-navigation/style-rtl.css 881 B 0 B
build/edit-navigation/style.css 885 B 0 B
build/edit-post/style-rtl.css 6.41 kB 0 B
build/edit-post/style.css 6.39 kB 0 B
build/edit-site/style-rtl.css 3.91 kB 0 B
build/edit-site/style.css 3.91 kB 0 B
build/edit-widgets/style-rtl.css 3.12 kB 0 B
build/edit-widgets/style.css 3.12 kB 0 B
build/editor/editor-styles-rtl.css 480 B 0 B
build/editor/editor-styles.css 482 B 0 B
build/editor/style-rtl.css 3.85 kB 0 B
build/editor/style.css 3.85 kB 0 B
build/escape-html/index.js 735 B 0 B
build/format-library/style-rtl.css 547 B 0 B
build/format-library/style.css 548 B 0 B
build/html-entities/index.js 623 B 0 B
build/i18n/index.js 3.57 kB 0 B
build/is-shallow-equal/index.js 712 B 0 B
build/list-reusable-blocks/style-rtl.css 476 B 0 B
build/list-reusable-blocks/style.css 476 B 0 B
build/nux/style-rtl.css 671 B 0 B
build/nux/style.css 668 B 0 B
build/primitives/index.js 1.43 kB 0 B
build/priority-queue/index.js 791 B 0 B
build/reusable-blocks/index.js 3.06 kB 0 B
build/shortcode/index.js 1.69 kB 0 B
build/url/index.js 4.06 kB 0 B
build/wordcount/index.js 1.22 kB 0 B

compressed-size-action

@gziolo gziolo self-requested a review November 3, 2020 07:29
@youknowriad youknowriad added the [Type] Experimental Experimental feature or API. label Nov 3, 2020
lib/class-wp-rest-template-parts-controller.php Outdated Show resolved Hide resolved
lib/load.php Outdated Show resolved Hide resolved
)
);
} else {
// Potentially we could decide to update the content if different.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we aren't comparing content as in the previous implementation, @david-szabo97 has been looking into re-writing the auto-drafts if the theme version changes - #26383

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, with this simpler syncing function, it's just a matter of comparing content here and updating the auto-draft if needed.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the updating of the auto-draft was already being done prior to #26383. The purpose of #26383 was to stop that code from running whenever it was not necessary to run it (i.e. theme version hasn't changed.) This PR seems to go backwards in that respect.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because gutenberg_resolve_template calls file_get_contents on every template in the theme while gutenberg_find_template_post_and_parts only called it on the one being requested. That means if you are loading 10 templates in the site editor then file_get_contents will get called 100 times.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do you see the template part update was being done before? By my own reading of the code, it was not being done anywhere.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was talking about templates actually. They were being updated inside gutenberg_find_template_post_and_parts if the file contents had changed. I don't know about template parts.

Copy link
Contributor

@Addison-Stavlo Addison-Stavlo Nov 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where do you see the template part update was being done before?

From what I can tell, the following was the closest to this. It looks like it didn't so much update the auto-draft itself, but create a new one. The current code only gets to this point if a publish version isn't found, but if the contents of the auto-draft don't match the contents of the file then a new auto-draft is created:

$file_contents = file_get_contents( $template_part_file_path );
if ( $template_part_post && $template_part_post->post_content === $file_contents ) {
$template_part_id = $template_part_post->ID;
} else {
$template_part_id = wp_insert_post(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I can see that difference and it's very problematic IMO. Creating a new auto-draft when the post_content changes is probably not the best approach to synchronization.

Let's leave this syncing behavior to its own dedicated PR since it's broken in master anyway (proliferation of auto-drafts).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's very problematic IMO.
Let's leave this syncing behavior to its own dedicated PR

Agreed, and that sounds like a good plan!

@ockham
Copy link
Contributor

ockham commented Nov 3, 2020

As you can see on the template-part e2e test failures that happen from time to time, creating and fetching drafts right now is broken and suffering a race condition. It's very hard to localize properly because it's very hard to reason about the flows to create those drafts right now.

Yeah, those are some complex algorithms, and it's quite likely that we're now running auto-draft creation too much as a consequence of trying to capture all cases where they're needed. (We had this kind of problem before: #23662)

I think it's time to step back and think about how we want template parts flows to work.

I definitely agree with the sentiment. I think part of the complexity results from the fact that we're really pushing the entities concept here, where a dedicated endpoint for template resolution might make more sense. We might want to revisit this design decision -- though it might mean an even bigger rewrite, so it's a decision that should not be taken lightly.

More importantly however, I think these methods really need unit test coverage to make sure their desired behavior is documented and guarded against regressions, since from experience, these are quite fragile and very easy to break in some cases. (Unit tests could also be used to make sure that auto-drafts aren't created unnecessarily.)

@youknowriad
Copy link
Contributor Author

More importantly however, I think these methods really need unit test coverage to make sure their desired behavior is documented and guarded against regressions, since from experience, these are quite fragile and very easy to break in some cases. (Unit tests could also be used to make sure that auto-drafts aren't created unnecessarily.)

Definitely, I do think these flows are very important to test properly, thought, at this point, I'm not really sure whether the code I removed is necessary or not and what flows they are trying to achieve. We do have end 2 end tests for template parts loading, creating and saving and these tests are passing properly. The only thing on this PR that I'm certain that is broken now and untested is the site zip export.

@gziolo
Copy link
Member

gziolo commented Nov 3, 2020

I personally wouldn't mind that this proposed code removal would break something that isn't covered with tests. It would be a good excuse to write more tests and better understand how we can further optimize existing flows. We should also keep in mind that at some point we will have to integrate all those hooks with the WordPress core so we better have the logic as much consolidated as possible.

@noahtallen
Copy link
Member

I agree with all of the sentiments about updating this code. I think it is complex partly because the initial concept was hacked together to get it working. We've been building off of that foundation ever since, so I doubt the code has ever gotten a thorough look-over (other than each of us trying to go through and understand it better.)

@youknowriad youknowriad marked this pull request as ready for review November 5, 2020 11:05
@youknowriad
Copy link
Contributor Author

Ok, I pushed this a bit further. It now touches template resolution as well 😬 and fixes site exports. The main idea on which all this PR relies on is:

Let's not mix templates from theme files and templates the CPT on our logic to render/edit templates. Let's just load the theme templates as CPT auto-drafts all the time and only rely on the CPT for the different flows. Noting that this was somehow how it used to work as well but it wasn't explicit, it was behind different function calls that are hard to follow.

I think I'm personally satisfied with where this is heading but I'd love some testing and thoughts.
I'm also planning to write some documentation on the architecture docs to clarify these things a little bit.

lib/edit-site-export.php Outdated Show resolved Hide resolved
lib/template-parts.php Outdated Show resolved Hide resolved
lib/edit-site-export.php Outdated Show resolved Hide resolved
@youknowriad
Copy link
Contributor Author

@david-szabo97 yes, I'm going to leave it out, just make a place for it

@youknowriad youknowriad force-pushed the try/fix-template-part-loading-flows branch from dbc60bd to 942aac1 Compare November 6, 2020 10:22
Copy link
Member

@TimothyBJacobs TimothyBJacobs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tested but looks good to me code wise.

lib/template-parts.php Show resolved Hide resolved
lib/template-parts.php Show resolved Hide resolved
Copons
Copons previously requested changes Nov 6, 2020
lib/templates-sync.php Outdated Show resolved Hide resolved
lib/template-loader.php Outdated Show resolved Hide resolved
lib/edit-site-export.php Outdated Show resolved Hide resolved
lib/templates.php Outdated Show resolved Hide resolved
lib/templates.php Outdated Show resolved Hide resolved
@youknowriad
Copy link
Contributor Author

I added a document to explain how template and template parts work. I think this is ready to land now.

@youknowriad youknowriad dismissed Copons’s stale review November 9, 2020 08:32

Should be good now.

@youknowriad youknowriad merged commit c6093bc into master Nov 9, 2020
@youknowriad youknowriad deleted the try/fix-template-part-loading-flows branch November 9, 2020 08:33
@github-actions github-actions bot added this to the Gutenberg 9.4 milestone Nov 9, 2020
Comment on lines +106 to +109
if ( ! $theme ) {
update_post_meta( $post_id, 'theme', wp_get_theme()->get_stylesheet() );
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im now realizing this means that custom template parts created through wp-admin will be given a theme value corresponding to the active theme. This will make it hard to differentiate TPs that were originally supplied by a theme vs. custom user created ones that should not be tied to a theme, which will in turn make it difficult to show a proper list of template parts in the site editor ( current theme + custom ). 🤔

I wonder how we can get around that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I wrote this, template parts were theme-specific on my mind even user created one but if they are not, maybe we can just show all the template parts from all themes (non auto-drafts) + the auto-drafts of the current theme in the UI.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be fixed by #27016, which has a new taxonomy term wp_file_based to indicate ones that originated from the theme. Ones that originated from the user won't have this taxonomy term.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that taxonomy is weird, what if I have something that originates from the theme but then I completely edit it, at what time it stops to be something that originates from the theme and that becomes user based?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've replied about it in another PR, so let me just link that comment here: #27016 (comment)

The tl;dr is that _wp_file_based only indicate the origin of a template/part, which is useful for things like reverting it to the file version.
Once a template/part is not auto-draft anymore, it's safe to say it's "user based", regardless of the _wp_file_based tag.

(I'm totally open to change the tag name if y'all feel it's confusing!)

Copy link
Contributor

@Addison-Stavlo Addison-Stavlo Nov 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I wrote this, template parts were theme-specific on my mind even user created one but if they are not, maybe we can just show all the template parts from all themes (non auto-drafts) + the auto-drafts of the current theme in the UI.

I don't think there has been much discussion on whether they should or shouldn't be tied to the theme. However, in the past slug/theme combos were required to be unique and creating a template part required inputing a specific theme value. When this restriction was taken off, custom template parts were created with a '' theme name making them distinguishable from theme supplied ones. Now that they do have the same theme value, there are other conflicts that can come into play such as unedited theme supplied templates potentially referencing the wrong template parts as described in the second half of my comment here -#26950 (comment) - but using something like the wp_file_based could clear that up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
REST API Interaction Related to REST API [Type] Experimental Experimental feature or API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants