Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AMP post-processor cache disabled spuriously, and attributed to ACME HTTP-01 challenge token #2694

Closed
willangley opened this issue Jun 27, 2019 · 13 comments
Labels
Support Help with setup, implementation, or "How do I?" questions.

Comments

@willangley
Copy link

I'm seeing a message in wp-admin on my website, willangley.org:

The post-processor cache was disabled due to detecting randomly generated content found on on this web page.

Randomly generated content was detected on this web page. To avoid filling up the cache with unusable content, the AMP plugin's post-processor cache was automatically disabled. Read more.

The content is an ACME challenge and has to be randomly generated to be secure. (I've replaced the actual token, which is the long hexadecimal string from an HTTP-01 challenge, with $TOKEN since the ACME documentation doesn't make it clear if this is safe to publicize or not).

The AMP plugin should know better than to break in this condition 😛 .

@willangley
Copy link
Author

I've checked Site Health and think everything is up-to-date:

### wp-core ###

version: 5.2.2
site_language: en_US
user_language: en_US
permalink: /%postname%/
https_status: true
user_registration: 0
default_comment_status: closed
multisite: false
user_count: 1
dotorg_communication: true

### wp-paths-sizes ###

wordpress_path: /usr/share/nginx/html/willangley.org/public
wordpress_size: 37.46 MB (39277648 bytes)
uploads_path: /usr/share/nginx/html/willangley.org/public/wp-content/uploads
uploads_size: 98.68 MB (103469940 bytes)
themes_path: /usr/share/nginx/html/willangley.org/public/wp-content/themes
themes_size: 3.00 MB (3141857 bytes)
plugins_path: /usr/share/nginx/html/willangley.org/public/wp-content/plugins
plugins_size: 33.56 MB (35195287 bytes)
database_size: 6.19 MB (6488064 bytes)
total_size: 178.88 MB (187572796 bytes)

### wp-dropins (2) ###

advanced-cache.php: true
object-cache.php: true

### wp-active-theme ###

name: Genesis Sample
version: 3.0.0-beta
author: StudioPress
author_website: https://www.studiopress.com/
parent_theme: Genesis
theme_features: menus, post-thumbnails, title-tag, automatic-feed-links, body-open, genesis-inpost-layouts, genesis-archive-layouts, genesis-admin-menu, genesis-seo-settings-menu, genesis-import-export-menu, genesis-readme-menu, genesis-customizer-theme-settings, genesis-customizer-seo-settings, genesis-auto-updates, genesis-breadcrumbs, genesis-menus, genesis-structural-wraps, widgets, html5, genesis-accessibility, custom-logo, genesis-after-entry-widget-area, genesis-footer-widgets, editor-styles, editor-style, align-wide, responsive-embeds, editor-font-sizes, editor-color-palette, amp
theme_path: /usr/share/nginx/html/willangley.org/public/wp-content/themes/genesis

### wp-themes (1) ###

Genesis: version: 3.0.1, author: StudioPress

### wp-plugins-active (13) ###

Akismet Anti-Spam: version: 4.1.2, author: Automattic
AMP: version: 1.2.0, author: AMP Project Contributors
Atomic Blocks - Gutenberg Blocks Collection: version: 2.0, author: atomicblocks
Code Syntax Block (with Server-Side Highlighting): version: 1.0.0, author: Weston Ruter
Genesis Beta Tester: version: 0.9.5, author: Nathan Rice
Genesis Simple Edits: version: 2.2.1, author: StudioPress
Jetpack by WordPress.com: version: 7.4.1, author: Automattic
jetpack filters: version: 0.0.1, author: Will Angley
Simple Social Icons: version: 3.0.1, author: StudioPress
VaultPress: version: 1.9.10, author: Automattic
WP Redis: version: 0.7.1, author: Pantheon, Josh Koenig, Matthew Boynes, Daniel Bachhuber, Alley Interactive
WP Super Cache: version: 1.6.7, author: Automattic
wp_mail filters: version: 0.0.1, author: Will Angley

### wp-media ###

image_editor: WP_Image_Editor_Imagick
imagick_module_version: 1687
imagemagick_version: ImageMagick 6.9.7-4 Q16 x86_64 20170114 http://www.imagemagick.org
imagick_limits: 
	imagick::RESOURCETYPE_AREA: 122 MB
	imagick::RESOURCETYPE_DISK: 1073741824
	imagick::RESOURCETYPE_FILE: 768
	imagick::RESOURCETYPE_MAP: 512 MB
	imagick::RESOURCETYPE_MEMORY: 256 MB
	imagick::RESOURCETYPE_THREAD: 1
gd_version: 2.2.5
ghostscript_version: 9.26

### wp-server ###

server_architecture: Linux 4.15.0-52-generic x86_64
httpd_software: nginx/1.14.0
php_version: 7.2.19-0ubuntu0.18.04.1 64bit
php_sapi: fpm-fcgi
max_input_variables: 1000
time_limit: 30
memory_limit: 256M
max_input_time: 60
upload_max_size: 2M
php_post_max_size: 8M
curl_version: 7.58.0 OpenSSL/1.1.0g
suhosin: false
imagick_availability: true

### wp-database ###

extension: mysqli
server_version: 5.5.5-10.1.40-MariaDB-0ubuntu0.18.04.1
client_version: mysqlnd 5.0.12-dev - 20150407 - $Id: 3591daad22de08524295e1bd073aceeff11e6579 $

### wp-constants ###

WP_HOME: undefined
WP_SITEURL: undefined
WP_CONTENT_DIR: /usr/share/nginx/html/willangley.org/public/wp-content
WP_PLUGIN_DIR: /usr/share/nginx/html/willangley.org/public/wp-content/plugins
WP_MAX_MEMORY_LIMIT: 256M
WP_DEBUG: false
WP_DEBUG_DISPLAY: true
WP_DEBUG_LOG: false
SCRIPT_DEBUG: false
WP_CACHE: true
CONCATENATE_SCRIPTS: undefined
COMPRESS_SCRIPTS: undefined
COMPRESS_CSS: undefined
WP_LOCAL_DEV: undefined

### wp-filesystem ###

wordpress: writable
wp-content: writable
uploads: writable
plugins: writable
themes: writable

### jetpack ###

site_id: 162362699
ssl_cert: No
time_diff: undefined
version_option: 7.4.1:1560818552
old_version: 7.4:1559695382
public: Public
master_user: #1 will ([email protected])
current_user: #1 will ([email protected])
tokens_set: Blog User
blog_token: [REDACTED]
user_token: [REDACTED]
version: 7.4.1
jp_plugin_dir: /usr/share/nginx/html/willangley.org/public/wp-content/plugins/jetpack/
plan: premium
HTTP_HOST: willangley.org
SERVER_PORT: 443
HTTPS: on
REMOTE_ADDR: 142.255.84.251
protect_header: {"trusted_header":"REMOTE_ADDR","segments":1,"reverse":false}
full_sync: {"started":"Sun, 19 May 2019 19:50:00 +0000","queue_finished":"Sun, 19 May 2019 19:50:00 +0000","send_started":"Sun, 19 May 2019 19:50:07 +0000","finished":"Sun, 19 May 2019 19:50:07 +0000","sent":{"constants":1,"functions":1,"options":1,"users":1},"sent_total":{"constants":-1,"functions":-1,"options":-1,"users":1},"queue":{"constants":1,"functions":1,"options":1,"users":1},"config":{"options":true,"functions":true,"constants":true,"users":[1]},"total":{"constants":1,"functions":1,"options":1,"users":1}}
sync_size: undefined
sync_lag: 0 seconds
full_sync_size: undefined
full_sync_lag: 0 seconds
idc_urls: {"home":"https:\/\/willangley.org","siteurl":"https:\/\/willangley.org","WP_HOME":"","WP_SITEURL":""}
idc_error_option: false
idc_optin: true
cxn_tests: All Pass.

### genesis ###

blog_cat_num: 6
content_archive: full
image_alignment: alignleft
posts_nav: numeric
site_layout: full-width-content
update: 1
comments_posts: 1
comments_pages: 1
theme_version: 3.0.0-beta2
db_version: 3001
upgrade: 1

@westonruter
Copy link
Member

Where is this token being added? Is it in the HTML or is it a response header?

@swissspidy swissspidy added Needs More Info Follow-up required in order to be actionable. Support Help with setup, implementation, or "How do I?" questions. Needs Testing Issues that need to be confirmed. labels Jun 27, 2019
@willangley
Copy link
Author

willangley commented Jun 27, 2019

Tokens under .well-known/acme/... are generated by python-certbot-nginx , which is why it's surprising that amp-wp cares about them at all.

I've looked at the HTML for the main page, a post, and a page on my site, and none of them contain a link to .well-known .

@westonruter
Copy link
Member

Question: How did you identify ACME as being the cause of the problem?

I think the issue may not be related to the ACME challenge. If I do two requests as follows:

curl https://willangley.org/early-summer-film/?1 > /tmp/1
curl https://willangley.org/early-summer-film/?2 > /tmp/2

And then I do a diff I get:

22c22
<                         <input type="hidden" name="action" value="subscribe"><input type="hidden" name="source" value="https://willangley.org/early-summer-film/?1"><input type="hidden" name="sub-type" value="widget"><input type="hidden" name="redirect_fragment" value="blog_subscription-4"><button type="submit" name="jetpack_subscriptions_widget">
---
>                         <input type="hidden" name="action" value="subscribe"><input type="hidden" name="source" value="https://willangley.org/early-summer-film/?2"><input type="hidden" name="sub-type" value="widget"><input type="hidden" name="redirect_fragment" value="blog_subscription-4"><button type="submit" name="jetpack_subscriptions_widget">
28,29c28,29
< <!-- Dynamic page generated in 0.111 seconds. -->
< <!-- Cached page generated by WP-Super-Cache on 2019-06-29 11:50:05 -->
---
> <!-- Dynamic page generated in 0.112 seconds. -->
> <!-- Cached page generated by WP-Super-Cache on 2019-06-29 11:50:43 -->

I think what may be going on is the variable content that is being added by WP-Super-Cache. Do you get that issue without

That being said, even without that specific plugin we've also noticed the post-processor caching being auto-disabled on amp-wp.org as well, so it seems that the logic for determining when variable content is present is not proving robust.

I wonder if the post-processor cache is actually proving effective n the wild, or if it constant winds up getting auto-deactivated.

@willangley
Copy link
Author

willangley commented Jun 29, 2019

Thanks Weston!

The AMP plugin admin console page has consistently attributed the problem to an ACME URL:

orQ6pXfdQ2v

but I'm not sure if they're the only pages with cache misses.

I've disabled the cache status messages from WP-Super-Cache to see if the issue persists:

z8SopBgv5z6

The other part of the diff you observed by changing GET parameters seems to be coming from Jetpack Subscriptions copying the GET parameters used in a request into a hidden field called "source".

I have no idea why it's doing that - I guess it makes sense for a site that hasn't configured permalinks, where GET parameters are the only way it could report what page a visitor was looking at when they subscribed, but it doesn't make much sense here.

@willangley willangley changed the title AMP post-processor cache disabled spuriously because of ACME HTTP-01 challenge token AMP post-processor cache disabled spuriously, and attributed to ACME HTTP-01 challenge token Jun 29, 2019
@westonruter
Copy link
Member

The Jetpack Subscriptions field wouldn't be a problem.

Also I think the WP-Super-Cache shouldn't actually be a problem because the timestamp shouldn't change for a given URL while it is cached.

But the well-known URL... what is currently serving the response for a request to this URL? Is a plugin generating the response there? What is contained in the page? It seems to be that the AMP plugin should be skipping to process that URL entirely.

@willangley
Copy link
Author

The well-known URLs are being served by python3-certbot-nginx and WordPress shouldn't be involved in serving them.

That said, I let the Ubuntu dpkg installer configure python3-certbot-nginx and wouldn't be surprised if the config I'm running is sending requests for those well-known URLs to WordPress too. I can try to debug today.

@westonruter
Copy link
Member

If WordPress is unexpectedly serving those well-known requests, then we may need to patch is_amp_endpoint() to return false when requesting that URL, similar to how we are handling other special requests (like feeds). I'm not very familiar with what this well-known URL returns normally.

@willangley
Copy link
Author

willangley commented Jun 30, 2019 via email

@westonruter
Copy link
Member

It turns out to actually be a problem with the 404.php template. It is including a search from which has a randomly-generated input field ID:

- <label class="search-form-label screen-reader-text" for="searchform-5d1a25f61f66c9.43057481">Search this website</label><input class="search-form-input" type="search" itemprop="query-input" name="s" id="searchform-5d1a25f61f66c9.43057481" placeholder="Search this website">
+ <label class="search-form-label screen-reader-text" for="searchform-5d1a25f7cf2f48.02951824">Search this website</label><input class="search-form-input" type="search" itemprop="query-input" name="s" id="searchform-5d1a25f7cf2f48.02951824" placeholder="Search this website">

So this should be updated to use a stable ID generator like wp_unique_id(). Given that you are using Genesis, we'll need to open a PR upstream to fix this I believe.

@westonruter
Copy link
Member

westonruter commented Jul 1, 2019

@willangley Here's a bit of code you can add to your child theme's functions.php to work around this problem:

add_filter(
	'genesis_search_form',
	function ( $form ) {
		static $counter = 0;
		$counter++;
		$form = preg_replace( '/(?<=="searchform-)[^"]+(?=")/', $counter, $form );
		return $form;
	}
);

I'll submit a PR to Genesis to fix this upstream.

@westonruter westonruter removed Needs Testing Issues that need to be confirmed. Needs More Info Follow-up required in order to be actionable. labels Jul 1, 2019
@westonruter
Copy link
Member

PR submitted for Genesis, so I hope it will be in the next version.

The submitted fix is basically doing the same thing that was done in core for Twenty Seventeen: https://core.trac.wordpress.org/ticket/44883

@westonruter
Copy link
Member

FYI: The fix should be released with Genesis v3.0.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Support Help with setup, implementation, or "How do I?" questions.
Projects
None yet
Development

No branches or pull requests

3 participants