Successive requests for "sitemap-theses.xml" returns differing sets of results: related to #3289 #3361
Comments
|
I also tested this with the j@598739:~/dev/google_search_console/site-map/2024-01-30/rev2$ wget https://era.library.ualberta.ca/sitemap-items.xml
--2024-01-30 10:07:46-- https://era.library.ualberta.ca/sitemap-items.xml
Resolving era.library.ualberta.ca (era.library.ualberta.ca)... 129.128.216.104
Connecting to era.library.ualberta.ca (era.library.ualberta.ca)|129.128.216.104|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/xml]
Saving to: 'sitemap-items.xml.1'
sitemap-items.xml.1 [ <=> ] 6.32M --.-KB/s in 0.1s
2024-01-30 10:07:46 (44.8 MB/s) - 'sitemap-items.xml.1' saved [6631714]j@598739:~/dev/google_search_console/site-map/2024-01-30/rev2$ wget https://era.library.ualberta.ca/sitemap-items.xml
--2024-01-30 10:07:51-- https://era.library.ualberta.ca/sitemap-items.xml
Resolving era.library.ualberta.ca (era.library.ualberta.ca)... 129.128.216.104
Connecting to era.library.ualberta.ca (era.library.ualberta.ca)|129.128.216.104|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/xml]
Saving to: 'sitemap-items.xml.2'
sitemap-items.xml.2 [ <=> ] 6.99M 39.5MB/s in 0.2s
2024-01-30 10:07:52 (39.5 MB/s) - 'sitemap-items.xml.2' saved [7326358]j@598739:~/dev/google_search_console/site-map/2024-01-30/rev2$ wget https://era.library.ualberta.ca/sitemap-items.xml
--2024-01-30 10:07:58-- https://era.library.ualberta.ca/sitemap-items.xml
Resolving era.library.ualberta.ca (era.library.ualberta.ca)... 129.128.216.104
Connecting to era.library.ualberta.ca (era.library.ualberta.ca)|129.128.216.104|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/xml]
Saving to: 'sitemap-items.xml.3'
sitemap-items.xml.3 [ <=> ] 6.32M --.-KB/s in 0.1s |
jefferya
changed the title
|
Summary:
Test 1. Does a local environment Results: unable to reproduce in a dev environment Test 2. Does each production application server return the same response? Test: change local Results: sitemap-theses.xml and sitemap-item.xml are identical on each app server. Why the discrepancy with past results? $ wget https://era.library.ualberta.ca/sitemap-theses.xml sitemap-theses.xml.6 [ <=> ] 6.99M --.-KB/s in 0.1s 2024-02-28 15:30:14 (65.0 MB/s) - 'sitemap-theses.xml.6' saved [7326358] $ wget https://era.library.ualberta.ca/sitemap-theses.xml sitemap-theses.xml.14 [ <=> ] 6.99M --.-KB/s in 0.09s 2024-02-28 15:31:33 (80.7 MB/s) - 'sitemap-theses.xml.14' saved [7326358] $ wget https://era.library.ualberta.ca/sitemap-theses.xml sitemap-theses.xml.15 [ <=> ] 6.99M --.-KB/s in 0.09s 2024-02-28 15:32:06 (75.1 MB/s) - 'sitemap-theses.xml.15' saved [7326358] $ wget https://era.library.ualberta.ca/sitemap-items.xml sitemap-items.xml [ <=> ] 6.99M --.-KB/s in 0.1s 2024-02-28 15:32:29 (65.2 MB/s) - 'sitemap-items.xml' saved [7326358] $ wget https://era.library.ualberta.ca/sitemap-items.xml sitemap-items.xml.1 [ <=> ] 6.99M --.-KB/s in 0.1s 2024-02-28 15:32:54 (68.7 MB/s) - 'sitemap-items.xml.1' saved [7326358] $ wget https://era.library.ualberta.ca/sitemap-items.xml sitemap-items.xml.2 [ <=> ] 6.99M --.-KB/s in 0.1s 2024-02-28 15:33:14 (54.9 MB/s) - 'sitemap-items.xml.2' saved [7326358] |
|
Testing the hypothesis that the The To verify the hypothesis
diff --git a/config/environments/development.rb b/config/environments/development.rb
index b5f7947c..3bbaf5ab 100644
--- a/config/environments/development.rb
+++ b/config/environments/development.rb
@@ -30,9 +30,9 @@ Rails.application.configure do
'Cache-Control' => "public, max-age=#{2.days.to_i}"
}
else
- config.action_controller.perform_caching = false
+ config.action_controller.perform_caching = true
- config.cache_store = :null_store
+ config.cache_store = :memory_store
end
To fix
References
|
…3361 * adds a distinct cache key to the partial used by both the theses and items sitemap view
|
After changes that cause sitemap/_object.xml.bulder to generate a unique cache key based on the view (theses or items) calling the partial (notice the unique cache key): irb(main):004> Rails.cache.instance_variable_get(:@data).keys
=>
["views/sitemap/_object:29b55f4c76b6c3fb21553304686aae88/sitemap-theses",
"read_only_mode.first.enabled",
"views/sitemap/_object:29b55f4c76b6c3fb21553304686aae88/sitemap-items"]The output looks correct: $ wget http://era.lvh.me:3000/sitemap-items.xml
--2024-03-04 13:54:32-- http://era.lvh.me:3000/sitemap-items.xml
Resolving era.lvh.me (era.lvh.me)... 127.0.0.1
Connecting to era.lvh.me (era.lvh.me)|127.0.0.1|:3000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 24668 (24K) [application/xml]
Saving to: 'sitemap-items.xml'
sitemap-items.xml 100%[==================================>] 24.09K --.-KB/s in 0s
2024-03-04 13:54:32 (533 MB/s) - 'sitemap-items.xml' saved [24668/24668]$ wget http://era.lvh.me:3000/sitemap-theses.xml
--2024-03-04 13:54:37-- http://era.lvh.me:3000/sitemap-theses.xml
Resolving era.lvh.me (era.lvh.me)... 127.0.0.1
Connecting to era.lvh.me (era.lvh.me)|127.0.0.1|:3000... connected.
HTTP request sent, awaiting response... 200 OK
Length: 20647873 (20M) [application/xml]
Saving to: 'sitemap-theses.xml'
sitemap-theses.xml 100%[==================================>] 19.69M --.-KB/s in 0.03s
2024-03-04 13:55:07 (726 MB/s) - 'sitemap-theses.xml' saved [20647873/20647873] |
…3361 * adds a distinct cache key to the partial used by both the theses and items sitemap view
…3361 * adds a distinct cache key to the partial used by both the theses and items sitemap view
Prevent theses and items sitemap views from using the same cache key #3361
|
In production |
Possibly one cause of Google Search Console errors: #3289
Requests to https://era.library.ualberta.ca/sitemap-theses.xml will intermittently return a different set of results.
For example, the following is a recording of multiple requests in a short timespan. Notice the differing size of the resulting file. The resulting file with the smaller size seems to miss some (maybe all) theses.
Interesting correlations:
6631714sitemap-thesis.xmlfiles are identical tositemap-items.xml; the size732635sitemap-thesis.xmlare not6631714results: https://era.library.ualberta.ca/items/97bd4b30-9e0d-41b3-b668-f58bcd1ea06b7326358results: https://era.library.ualberta.ca/items/269f3d9e-eecb-4902-84eb-16cd70ff7347sitemap-thesis.xmlroute intermittently returning thesitemap-item.xmlresults?To Reproduce
wget https://era.library.ualberta.ca/sitemap-theses.xmlExpected behavior
The text was updated successfully, but these errors were encountered: