Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make sitemapindex valid: only include *same site* sitemaps #145

Open
2 tasks
orbeckst opened this issue Jul 17, 2020 · 1 comment
Open
2 tasks

make sitemapindex valid: only include *same site* sitemaps #145

orbeckst opened this issue Jul 17, 2020 · 1 comment
Labels
search site search, SEO

Comments

@orbeckst
Copy link
Member

According to https://www.sitemaps.org/protocol.html#index

Note: A Sitemap index file can only specify Sitemaps that are found on the same site as the Sitemap index file. For example, http://www.yoursite.com/sitemap_index.xml can include Sitemaps on http://www.yoursite.com but not on http://www.example.com or http://yourhost.yoursite.com.

That means we cannot include docs.mdanalysis.org, userguide.mdanalysis.org with www.mdanalysis.org in the sitemapindex.xml.

Apparently, the algolia crawler does not care and still finds them (see algolia/docsearch-configs#2103) but technically this is not legal and we should change it

  • remove the doc and userguide sitemaps from the index
  • add them separately to the algolia config
@orbeckst orbeckst added the search site search, SEO label Jul 17, 2020
@orbeckst
Copy link
Member Author

I am not 100% sure if we need "proove ownership" for "cross submits" as detailed in https://www.sitemaps.org/protocol.html#location, namely our robots.txt file might need to explicitly list the sitemaps for doc and userguide. Or are these sitemaps ok because they are only accessed from the actual docs.mdanalysis.org/.... url?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
search site search, SEO
Projects
None yet
Development

No branches or pull requests

1 participant