A way to disable installation of default requirements

[Jackenmen]

Details

• Read the Docs project URL: https://readthedocs.org/projects/red-discordbot
• Build URL (if applicable): https://readthedocs.org/projects/red-discordbot/builds/13839524/
• Read the Docs username (if applicable): jack1142

Expected Result

I expect to have a way to not have RTD install the default requirements.

Actual Result

In an effort to speed up the project's builds, I looked through RTD's build logs and noticed that a not-insignificant amount of time is taken by RTD installing its "default" requirements. Later, a lot of these dependencies have to be installed again because my project is using different versions (generally newer ones as it seems from #5201 that they are quite outdated) of those libraries causing it to take almost double the time on pip install-ing the dependencies.

While the default build might need these dependencies, it does not appear that they are needed when I provide a custom conf.py. However, even if I need some of them, I could probably just determine and maintain such a list on my own and put any missing requirements as part of the requirements defined in .readthedocs.yml

[humitos]

Hi @jack1142! In #8103 we are talking about a new builder that won't install any dependency. We don't have an ETA about when it's going to be implemented, but that's the direction we are trying to follow going forward.

[humitos]

We are working in a POC that won't execute any command by default and could be useful for this use case: #9150

[Jackenmen]

I've noticed that this feature is now shipped as a beta. It does seem that once the issues such as the non-working flyout menu get resolved and it becomes stable, this would be a viable solution but I do wonder if in the future it's going to be possible to just override the default install job instead of all the build commands?
Right now I see that the "Extend the build process" section of the documentation mentions that currently, it's not possible to replace pre-defined jobs but it's a bit unclear whether there are plans to change this. And if there are, is there any issue I could track?

[Jackenmen]

Either way, the new build.commands feature works wonders for my project, simple change to the newer build process and a couple of manual commands and I'm set with 30 seconds improvement in build time pretty much for free, awesome!
Cog-Creators/Red-DiscordBot#5773

Can't really merge it until the rough edges (I only can notice the lack of a flyout menu, the search might be slower but I'm unsure because I think might be working somewhat differently on PR builder builds) but it's a great advancement and I can't wait to be able to take advantage of it :)

Edit: I've missed the unchanged version_slug variable as well which is used to show that a page was created from a pull request + we use it to warn about usage of latest docs instead of stable for our project. Generally, everything that RTD automatically changed in conf.py I suppose.

[humitos]

@jack1142

I do wonder if in the future it's going to be possible to just override the default install job instead of all the build commands?
Right now I see that the "Extend the build process" section of the documentation mentions that currently, it's not possible to replace pre-defined jobs but it's a bit unclear whether there are plans to change this. And if there are, is there any issue I could track?

You are correct. Currently, this is not possible. However, we have been discussing internally how to allow this in the future. We do have a plan mostly finished but we want to keep collecting some data about how people are using build.jobs and build.commands before closing it and starting implementing it.

We think users could define a backend builder and override some particular commands from that backend. An example of what we have talked and what could potentially be a solution:

# .readthedocs.yaml

build:
  backend: readthedocs-sphinx  # may comes from PyPI
  jobs:
    install: null
    build:
      - sphinx-build -W -b html docs/ _readthedocs/html

To be able to support something like this, we first need to define "The Build Contract ™️". This contract will define what the build.backend builder expects as input, what will be its output, and what Read the Docs expects as an input once the build finishes. This is being tracked at #9088

the search might be slower but I'm unsure because I think might be working somewhat differently on PR builder builds)

Full-Text search is not yet integrated. The results that you are seeing when you search are the default from Sphinx, but they are not using the Read the Docs' search backed yet. We are working on that at #9307

Edit: I've missed the unchanged version_slug variable as well which is used to show that a page was created from a pull request + we use it to warn about usage of latest docs instead of stable for our project. Generally, everything that RTD automatically changed in conf.py I suppose.

Exactly. There is no integrations at all currently when using build.commands. The "External version warning" comes from this Sphinx extension that we install automatically on behalf of the user (https://github.com/readthedocs/readthedocs-sphinx-ext), so you could install it and configure it by yourself when using build.commands.

[humitos]

I do wonder if in the future it's going to be possible to just override the default install job instead of all the build commands?

You are correct. Currently, this is not possible. However, we have been discussing internally how to allow this in the future. We do have a plan mostly finished but we want to keep collecting some data about how people are using build.jobs and build.commands before closing it and starting implementing it.

With the work we are doing on the new addons repository https://github.com/readthedocs/addons/ we are getting closer to be able to support this.

[humitos]

I expect to have a way to not have RTD install the default requirements.

This is going to be the default on August 7th. Read more about it at https://blog.readthedocs.com/python-core-requirements-changed/

I can enable it on your project before that date if you want.

[Jackenmen]

Read more about it at blog.readthedocs.com/python-core-requirements-changed

Do I understand correctly that adding python.install (with either a requirements or method: pip entry, I'd hope) would override the installation of "core requirements"? Or would the latest version of sphinx (which, by the way, is incompatible with the latest RTD theme at the current time) and readthedocs-sphinx-ext be installed before that either way (I assume mkdocs core requirement is only installed for mkdocs-based documentation so it wouldn't be installed regardless)?

I can enable it on your project before that date if you want.

That would be appreciated, the project is linked in the issue description and this account is linked up with the mentioned RTD user that has maintainer perms on that project, let me know if you'll need more from me to verify that I have maintainer rights.

[humitos]

For projects using Sphinx, it will execute:

pip install sphinx readthedocs-sphinx-ext

for projects using MkDocs, it will execute:

pip install mkdocs

If you want to install different versions, you can use a .readthedocs.yaml file with a requirements.txt.

That would be appreciated, the project is linked in the issue description

Enabled.