Match v1 config interface to new one #4456
Conversation
|
Isn't it better to merge this PR into #4355 so, it's easier to see the changes. Then, after that PR is merged, we can change merging branch again. What do you think? |
|
@humitos yeah, but unfortunately the branch is in my fork, so I can't change the base branch here :/ |
stsewd
force-pushed
the
build-config-v2-implementation
branch
from
6fb74d6 to
cef7c03
Compare
agjohnson
added
the
PR: work in progress
|
I tried to reset the base, but i'm not sure it worked. Looks like a manual rebase is in order. This sounds like you are going to implement all of these changes in one PR, which I would advise against. This is going to make another PR that takes a lot of effort to review. I'd rather we break this up into discrete chunks for each feature. |
stsewd
force-pushed
the
build-config-v2-implementation
branch
from
2afc899 to
5b37760
Compare
|
@agjohnson I just did the rebase to change the base, yeah, I think I can revert the latest change and skip the new tests (those were very helpful while refactoring btw p:). The other changes are only related to changing the v1 interface to match the new one. |
This reverts commit 5b37760.
stsewd
changed the title
| @@ -0,0 +1,37 @@ | |||
| """Models for the response of the configuration object.""" | |||
There was a problem hiding this comment.
We can refactor these to be dataclasses when we are fully running py3 😎
There was a problem hiding this comment.
You can create an issue for this and add it to the Refactor milestone.
| requirements_file_path = self.config.requirements_file | ||
| if not requirements_file_path: | ||
| requirements_file_path = self.config.python.requirements | ||
| if not requirements_file_path and requirements_file_path != '': |
There was a problem hiding this comment.
This is needed for the new behavior of the requirements file, anyway, this doesn't affect the current behavior of v1 (v1 config returns null here).
There was a problem hiding this comment.
I want to echo what I said regarding using explicit ignore for this.
Also, self.config.python.requirements could return IGNORE when the YAML contains '' (in case we won't change that for the explicit way).
There was a problem hiding this comment.
Adding the another solution in #4523
| @@ -295,7 +296,8 @@ def install_user_requirements(self): | |||
| '--exists-action=w', | |||
| '--cache-dir', | |||
| self.project.pip_cache_path, | |||
| '-r{0}'.format(requirements_file_path), | |||
| '-r', | |||
| requirements_file_path, | |||
There was a problem hiding this comment.
I split those to make it easy to test, nothing breaking anyway
readthedocs/projects/tasks.py
Outdated
| @@ -466,18 +472,20 @@ def run_build(self, docker, record): | |||
| self.update_documentation_type() | |||
|
|
|||
| python_env_cls = Virtualenv | |||
| if self.config.use_conda: | |||
| if self.config.conda: | |||
There was a problem hiding this comment.
Conda is None now, if the user is using conda this is a dictionary
There was a problem hiding this comment.
Check against explicit None here.
if self.config.conda is not None:
|
Looks like v1 is still working with this PR, I'm testing v2 a little too. |
|
And v2 is tested too! Just need to fix two minor things, but those are covered with the test that I'm skipping right now. |
| raw_python['setup_py_install']) | ||
|
|
||
| # Validate setup_py_path. | ||
| if 'setup_py_path' in raw_python: |
There was a problem hiding this comment.
In V1 we were validating that the path to setup.py exists and here we are removing this check.
There was a problem hiding this comment.
This option was never implemented in v1, we are considering implementing this in v2 #4354
| validate_string(extra_name) | ||
| ) | ||
| if not python['install_with_pip']: | ||
| python['extra_requirements'] = [] |
There was a problem hiding this comment.
I don't think it should affect, but this checking will be new for V1.
There was a problem hiding this comment.
Yeah, we were doing this validation in the rtd code before.
| @@ -451,11 +444,14 @@ def validate_conda(self): | |||
| self.PYTHON_INVALID_MESSAGE, | |||
There was a problem hiding this comment.
I think this message is wrong. It will say
"python" section must be a mapping.
but we are under conda section.
There was a problem hiding this comment.
Yeah, this was ported from the previous config file, I'm not sure if this is in the scope of this PR, but now we have a validate_dict function for this cases.
There was a problem hiding this comment.
If it's a problem, and won't be fixed in this PR, please open an issue for this so we don't forget to fix it.
| """True if the project use Conda.""" | ||
| return self._config.get('conda') is not None | ||
| requirements = self._config['requirements_file'] | ||
| self._config['python']['requirements'] = requirements |
There was a problem hiding this comment.
Are we sure that self._config['python'] will always exists and be a dictionary? In the init we are not defining it.
There was a problem hiding this comment.
Also, I think this method may return None, right?
There was a problem hiding this comment.
Yeah, we always have a valid dict for self._config['python']. How is that? We always need to call to validate before using any of this properties, there is a docstring about that. The method always returns a valid object
| @@ -212,7 +227,7 @@ def test_use_system_site_packages_defaults_to_false(): | |||
| build = get_build_config({'python': {}}, get_env_config()) | |||
| build.validate() | |||
| # Default is False. | |||
| assert not build.use_system_site_packages | |||
| assert not build.python.use_system_site_packages | |||
There was a problem hiding this comment.
What's the correct here? use_system_site_packages or system_packages?
It seems that we are using all the other names from V2 but this one only from V1. I think we should call it as in V2 (https://github.com/rtfd/readthedocs.org/pull/4451/files#diff-33bbde7fca852e881b51aa90f9f86b3fR83) build.python.system_packages
There was a problem hiding this comment.
use_system_site_packages is from the main interface. Yeah, system_packages is short
There was a problem hiding this comment.
But we are calling this as a property, so use_system_packages is more descriptive for me, what do you think?
There was a problem hiding this comment.
If the change it's just search and replace, I'd go for it. Why? Because I think that we can reflect the same names in the yaml file than in the Python interface.
It's easier to follow the mapping. That's all.
There was a problem hiding this comment.
So, it should be use_system_packages or just system_packages?
| update_docs = self.get_update_docs_task() | ||
| assert update_docs.config.build.image == build_image | ||
|
|
||
| def test_custom_build_image(self, checkout_path, tmpdir): |
There was a problem hiding this comment.
What would happen if they define a build image in the YAML and also there is a project.container_image?
There was a problem hiding this comment.
container_image takes precedence, there is a test for that in the config module, I didn't want to repeat all the tests here, just to make sure we are using the values
| self.create_config_file( | ||
| tmpdir, | ||
| { | ||
| 'python': {'requirements': ''} |
| args, kwargs = run.call_args | ||
|
|
||
| assert 'setup.py' in args | ||
| assert 'install' in args |
There was a problem hiding this comment.
Why not use assert_called_with_args here? or assert_has_calls?
There was a problem hiding this comment.
The command is very large ['/path/to/pip', 'install', '--cache'...]
| args, kwargs = run.call_args | ||
|
|
||
| assert 'setup.py' not in args | ||
| assert 'install' in args |
|
|
||
| assert 'setup.py' not in args | ||
| assert 'install' in args | ||
| assert '.[docs]' in args |
stsewd
force-pushed
the
build-config-v2-implementation
branch
from
564b966 to
4baf899
Compare
|
It seems that you This PR in particular is too big and it's complicated to follow and review over and over. Anyway, I think we are good here. |
Yeah, that was a mistake :/ instead of making a new commit I did an amend... |
|
I didn't see anything major, so we can probably just merge this and QA this next week before doing a release! |
This PR is based on #4355 and should be merged after that.
This will implement the actual logic of v2 and use the same nametuples for v1.
New changes are from ed1ac25
I'm using pytest flavor for the tests.
Closes #4220