Extension sphinxcontrib.bibtex doesn't work, nor can it be built

[jmlarson1]

Details

• Read the Docs project URL:
  https://readthedocs.org/projects/libensemble/

• Build URL (if applicable):
  https://readthedocs.org/projects/libensemble/builds/9934812/

Expected Result

I am trying to use the sphinxcontrib.bibtex extension in my conf.py. It is unknown.

I have included

   - sphinxcontrib.bibtex
   - sphinxcontrib-bibtex

in conda/environment.yml and in docs/requirements.txt. In my .readthedocs.yml, I have also included

conda:
    file: conda/environment.yml
    environment: environment.yml

build:
    image: latest

python:
    version: 3.6
    pip_install: true
    setup_py_install: true
    install:
      - method: pip
      - sphinxcontrib.bibtex
      - sphinxcontrib-bibtex
      - requirements: docs/requirements.txt
    system_packages: true

In

https://readthedocs.org/dashboard/libensemble/advanced/

I have also set the "Requirements file" field to be docs/requirements.txt.

With all of these places, I would hope that it would be picked up somewhere along the way.

Actual Result

The conda build does report:

Pip subprocess output:
Collecting sphinxcontrib.bibtex
  Downloading https://files.pythonhosted.org/packages/60/de/831ec5de791ba30b842a26e27c479ed34259fb1823aa681d8028c551f4d0/sphinxcontrib_bibtex-1.0.0-py3-none-any.whl

and shows it is successfully installed. Yet, I get an error when building my documentation that:

ModuleNotFoundError: No module named 'sphinxcontrib.bibtex'

[humitos]

Hi @jmlarson1! Thanks for the report. It seems that you are using the syntax for config v2 (https://docs.readthedocs.io/en/latest/config-file/v2.html) but you are not defining version: 2 at the top of the file.

Also, take a look at your python: of that file. You are using a syntax that it's not allowed there: adding dependencies directly there.

[james-d-mitchell]

Also, you might want to add sphinxcontrib.bibtex to the list of extensions in line 61 of your docs/conf.py file.

[jmlarson1]

@james-d-mitchell sphinxcontrib.bibtex is on line 61 of docs/conf.py on the branch debugging/bibtex, which is where the error is.

@humitos Thank you, I have now defined version: 2 and removed the unallowed syntax. My .readthedocs.yml file is:

version: 2

conda:
    environment: conda/environment.yml

sphinx:
    configuration: docs/conf.py

build:
    image: latest

python:
    version: 3.6
    install:
      - requirements: docs/requirements.txt
    system_packages: true

Yet, I still get an error at the end of the build that Could not import extension sphinxcontrib.bibtex (exception: No module named 'sphinxcontrib.bibtex')

https://readthedocs.org/projects/libensemble/builds/9944061/

Note: earlier in the build, during the step

conda env create --quiet --name debugging-bibtex --file conda/environment.yml

sphinxcontrib-bibtex is succesffully built. (But it does not appear to be avialable during the step:

python /home/docs/checkouts/readthedocs.org/user_builds/libensemble/conda/debugging-bibtex/bin/sphinx-build -T -b readthedocs -d _build/doctrees-readthedocs -D language=en . _build/html

Thank you again for your help

[stsewd]

@jmlarson1 when using conda, the python section is ignored. #5388

[jmlarson1]

Thank you @stsewd . I would like to use conda so I now have the .readthedocs.yml file

version: 2

conda:
    environment: conda/environment.yml

sphinx:
    configuration: docs/conf.py

build:
    image: latest

and conda/environment.yml is

dependencies:
 - python>=3.6
 - pip
 - pip:
   - sphinxcontrib-bibtex

The conda env create step successfully builds sphinxcontrib-bibtex, but it is not picked up in the last build step:
https://readthedocs.org/projects/libensemble/builds/9945117/

Is there a way to ensure the environment is used?

Thank you for your help.
Jeff

[stsewd]

@jmlarson1 please try wiping the environment before triggering a build https://docs.readthedocs.io/en/stable/guides/wipe-environment.html

[jmlarson1]

@stsewd I have wiped and I have the same failure:
https://readthedocs.org/projects/libensemble/builds/9945252/

(I even wiped/rebuilt twice for good measure.)
Thank you again

[jmlarson1]

After considerable effort, we were able to build our documentation with sphinxcontrib-bibtex. While the comment above says that "when using conda, the python section is ignored", we didn't observe this in practice (at least in how we were building). This would contradict some of the discussion on #5388.

In fact, when including both a conda section and a python section in .readthedocs.yml, we can use conda to build dependencies and then pip to build the package itself. Without the python section, our package was not built, and autodocs couldn't pick it up (see, for example, the autodoc warnings on this build: https://readthedocs.org/projects/libensemble/builds/9964359/ )

In the end, here is the completed build:
https://readthedocs.org/projects/libensemble/builds/9968068/

And here is the .readthedocs.yml file:

version: 2

conda:
    environment: conda/environment.yml

sphinx:
    configuration: docs/conf.py

build:
    image: latest

formats:
  - pdf

python:
    install:
      - method: pip
        path: .
        extra_requirements:
         - docs

Here is the conda/environment.yml file:

channels:
  - conda-forge
  - defaults

dependencies:
 - sphinxcontrib-bibtex
 - python>=3.6

and docs/requirements.txt contains only

sphinxcontrib-bibtex

We also added

    extras_require={
        'extras': ['scipy', 'nlopt', 'mpi4py', 'petsc', 'petsc4py'],
        'docs': ['sphinxcontrib.bibtex']},

to setup.py

[shuds13]

Note: I think @stsewd comment that the python section is ignored was true in that case as the sub-options were redundant when using Conda. But as we have it now, it is useful to build our package in place.

@jmlarson1

[stsewd]

So, the only part that isn't ignored from the python section is install with pip and setuptools. Python version, requirements file, etc, those are ignored.

Also, I think you can just use the pip section form the conda env file.