Docs are not building #382

andrewtavis · 2024-10-16T12:25:37Z

Terms

I have searched all open bug reports
I agree to follow Scribe-Data's Code of Conduct

Behavior

Something that I've noticed recently is that the docs have stopped building successfully. The original error message was:

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/m2r2.py", line 611, in run
    path = nodes.reprunicode(path)
AttributeError: module 'docutils.nodes' has no attribute 'reprunicode'
The full traceback has been saved in /tmp/sphinx-err-9j4a3arm.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

Looking into it, reprunicode was removed from Sphinx, one of the dependencies, and an initial attempt to fix this was to update the dependencies with sphinx-rtd-theme>=3.0.0. The new error is:

Running Sphinx v7.4.7
loading translations [en]... done
making output directory... done
Converting `source_suffix = '.rst'` to `source_suffix = {'.rst': 'restructuredtext'}`.

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/cmd/build.py", line 332, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/application.py", line 292, in __init__
    self._init_builder()
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/application.py", line 365, in _init_builder
    self.builder.init()
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/builders/html/__init__.py", line 226, in init
    self.init_templates()
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/builders/html/__init__.py", line 274, in init_templates
    theme_factory = HTMLThemeFactory(self.app)
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/theming.py", line 173, in __init__
    self._load_additional_themes(app.config.html_theme_path)
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/theming.py", line 185, in _load_additional_themes
    abs_theme_path = path.abspath(path.join(self._app.confdir, theme_path))
  File "/home/docs/.asdf/installs/python/3.9.19/lib/python3.9/posixpath.py", line 90, in join
    genericpath._check_arg_types('join', a, *p)
  File "/home/docs/.asdf/installs/python/3.9.19/lib/python3.9/genericpath.py", line 152, in _check_arg_types
    raise TypeError(f'{funcname}() argument must be str, bytes, or '
TypeError: join() argument must be str, bytes, or os.PathLike object, not 'module'

Exception occurred:
  File "/home/docs/.asdf/installs/python/3.9.19/lib/python3.9/genericpath.py", line 152, in _check_arg_types
    raise TypeError(f'{funcname}() argument must be str, bytes, or '
TypeError: join() argument must be str, bytes, or os.PathLike object, not 'module'
The full traceback has been saved in /tmp/sphinx-err-k8_r5pgy.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

Suggestions on this and a PR would be welcome :)

The text was updated successfully, but these errors were encountered:

Khushalsarode · 2024-10-16T14:44:36Z

@andrewtavis shall I?
Look into this?

andrewtavis · 2024-10-16T16:27:43Z

Would be great, @Khushalsarode :)

andrewtavis · 2024-10-16T23:03:40Z

CC also @AnishereMariam here, as your changes to the docs likely won't be reflected until this is figured out :)

OmarAI2003 · 2024-10-18T05:49:37Z

Hey! I'd love to help with this issue in any way I can. Let me know how I can assist!

AnishereMariam · 2024-10-18T06:37:19Z

I have been trying to debug this all night. I keep hitting roadblocks. @Khushalsarode, can we please have a discussion session on Matrix regarding this issue? I'd like us to fix it soon. @OmarAI2003, your input is highly welcomed. We can all discuss on Matrix if that is fine with you.

andrewtavis · 2024-10-18T06:43:53Z

Let me know if you all need support!

OmarAI2003 · 2024-10-18T07:36:29Z

I have been trying to debug this all night. I keep hitting roadblocks. @Khushalsarode, can we please have a discussion session on Matrix regarding this issue? I'd like us to fix it soon. @OmarAI2003, your input is highly welcomed. We can all discuss on Matrix if that is fine with you.

Thanks for the update! I sent you a message on Matrix, so we can discuss it there when you have a moment. Looking forward to collaborating!

OmarAI2003 · 2024-10-18T08:33:39Z

Hi @andrewtavis,Hi everyone,

I wanted to update you on the progress with the Sphinx documentation build .

After making adjustments to the conf.py file, the documentation now builds successfully. However, I've encountered a few warnings and errors that need to be addressed:

Import Errors: There was a failure to import the scribe_data.wikipedia.process_wiki module. This seems to be related to the removal of the get_language_words_to_ignore function from utils.py in PR #402 , as it was dependent on the old JSON structure.
Docstring Issues: The format_sublanguage_name function, which I defined in the same PR, has raised a warning regarding an unknown section in its docstring. will I need to adjust the docstring to fit a specifec format?
Raw Directive Errors: There are multiple instances where the docstring is missing a content block for the "raw" directive.
Theme Warning: Lastly, there's a warning about an unsupported theme option related to display_version.

I'm want to keep working on this, so any suggestions or insights would be greatly appreciated!

Thanks!

andrewtavis · 2024-10-18T08:44:06Z

Thanks for the efforts here, @OmarAI2003!

Re import errors, please replace the words to replace and words to ignore with empty list declarations, so [].

Re docstring error, yes we need to have all the docstrings with the same format so they're parsed correctly by auto documentation, so they all need to be numpy docstring form. I think I fixed the format, so it's a section. We have "Raises" in many docstrings, so it's likely "Example" that's problematic. Can you put the example in the description in the top and see if it works?

I'm not sure right now on the other two, but hopefully the docs will build from there and we can open new issues for those?

Thanks and let us know how it's going 😊

OmarAI2003 · 2024-10-18T09:41:10Z

Another update on the documentation build process. I've addressed the import errors by replacing the references to the removed functions with empty list declarations, as you suggested.

Regarding the other two issues you mentioned @andrewtavis , since I've successfully built the documentation without any errors or warnings, it seems we may not need to open new issues at this time.

Khushalsarode · 2024-10-18T12:00:08Z

@andrewtavis Documentation Running without error now

Firstly when I tried to run the documentation using make html file:

but it showed path error :

sphinx-autobuild source _build/html
[sphinx-autobuild] Starting initial build
[sphinx-autobuild] > python -m sphinx build source _build/html
Running Sphinx v8.1.3
WARNING: Calling get_html_theme_path is deprecated. If you are calling it to define html_theme_path, you are safe to remove that code.
loading translations [en]... done
Converting source_suffix = '.rst' to source_suffix = {'.rst': 'restructuredtext'}.
[autosummary] generating autosummary for: index.rst, notes.rst, scribe_data\check_language_data.rst, scribe_data\cli.rst, scribe_data\index.rst, scribe_data\language_data_extraction\index.rst, scribe_data\load\data_to_sqlite.rst, scribe_data\load\index.rst, scribe_data\load\send_dbs_to_scribe.rst, scribe_data\load\update_files\index.rst, ..., scribe_data\wikidata\check_query\query.rst, scribe_data\wikidata\check_query\sparql.rst, scribe_data\wikidata\index.rst, scribe_data\wikidata\query_data.rst, scribe_data\wikidata\query_profanity.rst, scribe_data\wikipedia\extract_wiki.rst, scribe_data\wikipedia\gen_autosuggestions.rst, scribe_data\wikipedia\index.rst, scribe_data\wikipedia\process_wiki.rst, scribe_data\wiktionary\index.rst
building [mo]: targets for 0 po files that are out of date
writing output... 
building [html]: targets for 31 source files that are out of date
updating environment: [new config] 31 added, 0 changed, 0 removed
reading sources... [  6%] notes
Exception occurred:
  File "C:\Users\HP.KHUSHALSARODE.000\Desktop\programs\outreachy\Scribe-Data\venv\Lib\site-packages\m2r2.py", line 611, in run
    path = nodes.reprunicode(path)
           ^^^^^^^^^^^^^^^^^
AttributeError: module 'docutils.nodes' has no attribute 'reprunicode'
The full traceback has been saved in C:\Users\HPKHUS~1.000\AppData\Local\Temp\sphinx-err-3o77lkk5.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!Sphinx exited with exit code: 2
The server will continue serving the build folder, but the contents being served are no longer in 
sync with the documentation sources. Please fix the cause of the error above or press Ctrl+C to stop the server.
[sphinx-autobuild] Serving on http://127.0.0.1:8000
[sphinx-autobuild] Waiting to detect changes..

There was some package issue that docutils was not compatible with m2r2
pip install docutils==0.16
upgraded the module but the error persisted as: server was emitting: page not found
error:

building [mo]: targets for 0 po files that are out of date
writing output... 
building [html]: targets for 31 source files that are out of date
updating environment: [new config] 31 added, 0 changed, 0 removed
reading sources... [  6%] notes
Exception occurred:
  File "C:\Users\HP.KHUSHALSARODE.000\Desktop\programs\outreachy\Scribe-Data\venv\Lib\site-packages\m2r2.py", line 611, in run
    path = nodes.reprunicode(path)
           ^^^^^^^^^^^^^^^^^
AttributeError: module 'docutils.nodes' has no attribute 'reprunicode'

then read about: recommonmark for handling markdown or MyST-Parser as native Markdown support in Sphinx (introduced in Sphinx 4.0)

changes made in config.py

Before:

line 38: 
extensions = [
    "m2r2",
    "sphinx.ext.autodoc",
    "numpydoc",
    "sphinx.ext.viewcode",
    "sphinx.ext.imgmath",
]
 line 87
source_suffix = ".rst"

After:

line 38: 
extensions = [
    #"m2r2",
    'recommonmark'
    "sphinx.ext.autodoc",
    "numpydoc",
    "sphinx.ext.viewcode",
    "sphinx.ext.imgmath",
]
 line 87
#source_suffix = ".rst"
line 89:
source_suffix = {
    '.rst': 'restructuredtext',
    '.md': 'markdown',
}

Tried by autobuild and make.bat html

autobuild:

installed: pip install sphinx-autobuild
sphinx-autobuild source _build/html

build succeeded, 1 warning.

The HTML pages are in _build\html.
[sphinx-autobuild] Serving on http://127.0.0.1:8000

make.bat html
python -m http.server

build succeeded, 1 warning.

The HTML pages are in _build\html.
[sphinx-autobuild] Serving on http://127.0.0.1:8000

updated requirement.txt

andrewtavis · 2024-10-18T12:19:38Z

@OmarAI2003, does the above also get solved by your PR?

Khushalsarode · 2024-10-18T12:28:02Z

@OmarAI2003, does the above also get solved by your PR?

ya! saw his PR solving issue, that why I only just shared the description to issue.

OmarAI2003 · 2024-10-18T12:44:29Z

@OmarAI2003, does the above also get solved by your PR?

The work in my PR was based solely on the modules listed in the requirements.txt file, and when I ran sphinx-build -b html docs/source/ build/, everything worked without any errors or warnings. However, when I tried to reproduce the issue he encountered, I downloaded sphinx-autobuild and ran the command. This time, it generated some warnings, specifically related to docstring formatting (e.g., missing content blocks for the raw directive and NumpyDoc issues) and an unsupported theme option (display_version). These warnings weren't present in my initial build process but appeared when using sphinx-autobuild.

Khushalsarode · 2024-10-18T15:45:40Z

yes! @OmarAI2003
Docs were building correctly with come warning: So I commented the respective statements in conf.py as method is being deprecated and after that it build with 0 warning 0 error!
"display_version": True, and html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]

OmarAI2003 · 2024-10-18T15:54:34Z

Thanks, @Khushalsarode! This is indeed a correct and valid edit for autobuild, and I've included it in a later commit.

andrewtavis · 2024-10-19T11:49:58Z

Docs are still failing, @OmarAI2003:

Running Sphinx v7.4.7
loading translations [en]... done

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/registry.py", line 453, in load_extension
    mod = import_module(extname)
  File "/home/docs/.asdf/installs/python/3.9.19/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'recommonmark'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/cmd/build.py", line 332, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/application.py", line 253, in __init__
    self.setup_extension(extension)
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/application.py", line 429, in setup_extension
    self.registry.load_extension(self, extname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/scribe-data/envs/latest/lib/python3.9/site-packages/sphinx/registry.py", line 456, in load_extension
    raise ExtensionError(__('Could not import extension %s') % extname,
sphinx.errors.ExtensionError: Could not import extension recommonmark (exception: No module named 'recommonmark')

Extension error:
Could not import extension recommonmark (exception: No module named 'recommonmark')

Command for this is:

python -m sphinx -T -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html

OmarAI2003 · 2024-10-19T12:11:01Z

Yeah @andrewtavis , this is because the edits Kushal suggested require the recommonmark module to be installed in order to build using sphinx-autobuild. If autobuild isn't necessary, we can remove these edits or otherwise install recommonmark.

The (Bash) command you suggested:
python -m sphinx -T -b html -d _build/doctrees -D language=en . $READTHEDOCS_OUTPUT/html
works for me (on the main branch now after I pulled your edits) since I installed Kushal's modules.

OmarAI2003 · 2024-10-19T12:16:13Z

I just copied your command and pasted it and it worked as shown in the image. What do you recommend—removing the edits or installing the recommonmark module?

andrewtavis · 2024-10-19T12:54:40Z

Let's remove the edits so that we can keep it as simple as possible :) If we don't need a dependency, let's not include it. Can you send along a PR?

OmarAI2003 · 2024-10-19T13:07:22Z

Alright I will

andrewtavis · 2024-10-19T14:29:11Z

Thanks for the great work here, @OmarAI2003, and also for the conversation and collaboration in fixing everything, all!

@mhmohona: Let's keep in mind that the docs are slowly becoming wayyyy out of date with all of the work. Maybe you can do a scan and make some good first issues to fix things in them?

andrewtavis · 2024-10-19T14:29:23Z

Others are welcome to check the docs and open issues as well!

OmarAI2003 · 2024-10-22T19:19:18Z

@mhmohona: Let's keep in mind that the docs are slowly becoming wayyyy out of date with all of the work. Maybe you can do a scan and make some good first issues to fix things in them?

Hi Andrew, I reached out to @mhmohona and had a brief chat about the documentation. I've also submitted this PR #465

andrewtavis added bug Something isn't working help wanted Extra attention is needed hacktoberfest Included as a part of Hacktoberfest labels Oct 16, 2024

andrewtavis added this to Scribe Board Oct 16, 2024

github-project-automation bot moved this to Todo in Scribe Board Oct 16, 2024

andrewtavis assigned Khushalsarode Oct 16, 2024

andrewtavis moved this from Todo to In Progress in Scribe Board Oct 16, 2024

andrewtavis assigned AnishereMariam and OmarAI2003 Oct 18, 2024

OmarAI2003 mentioned this issue Oct 18, 2024

Fix/sphinx docs build issue #418

Merged

2 tasks

OmarAI2003 mentioned this issue Oct 19, 2024

building the docs based on the modules in requirements.txt #438

Merged

2 tasks

andrewtavis closed this as completed Oct 19, 2024

github-project-automation bot moved this from In Progress to Done in Scribe Board Oct 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docs are not building #382

Docs are not building #382

andrewtavis commented Oct 16, 2024

Khushalsarode commented Oct 16, 2024 •

edited

Loading

andrewtavis commented Oct 16, 2024

andrewtavis commented Oct 16, 2024

OmarAI2003 commented Oct 18, 2024

AnishereMariam commented Oct 18, 2024

andrewtavis commented Oct 18, 2024

OmarAI2003 commented Oct 18, 2024

OmarAI2003 commented Oct 18, 2024 •

edited

Loading

andrewtavis commented Oct 18, 2024

OmarAI2003 commented Oct 18, 2024 •

edited

Loading

Khushalsarode commented Oct 18, 2024 •

edited

Loading

andrewtavis commented Oct 18, 2024

Khushalsarode commented Oct 18, 2024 •

edited

Loading

OmarAI2003 commented Oct 18, 2024 •

edited

Loading

Khushalsarode commented Oct 18, 2024

OmarAI2003 commented Oct 18, 2024

andrewtavis commented Oct 19, 2024

OmarAI2003 commented Oct 19, 2024 •

edited

Loading

OmarAI2003 commented Oct 19, 2024

andrewtavis commented Oct 19, 2024

OmarAI2003 commented Oct 19, 2024

andrewtavis commented Oct 19, 2024

andrewtavis commented Oct 19, 2024

OmarAI2003 commented Oct 22, 2024

Docs are not building #382

Docs are not building #382

Comments

andrewtavis commented Oct 16, 2024

Terms

Behavior

Khushalsarode commented Oct 16, 2024 • edited Loading

andrewtavis commented Oct 16, 2024

andrewtavis commented Oct 16, 2024

OmarAI2003 commented Oct 18, 2024

AnishereMariam commented Oct 18, 2024

andrewtavis commented Oct 18, 2024

OmarAI2003 commented Oct 18, 2024

OmarAI2003 commented Oct 18, 2024 • edited Loading

andrewtavis commented Oct 18, 2024

OmarAI2003 commented Oct 18, 2024 • edited Loading

Khushalsarode commented Oct 18, 2024 • edited Loading

changes made in config.py

andrewtavis commented Oct 18, 2024

Khushalsarode commented Oct 18, 2024 • edited Loading

OmarAI2003 commented Oct 18, 2024 • edited Loading

Khushalsarode commented Oct 18, 2024

OmarAI2003 commented Oct 18, 2024

andrewtavis commented Oct 19, 2024

OmarAI2003 commented Oct 19, 2024 • edited Loading

OmarAI2003 commented Oct 19, 2024

andrewtavis commented Oct 19, 2024

OmarAI2003 commented Oct 19, 2024

andrewtavis commented Oct 19, 2024

andrewtavis commented Oct 19, 2024

OmarAI2003 commented Oct 22, 2024

Khushalsarode commented Oct 16, 2024 •

edited

Loading

OmarAI2003 commented Oct 18, 2024 •

edited

Loading

OmarAI2003 commented Oct 18, 2024 •

edited

Loading

Khushalsarode commented Oct 18, 2024 •

edited

Loading

Khushalsarode commented Oct 18, 2024 •

edited

Loading

OmarAI2003 commented Oct 18, 2024 •

edited

Loading

OmarAI2003 commented Oct 19, 2024 •

edited

Loading