projects plugin sub-project watcher for "external" files

[Blacksmoke16]

Context

• `projects` plugin sub-project watcher #6944

Description

As per the related discussion, I'm creating this as a feature request so that ideally the projects plugin can be compatible with mkdocs-gen-files.

An example reproduction of this behavior can be seen in mkdocs-testing.zip.

NOTE: This does require https://crystal-lang.org/install/ as it is needed on path in order to generate the intermediary documentation format used by mkdocstrings-crystal.

1. python3 -m venv venv
2. . venv/bin/activate
3. pip install -r requirements.txt
4. mkdocs serve

From here navigate to the My App project in the nav. And notice you see the docs for class Athena::Foo, along with the #bar method. If you were to go and edit projects/my_app/src/test.cr to remove the fdgfdgfddfg from line 2 and save, you will see it triggers the following, with the browser reloading:

INFO    -  [23:59:58] Detected file changes
INFO    -  Schedule build due to 'projects/my_app/src/test.cr' in '.'
INFO    -  Building documentation...
INFO    -  Documentation built in 0.08 seconds
INFO    -  [23:59:58] Reloading browsers
INFO    -  [23:59:58] Browser connected: http://localhost:8000/my_app/

However you'll notice fdgfdgfddfg is still a part of the docs.

Now if you open your editor to projects/my_app/mkdocs.yml and save that file you'll see when the browser reloads that it now properly reflects your changes in projects/my_app/src/test.cr with the following output:

INFO    -  Schedule build due to 'mkdocs.yml' in 'projects/my_app'
INFO    -  [00:03:30] Detected file changes
INFO    -  Building documentation...
INFO    -  Schedule build due to 'projects/my_app/mkdocs.yml' in '.'
INFO    -  [project://.my_app] Cleaning site directory
INFO    -  [project://.my_app] Building documentation to directory:
           /home/george/downloads/mkdocs-testing/projects/my_app/site
INFO    -  [project://.my_app] Documentation built in 0.26 seconds
INFO    -  Documentation built in 0.46 seconds
INFO    -  [00:03:31] Reloading browsers
INFO    -  [00:03:31] Detected file changes
INFO    -  [00:03:31] Browser connected: http://localhost:8000/my_app/
INFO    -  Building documentation...
INFO    -  Schedule build due to 'projects/my_app/site/assets/_mkdocstrings.css' in '.'
INFO    -  Schedule build due to 'projects/my_app/site/sitemap.xml.gz' in '.'
INFO    -  Schedule build due to 'projects/my_app/site/index.html' in '.'
INFO    -  Schedule build due to 'projects/my_app/site/sitemap.xml' in '.'
INFO    -  Schedule build due to 'projects/my_app/site/404.html' in '.'
INFO    -  [00:03:31] Reloading browsers
INFO    -  [00:03:31] Detected file changes
INFO    -  [00:03:31] Browser connected: http://localhost:8000/my_app/
INFO    -  Building documentation...
INFO    -  [00:03:31] Reloading browsers
INFO    -  [00:03:31] Browser connected: http://localhost:8000/my_app/

The same behavior can be seen if you were to add and edit new doc files within projects/my_app/docs as well. So it seems to me the file watcher events watching docs/ and mkdocs.yml is different than event of "external" files. I.e. non-mkdocs files added to https://www.mkdocs.org/user-guide/configuration/#watch.

Related links

• Requesting a change
• https://squidfunk.github.io/mkdocs-material/plugins/projects/
• https://github.com/oprypin/mkdocs-gen-files
• https://github.com/mkdocstrings/crystal
• https://www.mkdocs.org/user-guide/configuration/#watch
• https://crystal-lang.org/install/

Use Cases

This would make it easier to work on API docs generated from language source code files. Which ultimately provides a good ecosystem to use contextual mkdoc docs along side API docs from the source code.

Visuals

No response

Before submitting

• I have read and followed the change request guidelines.
• I have verified that my idea is a change request and not a bug report.
• I have ensured that, to the best of my knowledge, my idea will benefit the entire community.
• I have included relevant links to the documentation, related issues, and discussions to underline the need for my idea.

[squidfunk]

Thanks for suggesting!

NOTE: This does require https://crystal-lang.org/install/ as it is needed on path in order to generate the intermediary documentation format used by mkdocstrings-crystal.

Sorry, but could you please create a reproduction that does not require us to install some external dependencies. We can only evaluate this, if we don't need to add additional dependencies to our systems. I hope for your understanding. Ideally create a much more minimal reproduction, which will make it easier to isolate the things we need to change in order to add support for the feature you requested.

[Blacksmoke16]

Actually yea, that ended up being a lot easier than I thought it would be...

mkdocs-testing.zip

The setup steps are the same, but now instead of editing projects/my_app/src/test.cr you can edit ./gen_doc_stubs.py. E.g. notice removing the dfgfddfg from line 4 produces a similar output to as when you were editing test.cr with the /my_app/foo/ page not reflecting the change. But once you open projects/my_app/mkdocs.yml and save, it does.

[squidfunk]

Thanks for the reproduction. Indeed, a file change is detected, but since the file is located in the top-level project, only the top-level is rebuilt, since the subproject is not considered tainted, which is correct (read on to understand why). This is also why saving the project-level mkdocs.yml solves the issue. The projects plugin already honors the watch setting in projects, so adding the following lines to my_app will rebuild the project upon saving this file (in your reproduction):

watch:
  - ../../gen_doc_stubs.py

This is not a bug in the projects plugin. To make this work "automagically", the projects plugin would either need to rebuild everything on every change (and defeat its purpose) or try to understand what parts of mkdocs.yml are file references that point outside of the project. This makes no sense from our perspective, since we would need to add support for each third-party plugin setting. A better idea: a plugin should add a setting to automatically add referenced files to the list of watched files, so MkDocs will automatically trigger a rebuild. This does not only benefit the project plugin, but the editing experience without the project plugin. You might raise this to the maintainer of mkdocs-gen-files.

If this does not solve your original case, please provide more information and create another reproduction. Until then, we're considering this issue resolved by configuration.

---

Tip: you can run the projects plugin with the following settings to get more verbose output of a specific project:

plugins:
  - projects:
      log_level: debug

[Blacksmoke16]

Ahhh that makes a lot sense. In the full application I was indeed able move the src/ watch settings to the project's mkdocs.yml files and deff seemed to do the trick! Really appreciate you looking into this 🙏. This'll make the dev experience on my project so much better 💪.