Document the API

[charlax]

Feature Request

It would be great to include API documentation. The Usage docs are great as a walkthrough, but when you want to go fast ("what is the name for the Field parameter for a constant value?"), API docs provide a faster answer.

Another use case: I want to inspect the __fields__ on a model, and had to read the code to get the attribute name I was looking for.

API docs would be such a great addition to this amazing library! Super useful!

[samuelcolvin]

I disagree, documenting everything would be extremely difficult. In the end, the only real source of truth is the code itself. One of the reasons we're all writing python not c is that it's easily readable.

I'd be happy to accept a PR to add info on __fields__ to the model documentation. And many more PRs to extend the documentation in other ways. But I don't think "API docs" as a separate section either auto generated, or hand written are a good idea.

[charlax]

I'm not asking about documenting absolutely everything :) but Field is a good candidate.

The API documentation is usually auto-generated (meaning no inconsistency possible between the code and the doc) and makes it possible for advanced users to go straight to the answers, whereas guides provide a more narrated way of discovering the library.

Here are some good examples of API reference from some major packages:

• https://reactjs.org/docs/react-api.html
• https://flask.palletsprojects.com/en/1.1.x/api/
• https://docs.djangoproject.com/en/3.0/ref/
• https://www.attrs.org/en/stable/api.html
• https://requests.readthedocs.io/en/master/api/
• https://marshmallow.readthedocs.io/en/stable/api_reference.html
• https://jinja.palletsprojects.com/en/2.11.x/api/
• ...

Here are some good arguments in favor of having an API reference:

• It provides immediate access to a comprehensive interface definition, instead of having to follow a narrated guide that only focuses on certain elements.
• It lets the reader discover all the functionalities easily from a single place.

[samuelcolvin]

See this fairly long discussion on the matter.

I'm strongly opposed to multiple different types of documentation, I think there should be one and only one section of documentation on a feature. That's impossible to achieve entirely, but adding a whole new "api" section to the docs makes it impossible.

The API documentation is usually auto-generated

Is that really true? It's not for python itself, or aiohttp, I wonder if it is for many of the projects you list above. As mentioned on the issue above, it generally doesn't work that well.

There's also no widely used way to auto-generate docs in markdown - maybe that's a strong hint that it's not a populate approach anymore.

Even if they were, the value in docs is in the examples and descriptions. That still has to be written even if it's in docstrings, not md files.

---

More generally I know the docs could be better, however I don't much enjoy writing docs myself and I get very little help on it from others.

I maintain pydantic for fun and I don't want to spend a significant amount of my free time writing docs, apparently neither does anyone else.

I setup patreon to see if people would who use pydantic would contribute financially, and thereby pay me to do the boring donkey work like docs. It pays me $13/mo. I charge £100/hr for development, so that would pay me for 6'30" a month to improve the docs (at the current exchange rate).

There's also the problem of upkeep once the docs are written. I find it hard enough to get people to document the features they add or modify in pull requests. That would only get harder if they had to extend the API docs as well as modify the main documentation.

---

In short, I think best to submit a PR to add docs for specific things instead of adding another massive section to the docs.

[charlax]

Hey Samuel, thanks for taking the time to answer.

I understand your position. I would still submit that it goes against what most packages are doing, but that would be a somewhat fallacious argument from authority.

I think there are two discussions.

Whether or not pydantic should have an API reference.

I strongly think it should be the case. You don't. In the end, if I don't manage to convince you that it's a useful thing, there's no point in moving further on this ticket and you can close it because you're the repo owner.

Re. your arguments:

• Some of those projects don't auto-generate the docs.
  • That's true, but some do autogenerate:
    • https://github.com/pallets/flask/blob/master/docs/api.rst
    • https://github.com/psf/requests/blob/master/docs/api.rst
    • https://github.com/marshmallow-code/marshmallow/blob/dev/docs/api_reference.rst
• There's no widely used way to auto-generate in Markdown.
  • That's true, but not really a reason you should not have an API reference.
  • Using Markdown or not for documentation is a pretty heated topic, with lots of good arguments on both sides.
  • AFAIK Sphinx does support Markdown for documentation. Never used it though. autodoc does use the rst format though.

If you agree there should an API reference, how to build it

I'd be happy to help! But it's true that the API absolutely needs to be generated, otherwise, it will be a nightmare to maintain them. Another way would be to just include the main parts of the code in the docs (e.g. the Field constructor).

Anyway, in conclusion, just wanted to say that an open source contributor and maintainer myself, I empathize with your feelings re. the time it represents.

[samuelcolvin]

Thanks for your input, I get where you're coming from.

Let's see what others think, if lots of people support this I'll think about it more.

Requirements:

• follows the same formatting and structure as the following docs - I don't want a mkdocs site with a sphinx site bolted on somehow. It needs to work with the current menu and search.
• easy way of linking from the current docs to the API docs, and the other way
• docs auto-generated from signatures (including type hints) and docstrings
• easy way to decide what's included in the API docs - it'll be no use if every single function and attribute is shown.

There's currently no way to render rst documents within mkdocs, see mkdocs/mkdocs#1817.

So the options are:

1. find a way to render rst in mkdocs while keeping the formatting unchanged - sounds like hell
2. build a new library that parses the AST of a python file and generates a md file
3. improve or coerse an existing py -> md library. I tried a few of the libraries currently in existence a few months ago when building rtoml, but none of them worked for me.
4. include the entire pydantic source code in the docs, nicely highlighted and linkable with links back to the normal docs marked-up. We could then in future extend that to "fold" function definitions, mark private functions as private, go to definition etc. etc.

If we do have to do this, my preference would be option 4, but even this is a big piece of work that would require a library of its own and ~10 hours of work before we could start using it.

[StephenBrown2]

Re: Autogenerating documentation from markdown docstrings, https://github.com/pawamoy/mkdocstrings has sprung up as a mkdocs plugin/extension to do just that, and @pawamoy is currently working on implementing a filter option for more detailed control.

It currently:

• follows the same formatting and structure as regular mkdocs
• allows for easy way of linking from the current docs to the API docs, and the other way
• auto-generates docs from signatures (including type hints) and docstrings (the entire point of it)
• gives an easy way to decide what's included in the API docs, as far as top-level items go, e.g.

  # Markdown header
  ::: pydantic.fields.Field
  

I can try putting together a PR for documenting things, if you or @charlax have a suggestion of classes to doc, Field et al.

[pawamoy]

Funny, seeing pydantic and fastapi docs is what made me want to use mkdocs, and then build mkdocstrings 😄

• follows the same formatting and structure as the following docs - I don't want a mkdocs site with a sphinx site bolted on somehow. It needs to work with the current menu and search.
  mkdocstrings is a plugin for mkdocs so it's completely integrated with it. It supports the Material theme.
• easy way of linking from the current docs to the API docs, and the other way.
  I added a cross-reference ability. As long as your headings are unique (their ids always are but sometimes are suffixed with _1, etc.), you can link back and forth using the classic markdown syntax for references: [title][heading_id] or directly [heading_id][]. Since the generated docs use a heading for every Python object, you can link to them easily, from your markdown pages or from your docstrings. Cross-references will still work even if you move the objects/heading to other pages.
• docs auto-generated from signatures (including type hints) and docstrings. Check. Docstrings must be written in Markdown. mkdocstrings can also parse the Google-style docstring format to generate tables for parameters, etc.
• easy way to decide what's included in the API docs - it'll be no use if every single function and attribute is shown.
  As @StephenBrown2 said, I'm currently working on it 🙂

UPDATE: mkdocstrings now has a member option, check the docs

[spacemanspiff2007]

I would love to see Field documented. It really confused me that there is no documentation of possible Field args in the docs.

[samuelcolvin]

There is documentation on field arguments.

[spacemanspiff2007]

O_o
Sorry I missed it.

[cblegare]

I tend to use sphinx for my doc. Mostly prose, but I like generated api doc to exist so I can hyperlink to it. That being said, there is no objects.inv provided for pydantic. That means I need to manually ignore each class or function (using https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-nitpick_ignore) to suppress warnings in the building of my documentation, since my code contains reference to pydantic.

I think that providing objects.inv would be very nice!

[webknjaz]

FYI there's MyST for Sphinx+Markdown+RST directives now: https://myst-parser.readthedocs.io/.

[shufflebits]

API docs. Gotta have.
The Usage docs tell you some of the things you can do. They don't tell you everything. And they don't tell you what you can't do.
I'm looking at validators, shared validators, and so bump in to allow_reuse=True. What does that do? What would be the effect of not passing it?
And, validator seems to be a decorator. Except when it's not.
All of these things would be answered by an API doc.
If "documenting everything would be extremely difficult", then maybe understanding how to use it is also "extremely difficult". Reverse-engineering the code isn't a good way to find out how to use it. And does not allow us to determine the changes we need to make when upgrading.

[charlax]

Great resource on the topic: https://documentation.divio.com/reference/

[PrettyWood]

I'm not sure what's expected here. If the goal is to have more information on __fields__, I feel like the solution would be to expose a public method with dedicated method fields(My model) like for dataclass. But never explain the internal of pydantic.
Same if some doc is missing on validators PRs are always welcome to explain more how things work.
But I feel like the doc of pydantic is already great and if some things are missing we can just add stuff there.

[Czaki]

I would also love to have API documentation with objects.inv that will simplify cross-linking from other documentation (sphinx based).

[samuelcolvin]

I have plans for this.

It it won't be using sphinx, but rather mkdocstrings which I've used very successfully recently in dirty-equals and watchfiles.

[gwerbin-tive]

+1 for this! A cross-linked API reference page is extremely valuable in nearly every library/framework that I've used.

I assume that @Czaki was specifically talking about setting this up to be compatible with Intersphinx which automagically creates links to external docs for directives like so:

:func:`This is a link to the Numpy docs! <numpy.asarray>`

I unfortunately don't know what's required to make that work (perhaps you just need URLs like /doc/<classname>), but it seems to "just work" with most (all?) RtD/Sphinx docs sites.

[samuelcolvin]

Mkdocstrings can do the same thing I think.

[pawamoy]

Indeed, it's documented here: https://mkdocstrings.github.io/usage/#cross-references-to-other-projects-inventories :)

[Czaki]

But this is the documentation on how to add references. Did mkdocs generate objects.inv file, and if yes, then where could we find it (as it is required to reference from another project to pydantic)?

[samuelcolvin]

Interesting question, @pawamoy would there be a way for others to reference pydantics docs from their docs?

[pawamoy]

That's probably not very clear in the docs, but yes:

Reciprocally, mkdocstrings also allows to generate an inventory file in the Sphinx format. It will be enabled by default if the Python handler is used, and generated as objects.inv in the final site directory.

It means objects.inv would be served at https://pydantic-docs.helpmanual.io/objects.inv. I'll improve the paragraph above in the docs.

[pchemguy]

I second the need for API reference. This is important. It does not mean that every feature needs very detailed docs. At the same time, one of the first things I see is the Field object and no documentation whatsoever on it. This is not right.

[samuelcolvin]

It does: https://pydantic-docs.helpmanual.io/usage/schema/#field-customization

API documentation will be done in V2, this is already stated in the blog about V2.

[pchemguy]

Thanks!

[samuelcolvin]

fixed in #5565, still some work to do, but we're on it.