Reevaluating the public API

[just-ero]

Motivation

I have been looking for a tool to help me generate documentation for my .NET library from its XML documentation comments for a while now. I was considering creating my own, but was made aware of DocFX via the DotNet Evolution Discord server. I thought it was great, if a bit over-complicated for my purposes: I simply need some Markdown files which link against one another and have the same look as any https://learn.microsoft.com page.

No issues there, I opened a discussion (#9717) and was made aware that DocFX is indeed capable of formatting Markdown files from its output. "Great," I thought. Until I realized that they are quite badly formatted. Bad for readability, bad for compatibility. Users are already used to looking at Microsoft's documentation pages, so I do not understand why DocFX needs to have a different, less readable format.

Some qualms I have with the format:

• By default, all members' information is included on the same page as the type that contains them; this leads to huge pages where finding the information one needs via scrolling is nearly impossible due to the next point.
• It is very difficult for a user to spot the difference between an H3 and an H4 at a glance, leading to a distinct lack of separation between entries and headers.
  • This can be worked around by specifying "memberLayout": "separatePages", but this leads to another inconsistency: members are listed as separate headers with their summary below them, instead of a header-less table like on the Microsoft documentation pages.
• Extension methods and inherited members are simply laid out in a comma-separated list, which makes it hard to really parse anything meaningful at a glance.

I could go on for a while, but that's not the point of this issue.

Once I realized these (and many more) issues with the formatting, I started thinking about the possibility of simply providing my own formatter. Perhaps I could also parse the YAML output. Luckily, I thought, DocFX is available as a NuGet package. Once I included it and attempted looking at the available API, however, I realized that it is essentially useless to me.

Let me explain why:

• The methods for generating any data all require the user pass it a path to DocFX config a file. Docset.Build, DotnetApiCatalog.GenerateManagedReferenceYamlFiles, the list goes on. It is not possible to pass a raw json string into them, nor is it possible for the user to create an instance of the options used for the generation of the files.
  • This leads to the often impossible requirement of having the config file present on disk just to generate the documentation. I would need to embed the config file into my assembly, extract it to a temporary directory, just to feed it back into the same app.
• It is not possible for a user to provide their own formatter in any kind of capacity. I would need to use reflection just to call internal and private methods which do take some sort of formatting methods.

All this leads me to the following.

Description

I propose that the DocFX team severely reevaluate the use-cases of this library. I want to be able to – in code – generate any kind of format of documentation files from a given set of symbols. For this, some simple steps can be taken:

• Make more of the API public. The amount of things which are unnecessarily internal or private are staggering. Users may want to generate or collect only part of the data.
• Provide interfaces for many, many corners of the library. An IFormat or IFormatter interface is an obvious one to think of.
• Provide public types for all kinds of configs and provide methods which take only instances of these config classes. Taking a path to a config file is not something I have ever seen in other parts of the BCL. Parsing any input into these types of classes should be the user's responsibility.

These practices should already help users immensely in their attempt at using DocFX for generating their own format of documentation.

Discussion and input are greatly appreciated. If you would like to split this issue into multiple smaller ones, feel free. This issue is meant to bring the lack of code-side support into a broader light.

[yufeih]

Same as #8872 which @filzrev once proposed a PR in #9140.

There was a bit of clean up work to do before we can provide a stable public API for the long term. System.Text.Json is a big one, some other changes I'd like to see are mentioned here.

If you are willing to contribute, let's continue the discussion here, perhaps starting with a proposal for the public API? We can start with functionalities already exposed in CLI, and then extend to new features.