Implement Documentation Comments #185
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AST nodes now have a `doc` property that contains any documentation for that node. The documentation for a node is the concatenation of every comment line that directly precedes the node. A blank line of padding is the end of the documentation block from the bottom up. For example: ``` def foo; end def foo; end def foo; end def foo; end ``` Newlines within the doc comment are preserved.
…ts at runtime. Doc comments can now be retrived from arbitrary values using the `doc` method. This also works on functions by capturing the function and passing the captured function as an arguments (_not_ the block argument). For example: ``` def foo; end doc((&foo)) #=> "Docs for foo.\n" ``` Notice the double quotes around the function capture in the call to `doc`. This is to prevent the capture from being interpreted as the block argument for the call.
…o JSON. `DocGenerator` is an AST visitor that plucks out doc comments from Modules, Types, and methods, and emits a JSON structure that matches the type structure of the code. With this commit, other tools should be able to read the JSON and start creating documentation viewers, though the structure and content will likely undergo many changes in the near future.
…ctory for files. `auto_document` is now the default when using the `--docs` flag, and you must specify a directory to scan (just `.` should work for the current directory). `auto_document` also takes care of merging all the documentation into a single JSON blob.
Doc comments are now a special language construct beyond normal comments. Instead of trying to guess at which comments count as documentation, Myst now uses a "special comment" structure to explicitly indicate documentation. To create a doc comment, use `#doc` followed by a reference to the method/type/module/property being documented, and optionally a "return value" of the method. Then, write the content of the documentation on the following lines by starting them with `#|`. All together, a full doc comment could look like this: ``` def each(&block) # ... end ``` This special structure provides a few main benefits: - Documentation within a codebase is more obvious and called out. - Highlighters and code analyzers can expect a consistent structure for documentation. - Documentation can be written anywhere in the code base and attached to objects defined anywhere else. This commit only adds the lexing and parsing of doc comments and does not yet attach them to runtime values.
References can start with a `.` or `#` to disambiguate between static and instance references.
The interpreter now knows how to interpret DocComments, and stores their content in a flat table (the `doc_table`) in the interpreter itself. Users can access documentation for any module, type, method, or property at run time using the `doc` method and providing the absolute path to the value they want. For example, getting documentation on the assert library's `is_falsey` would look like `Assert.Assertion#is_falsey`. I'd like to improve this to allow documentation lookup based on a value. That would help users who are _not_ familiar with a library learn more about it through a REPL or other runtimes. Since that's the primary audience for documentation, it makes sense to try to support their use case the most.
… wrap their target. DocComments are now simply made up of an arbitrary header string and their content. References have been removed for simplicity. Instead, the parser now parses DocComments as wrappers around their target expressions. The interpreter then treats DocComment nodes as passthroughs, leaving whatever value was on the stack to be passed back up to the parent expression. This should allow for both runtime and parse-time interpretation of doc comments. At runtime, the interpreter can recurse through the DocComment node and passthrough the result, saving the documentation into the `doc_table`. At parse-time, the DocGenerator can simply scan for DocComment nodes, then get the target directly from the node. This should remove any ambiguity about what a DocComment is meant to attach to.
…e data. `Doc::Generator` (renamed from `DocGenerator`) is the command line tool users can run to auto-generate documentation from their codebase using the `--docs` flag. Providing a directory automatically scans that directory's content for Myst source files, and merges their contents together into one large Documentation structure. The generator also creates a more consistent and complete structure, using the new `*Doc` classes to represent each type of object (constant, method, clause, module, and type) more accurately and with relevant information. The resulting JSON is a complete picture of the codebase, for objects with and without DocComments on them. This generator still needs testing, but the structure is likely complete for the time being.
faultyserver
force-pushed
the
feature/doc-nodes
branch
from
94ae0e1 to
3dc89b7
Compare
…g for documentation files. Using both a single-level glob (`/**`) as well as a recursive one (`/**/*`) ensures that every file in the tree will be hit at least once. Using `uniq` then ensures that each file is hit exactly once.
…lder. The native folder acts like a "magic" directory that contains no implementation code, but provides objects for the documentation generator to scan for and parse into the documentation blob for the standard library. Using this approach allows the generator to create a single structure for the entire library exposed by Myst, without regard for the implementations of the methods themselves. Files in the `stdlib/native` folder should never be included by Myst programs, as they will be redundant and may possibly cause errors depending on the content of the file.
`.mystdoc.yml` is a file that specifies some metadata to include during documentation generation, including the name of the project, a version string, a homepage url, and a logo url. These can all be used by documentation renderers, then, to provide a more complete, customizable experience for individual projects.
…s static methods.
faultyserver
force-pushed
the
feature/doc-nodes
branch
from
4a1ecff to
317bda8
Compare
faultyserver
force-pushed
the
feature/doc-nodes
branch
from
d17c2c3 to
42463a5
Compare
This documentation should not be considered complete. A more complete explanation of how function invocation works should be added to the top summary.
This should be the last commit for the initial upbringing of native library documentation.
|
I think I'm fine with calling this Good to Go as an initial implementation. There are other features I'd like, including linking to implementations, showing inheritance, etc., but all of those can be added later. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DocComments are specific comments in a codebase that are intended to document some object (a method, type, module, etc.).
DocComments use a distinct syntax to make it explicit that the content is documentation, rather than a code comment. In this sense, it is much like Elixir's
@doc, implying that Documentation != Comments.To create a doc comment, use
#doc. The documentation can be expanded to multiple lines by starting those lines with#|. For example, the assert library provides documentation for its modules and functions:The approach taken in this PR should enable DocComments to be parsed both at runtime (as they are already) as well as statically. Since they a structural part of the AST, DocComment nodes are un-ambiguous with regards to their target.
An actual documentation generator would be the next step for documentation in Myst.