From 574ebed74bcb7bba6d08b7d3c4c04046a16379dc Mon Sep 17 00:00:00 2001 From: Johannes Kirschbauer Date: Sun, 11 Feb 2024 17:23:03 +0700 Subject: [PATCH 1/5] init: doc comment support Co-authored-by: Shahar Dawn Or --- CHANGELOG.md | 20 +- README.md | 63 +++- doc/migration.md | 119 ++++++ src/comment.rs | 110 ++++++ src/format.rs | 104 ++++++ src/legacy.rs | 116 ++++++ src/main.rs | 339 +++--------------- ...snap => nixdoc__test__arg_formatting.snap} | 0 ...xdoc__test__description_of_lib_debug.snap} | 0 src/snapshots/nixdoc__test__doc_comment.snap | 58 +++ ...test__doc_comment_section_description.snap | 8 + ...p => nixdoc__test__inherited_exports.snap} | 0 ....snap => nixdoc__test__line_comments.snap} | 0 ...doc__main.snap => nixdoc__test__main.snap} | 0 ...ine.snap => nixdoc__test__multi_line.snap} | 0 src/test.rs | 172 +++++++++ test.nix | 37 ++ test/doc-comment-sec-heading.nix | 4 + test/doc-comment.nix | 53 +++ 19 files changed, 916 insertions(+), 287 deletions(-) create mode 100644 doc/migration.md create mode 100644 src/comment.rs create mode 100644 src/format.rs create mode 100644 src/legacy.rs rename src/snapshots/{nixdoc__arg_formatting.snap => nixdoc__test__arg_formatting.snap} (100%) rename src/snapshots/{nixdoc__description_of_lib_debug.snap => nixdoc__test__description_of_lib_debug.snap} (100%) create mode 100644 src/snapshots/nixdoc__test__doc_comment.snap create mode 100644 src/snapshots/nixdoc__test__doc_comment_section_description.snap rename src/snapshots/{nixdoc__inherited_exports.snap => nixdoc__test__inherited_exports.snap} (100%) rename src/snapshots/{nixdoc__line_comments.snap => nixdoc__test__line_comments.snap} (100%) rename src/snapshots/{nixdoc__main.snap => nixdoc__test__main.snap} (100%) rename src/snapshots/{nixdoc__multi_line.snap => nixdoc__test__multi_line.snap} (100%) create mode 100644 src/test.rs create mode 100644 test.nix create mode 100644 test/doc-comment-sec-heading.nix create mode 100644 test/doc-comment.nix diff --git a/CHANGELOG.md b/CHANGELOG.md index 0170e3c..1247066 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,8 +1,26 @@ # Changelog +## Version 3.0.0 + +### New Features + +- **Official Doc-Comments Support:** We've introduced support for official doc-comments as defined in [RFC145](https://github.com/NixOS/rfcs/pull/145). This enhancement aligns nixdoc with our latest documentation standard. + +### Deprecated Features + +- **Legacy Custom Format:** The custom nixdoc format is now considered a legacy feature. We plan to phase it out in future versions to streamline documentation practices. +- We encourage users to transition to the official doc-comment format introduced in this release. +- We will continue to maintain the legacy format, we will not accept new features or enhancements for it. This decision allows for a period of transition to the new documentation practices. + +See [Migration guide](./doc/migration.md) for smooth transition + + by @hsjobeki; co-authored by @mightyiam + + in https://github.com/nix-community/nixdoc/pull/91. + ## 2.7.0 -- Added support to customise the attribute set prefix, which was previously hardcoded to `lib`. +- Added support to customize the attribute set prefix, which was previously hardcoded to `lib`. The default is still `lib`, but you can pass `--prefix` now to use something else like `utils`. By @Janik-Haag in https://github.com/nix-community/nixdoc/pull/97 diff --git a/README.md b/README.md index a593696..978697d 100644 --- a/README.md +++ b/README.md @@ -10,19 +10,72 @@ function set. ## Comment format -Currently, identifiers are included in the documentation if they have -a preceding comment in multiline syntax `/* something */`. +This tool implements a subset of the doc-comment standard specified in [RFC-145/doc-comments](https://github.com/NixOS/rfcs/blob/master/rfcs/0145-doc-strings.md). +But, it is currently limited to generating documentation for statically analyzable attribute paths only. +In the future, it could be the role of a Nix interpreter to obtain the values to be documented and their doc-comments. -Two special line beginnings are recognised: +It is important to start doc-comments with the additional asterisk (`*`) -> `/**` which renders as a doc-comment. + +The content of the doc-comment should be some markdown. ( See [Commonmark](https://spec.commonmark.org/0.30/) specification) + +### Example + +The following is an example of markdown documentation for new and current users of nixdoc. + +> Sidenote: Indentation is automatically detected and should be consistent across the content. +> +> If you are used to multiline-strings (`''`) in nix this should be intuitive to follow. + +````nix +{ + /** + This function adds two numbers + + # Example + + ```nix + add 4 5 + => + 9 + ``` + + # Type + + ``` + add :: Number -> Number -> Number + ``` + + # Arguments + + - a: The first number + - b: The second number + + */ + add = a: b: a + b; +} +```` + +## Custom nixdoc format (Legacy) + +You should consider migrating to the newer format described above. + +See [Migration guide](./doc/migration.md). + +### Comment format (legacy) + +Identifiers are included in the documentation if they have +a preceding comment in multiline syntax `/* something */`. You should consider migrating to the new format described above. + +Two special line beginnings are recognized: * `Example:` Everything following this line will be assumed to be a verbatim usage example. -* `Type:` This line will be interpreted as a faux type signature. +* `Type:` This line will be interpreted as a faux-type signature. These will result in appropriate elements being inserted into the output. -## Function arguments +### Function arguments (legacy) Function arguments can be documented by prefixing them with a comment: diff --git a/doc/migration.md b/doc/migration.md new file mode 100644 index 0000000..fc6e50c --- /dev/null +++ b/doc/migration.md @@ -0,0 +1,119 @@ +# Migration Guide + +Upgrading from nixdoc <= 2.x.x to >= 3.0.0 + +To leverage the new doc-comment features and prepare for the deprecation of the legacy format, follow these guidelines: + +## Documentation Comments + +- Use double asterisks `/** */` to mark comments intended as documentation. This differentiates them from internal comments and ensures they are properly processed as part of the documentation. + +**Example:** + +`lib/attrsets.nix (old format)` +````nix +/* Filter an attribute set by removing all attributes for which the + given predicate return false. + Example: + filterAttrs (n: v: n == "foo") { foo = 1; bar = 2; } + => { foo = 1; } + Type: + filterAttrs :: (String -> Any -> Bool) -> AttrSet -> AttrSet +*/ +filterAttrs = + # Predicate taking an attribute name and an attribute value, which returns `true` to include the attribute or `false` to exclude the attribute. + pred: + # The attribute set to filter + set: + listToAttrs (concatMap (name: let v = set.${name}; in if pred name v then [(nameValuePair name v)] else []) (attrNames set)); +```` + +-> + +`lib/attrsets.nix (new format)` +````nix +/** + Filter an attribute set by removing all attributes for which the + given predicate return false. + + # Example + + ```nix + filterAttrs (n: v: n == "foo") { foo = 1; bar = 2; } + => { foo = 1; } + ``` + + # Type + + ``` + filterAttrs :: (String -> Any -> Bool) -> AttrSet -> AttrSet + ``` + + # Arguments + - **pred**: Predicate taking an attribute name and an attribute value, which returns `true` to include the attribute, or `false` to exclude the attribute. + - **set**: The attribute set to filter +*/ +filterAttrs = + pred: + set: + listToAttrs (concatMap (name: let v = set.${name}; in if pred name v then [(nameValuePair name v)] else []) (attrNames set)); +```` + +## Documenting Arguments + +With the introduction of RFC145, there is a shift in how arguments are documented. While direct "argument" documentation is not specified, you can still document arguments effectively within your doc-comments by writing explicit markdown. + +**Example:** Migrating **Single Argument Documentation** + +The approach to documenting single arguments has evolved. Instead of individual argument comments, document the function and its arguments together. + +```nix +{ + /** + The `id` function returns the provided value unchanged. + + # Arguments + + - **x**: The value to be returned. + + */ + id = x: x; +} +``` + +If arguments require more complex documentation consider starting an extra section per argument + +```nix +{ + /** + The `id` function returns the provided value unchanged. + + # Arguments + + ## **x** + + (...Some comprehensive documentation) + + */ + id = x: x; +} +``` + +**Example:** Documenting Structured Arguments +Structured arguments can be documented (described in RFC145 as 'lambda formals'), using doc-comments. + +```nix +{ + /** + * The `add` function calculates the sum of `a` and `b`. + */ + add = { + /** The first number to add. */ + a, + /** The second number to add. */ + b + }: a + b; +} +``` + +Ensure your documentation comments start with double asterisks to comply with the new standard. The legacy format remains supported for now but will not receive new features. It will be removed once important downstream projects have been migrated. diff --git a/src/comment.rs b/src/comment.rs new file mode 100644 index 0000000..ec28039 --- /dev/null +++ b/src/comment.rs @@ -0,0 +1,110 @@ +use rnix::ast::{self, AstToken}; +use rnix::{match_ast, SyntaxNode}; +use rowan::ast::AstNode; + +/// Implements functions for doc-comments according to rfc145. +pub trait DocComment { + fn doc_text(&self) -> Option<&str>; +} + +impl DocComment for ast::Comment { + /// Function returns the contents of the doc-comment, if the [ast::Comment] is a + /// doc-comment, or None otherwise. + /// + /// Note: [ast::Comment] holds both the single-line and multiline comment. + /// + /// /**{content}*/ + /// -> {content} + /// + /// It is named `doc_text` to complement [ast::Comment::text]. + fn doc_text(&self) -> Option<&str> { + let text = self.syntax().text(); + // Check whether this is a doc-comment + if text.starts_with(r#"/**"#) && self.text().starts_with('*') { + self.text().strip_prefix('*') + } else { + None + } + } +} + +/// Function retrieves a doc-comment from the [ast::Expr] +/// +/// Returns an [Option] of the first suitable doc-comment. +/// Returns [None] in case no suitable comment was found. +/// +/// Doc-comments can appear in two places for any expression +/// +/// ```nix +/// # (1) directly before the expression (anonymous) +/// /** Doc */ +/// bar: bar; +/// +/// # (2) when assigning a name. +/// { +/// /** Doc */ +/// foo = bar: bar; +/// } +/// ``` +/// +/// If the doc-comment is not found in place (1) the search continues at place (2) +/// More precisely before the NODE_ATTRPATH_VALUE (ast) +/// If no doc-comment was found in place (1) or (2) this function returns None. +pub fn get_expr_docs(expr: &SyntaxNode) -> Option { + if let Some(doc) = get_doc_comment(expr) { + // Found in place (1) + doc.doc_text().map(|v| v.to_owned()) + } else if let Some(ref parent) = expr.parent() { + match_ast! { + match parent { + ast::AttrpathValue(_) => { + if let Some(doc_comment) = get_doc_comment(parent) { + doc_comment.doc_text().map(|v| v.to_owned()) + }else{ + None + } + }, + _ => { + // Yet unhandled ast-nodes + None + } + + } + } + // None + } else { + // There is no parent; + // No further places where a doc-comment could be. + None + } +} + +/// Looks backwards from the given expression +/// Only whitespace or non-doc-comments are allowed in between an expression and the doc-comment. +/// Any other Node or Token stops the peek. +fn get_doc_comment(expr: &SyntaxNode) -> Option { + let mut prev = expr.prev_sibling_or_token(); + loop { + match prev { + Some(rnix::NodeOrToken::Token(ref token)) => { + match_ast! { match token { + ast::Whitespace(_) => { + prev = token.prev_sibling_or_token(); + }, + ast::Comment(it) => { + if it.doc_text().is_some() { + break Some(it); + }else{ + //Ignore non-doc comments. + prev = token.prev_sibling_or_token(); + } + }, + _ => { + break None; + } + }} + } + _ => break None, + }; + } +} diff --git a/src/format.rs b/src/format.rs new file mode 100644 index 0000000..1eece5b --- /dev/null +++ b/src/format.rs @@ -0,0 +1,104 @@ +use textwrap::dedent; + +/// Ensure all lines in a multi-line doc-comments have the same indentation. +/// +/// Consider such a doc comment: +/// +/// ```nix +/// { +/// /* foo is +/// the value: +/// 10 +/// */ +/// foo = 10; +/// } +/// ``` +/// +/// The parser turns this into: +/// +/// ``` +/// foo is +/// the value: +/// 10 +/// ``` +/// +/// +/// where the first line has no leading indentation, and all other lines have preserved their +/// original indentation. +/// +/// What we want instead is: +/// +/// ``` +/// foo is +/// the value: +/// 10 +/// ``` +/// +/// i.e. we want the whole thing to be dedented. To achieve this, we remove all leading whitespace +/// from the first line, and remove all common whitespace from the rest of the string. +pub fn handle_indentation(raw: &str) -> Option { + let result: String = match raw.split_once('\n') { + Some((first, rest)) => { + format!("{}\n{}", first.trim_start(), dedent(rest)) + } + None => raw.into(), + }; + + Some(result.trim().to_owned()).filter(|s| !s.is_empty()) +} + +/// Shift down markdown headings +/// +/// Performs a line-wise matching to ' # Heading ' +/// +/// Counts the current numbers of '#' and adds levels: [usize] to them +/// levels := 1; gives +/// '# Heading' -> '## Heading' +/// +/// Markdown has 6 levels of headings. Everything beyond that (e.g., H7) may produce unexpected renderings. +/// by default this function makes sure, headings don't exceed the H6 boundary. +/// levels := 2; +/// ... +/// H4 -> H6 +/// H6 -> H6 +/// +pub fn shift_headings(raw: &str, levels: usize) -> String { + let mut result = String::new(); + for line in raw.lines() { + if line.trim_start().starts_with('#') { + result.push_str(&format!("{}\n", &handle_heading(line, levels))); + } else { + result.push_str(&format!("{line}\n")); + } + } + result +} + +// Dumb heading parser. +pub fn handle_heading(line: &str, levels: usize) -> String { + let chars = line.chars(); + + let mut leading_trivials: String = String::new(); + let mut hashes = String::new(); + let mut rest = String::new(); + for char in chars { + match char { + ' ' | '\t' if hashes.is_empty() => { + // only collect trivial before the initial hash + leading_trivials.push(char) + } + '#' if rest.is_empty() => { + // only collect hashes if no other tokens + hashes.push(char) + } + _ => rest.push(char), + } + } + let new_hashes = match hashes.len() + levels { + // We reached the maximum heading size. + 6.. => "#".repeat(6), + _ => "#".repeat(hashes.len() + levels), + }; + + format!("{leading_trivials}{new_hashes} {rest}") +} diff --git a/src/legacy.rs b/src/legacy.rs new file mode 100644 index 0000000..ce01d2c --- /dev/null +++ b/src/legacy.rs @@ -0,0 +1,116 @@ +use rnix::{ + ast::{AstToken, Comment, Expr, Lambda, Param}, + SyntaxKind, SyntaxNode, +}; +use rowan::ast::AstNode; + +use crate::{ + commonmark::{Argument, SingleArg}, + format::handle_indentation, + retrieve_doc_comment, +}; + +/// Retrieve documentation comments. +pub fn retrieve_legacy_comment(node: &SyntaxNode, allow_line_comments: bool) -> Option { + // if the current node has a doc comment it'll be immediately preceded by that comment, + // or there will be a whitespace token and *then* the comment tokens before it. We merge + // multiple line comments into one large comment if they are on adjacent lines for + // documentation simplicity. + let mut token = node.first_token()?.prev_token()?; + if token.kind() == SyntaxKind::TOKEN_WHITESPACE { + token = token.prev_token()?; + } + if token.kind() != SyntaxKind::TOKEN_COMMENT { + return None; + } + // if we want to ignore line comments (eg because they may contain deprecation + // comments on attributes) we'll backtrack to the first preceding multiline comment. + while !allow_line_comments && token.text().starts_with('#') { + token = token.prev_token()?; + if token.kind() == SyntaxKind::TOKEN_WHITESPACE { + token = token.prev_token()?; + } + if token.kind() != SyntaxKind::TOKEN_COMMENT { + return None; + } + } + + if token.text().starts_with("/*") { + return Some(Comment::cast(token)?.text().to_string()); + } + + // backtrack to the start of the doc comment, allowing only adjacent line comments. + // we don't care much about optimization here, doc comments aren't long enough for that. + if token.text().starts_with('#') { + let mut result = String::new(); + while let Some(comment) = Comment::cast(token) { + if !comment.syntax().text().starts_with('#') { + break; + } + result.insert_str(0, comment.text().trim()); + let ws = match comment.syntax().prev_token() { + Some(t) if t.kind() == SyntaxKind::TOKEN_WHITESPACE => t, + _ => break, + }; + // only adjacent lines continue a doc comment, empty lines do not. + match ws.text().strip_prefix('\n') { + Some(trail) if !trail.contains('\n') => result.insert(0, ' '), + _ => break, + } + token = match ws.prev_token() { + Some(c) => c, + _ => break, + }; + } + return Some(result); + } + + None +} + +/// Traverse directly chained nix lambdas and collect the identifiers of all lambda arguments +/// until an unexpected AST node is encountered. +pub fn collect_lambda_args(mut lambda: Lambda) -> Vec { + let mut args = vec![]; + + loop { + match lambda.param().unwrap() { + // a variable, e.g. `x:` in `id = x: x` + // Single args are not supported by RFC145, due to ambiguous placement rules. + Param::IdentParam(id) => { + args.push(Argument::Flat(SingleArg { + name: id.to_string(), + doc: handle_indentation( + &retrieve_legacy_comment(id.syntax(), true).unwrap_or_default(), + ), + })); + } + // an ident in a pattern, e.g. `a` in `foo = { a }: a` + Param::Pattern(pat) => { + // collect doc-comments for each lambda formal + // Lambda formals are supported by RFC145 + let pattern_vec: Vec<_> = pat + .pat_entries() + .map(|entry| SingleArg { + name: entry.ident().unwrap().to_string(), + doc: handle_indentation( + &retrieve_doc_comment(entry.syntax(), Some(1)) + .or(retrieve_legacy_comment(entry.syntax(), true)) + .unwrap_or_default(), + ), + }) + .collect(); + + args.push(Argument::Pattern(pattern_vec)); + } + } + + // Curried or not? + match lambda.body() { + Some(Expr::Lambda(inner)) => lambda = inner, + _ => break, + } + } + + args +} diff --git a/src/main.rs b/src/main.rs index 3a0b2a1..cfa5618 100644 --- a/src/main.rs +++ b/src/main.rs @@ -21,16 +21,25 @@ //! * extract line number & add it to generated output //! * figure out how to specify examples (& leading whitespace?!) +mod comment; mod commonmark; +mod format; +mod legacy; +#[cfg(test)] +mod test; +use crate::{format::handle_indentation, legacy::retrieve_legacy_comment}; + +use self::comment::get_expr_docs; use self::commonmark::*; +use format::shift_headings; +use legacy::collect_lambda_args; use rnix::{ - ast::{AstToken, Attr, AttrpathValue, Comment, Expr, Inherit, Lambda, LetIn, Param}, + ast::{Attr, AttrpathValue, Expr, Inherit, LetIn}, SyntaxKind, SyntaxNode, }; use rowan::{ast::AstNode, WalkEvent}; use std::fs; -use textwrap::dedent; use std::collections::HashMap; use std::io; @@ -70,9 +79,11 @@ struct DocComment { doc: String, /// Optional type annotation for the thing being documented. + /// This is only available as legacy feature doc_type: Option, /// Usage example(s) (interpreted as a single code block) + /// This is only available as legacy feature example: Option, } @@ -102,127 +113,61 @@ impl DocItem { } } -/// Retrieve documentation comments. -fn retrieve_doc_comment(node: &SyntaxNode, allow_line_comments: bool) -> Option { - // if the current node has a doc comment it'll be immediately preceded by that comment, - // or there will be a whitespace token and *then* the comment tokens before it. We merge - // multiple line comments into one large comment if they are on adjacent lines for - // documentation simplicity. - let mut token = node.first_token()?.prev_token()?; - if token.kind() == SyntaxKind::TOKEN_WHITESPACE { - token = token.prev_token()?; - } - if token.kind() != SyntaxKind::TOKEN_COMMENT { - return None; - } - - // if we want to ignore line comments (eg because they may contain deprecation - // comments on attributes) we'll backtrack to the first preceding multiline comment. - while !allow_line_comments && token.text().starts_with('#') { - token = token.prev_token()?; - if token.kind() == SyntaxKind::TOKEN_WHITESPACE { - token = token.prev_token()?; - } - if token.kind() != SyntaxKind::TOKEN_COMMENT { - return None; - } - } - - if token.text().starts_with("/*") { - return Some(Comment::cast(token)?.text().to_string()); - } - - // backtrack to the start of the doc comment, allowing only adjacent line comments. - // we don't care much about optimization here, doc comments aren't long enough for that. - if token.text().starts_with('#') { - let mut result = String::new(); - while let Some(comment) = Comment::cast(token) { - if !comment.syntax().text().starts_with('#') { - break; - } - result.insert_str(0, comment.text().trim()); - let ws = match comment.syntax().prev_token() { - Some(t) if t.kind() == SyntaxKind::TOKEN_WHITESPACE => t, - _ => break, - }; - // only adjacent lines continue a doc comment, empty lines do not. - match ws.text().strip_prefix('\n') { - Some(trail) if !trail.contains('\n') => result.insert(0, ' '), - _ => break, - } - token = match ws.prev_token() { - Some(c) => c, - _ => break, - }; - } - return Some(result); - } +pub fn retrieve_doc_comment(node: &SyntaxNode, shift_headings_by: Option) -> Option { + // Return a rfc145 doc-comment if one is present + // Otherwise do the legacy parsing + // If there is a doc comment according to RFC145 just return it. + let doc_comment = match node.kind() { + // NODE_IDENT_PARAM: Special case, for backwards compatibility with function args + // In rfc145 this is equivalent with lookup of the lambda docs of the partial function. + // a: /** Doc comment */b: + // NODE_LAMBDA(b:) <- (parent of IDENT_PARAM) + // NODE_IDENT_PARAM(b) + SyntaxKind::NODE_IDENT_PARAM => get_expr_docs(&node.parent().unwrap()), + _ => get_expr_docs(node), + }; - None + doc_comment.map(|doc_comment| { + shift_headings( + &handle_indentation(&doc_comment).unwrap_or(String::new()), + // H1 to H4 can be used in the doc-comment with the current rendering. + // They will be shifted to H3, H6 + // H1 and H2 are currently used by the outer rendering. (category and function name) + shift_headings_by.unwrap_or(2), + ) + }) } /// Transforms an AST node into a `DocItem` if it has a leading /// documentation comment. fn retrieve_doc_item(node: &AttrpathValue) -> Option { - let comment = retrieve_doc_comment(node.syntax(), false)?; let ident = node.attrpath().unwrap(); // TODO this should join attrs() with '.' to handle whitespace, dynamic attrs and string // attrs. none of these happen in nixpkgs lib, and the latter two should probably be // rejected entirely. let item_name = ident.to_string(); - Some(DocItem { - name: item_name, - comment: parse_doc_comment(&comment), - args: vec![], - }) -} - -/// Ensure all lines in a multi-line doc-comments have the same indentation. -/// -/// Consider such a doc comment: -/// -/// ```nix -/// { -/// /* foo is -/// the value: -/// 10 -/// */ -/// foo = 10; -/// } -/// ``` -/// -/// The parser turns this into: -/// -/// ``` -/// foo is -/// the value: -/// 10 -/// ``` -/// -/// -/// where the first line has no leading indentation, and all other lines have preserved their -/// original indentation. -/// -/// What we want instead is: -/// -/// ``` -/// foo is -/// the value: -/// 10 -/// ``` -/// -/// i.e. we want the whole thing to be dedented. To achieve this, we remove all leading whitespace -/// from the first line, and remove all common whitespace from the rest of the string. -fn handle_indentation(raw: &str) -> Option { - let result: String = match raw.split_once('\n') { - Some((first, rest)) => { - format!("{}\n{}", first.trim_start(), dedent(rest)) + let doc_comment = retrieve_doc_comment(node.syntax(), Some(2)); + match doc_comment { + Some(comment) => Some(DocItem { + name: item_name, + comment: DocComment { + doc: comment, + doc_type: None, + example: None, + }, + args: vec![], + }), + // Fallback to legacy comment is there is no doc_comment + None => { + let comment = retrieve_legacy_comment(node.syntax(), false)?; + Some(DocItem { + name: item_name, + comment: parse_doc_comment(&comment), + args: vec![], + }) } - None => raw.into(), - }; - - Some(result.trim().to_owned()).filter(|s| !s.is_empty()) + } } /// Dumb, mutable, hacky doc comment "parser". @@ -266,49 +211,6 @@ fn parse_doc_comment(raw: &str) -> DocComment { } } -/// Traverse a Nix lambda and collect the identifiers of arguments -/// until an unexpected AST node is encountered. -fn collect_lambda_args(mut lambda: Lambda) -> Vec { - let mut args = vec![]; - - loop { - match lambda.param().unwrap() { - // a variable, e.g. `id = x: x` - Param::IdentParam(id) => { - args.push(Argument::Flat(SingleArg { - name: id.to_string(), - doc: handle_indentation( - &retrieve_doc_comment(id.syntax(), true).unwrap_or_default(), - ), - })); - } - // an attribute set, e.g. `foo = { a }: a` - Param::Pattern(pat) => { - // collect doc-comments for each attribute in the set - let pattern_vec: Vec<_> = pat - .pat_entries() - .map(|entry| SingleArg { - name: entry.ident().unwrap().to_string(), - doc: handle_indentation( - &retrieve_doc_comment(entry.syntax(), true).unwrap_or_default(), - ), - }) - .collect(); - - args.push(Argument::Pattern(pattern_vec)); - } - } - - // Curried or not? - match lambda.body() { - Some(Expr::Lambda(inner)) => lambda = inner, - _ => break, - } - } - - args -} - /// Traverse the arena from a top-level SetEntry and collect, where /// possible: /// @@ -409,8 +311,9 @@ fn retrieve_description(nix: &rnix::Root, description: &str, category: &str) -> category, &nix.syntax() .first_child() - .and_then(|node| retrieve_doc_comment(&node, false)) - .and_then(|comment| handle_indentation(&comment)) + .and_then(|node| retrieve_doc_comment(&node, Some(1)) + .or(retrieve_legacy_comment(&node, false))) + .and_then(|doc_item| handle_indentation(&doc_item)) .unwrap_or_default() ) } @@ -438,129 +341,3 @@ fn main() { .expect("Failed to write section") } } - -#[test] -fn test_main() { - let mut output = Vec::new(); - let src = fs::read_to_string("test/strings.nix").unwrap(); - let locs = serde_json::from_str(&fs::read_to_string("test/strings.json").unwrap()).unwrap(); - let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); - let desc = "string manipulation functions"; - let prefix = "lib"; - let category = "strings"; - - // TODO: move this to commonmark.rs - writeln!( - output, - "# {} {{#sec-functions-library-{}}}\n", - desc, category - ) - .expect("Failed to write header"); - - for entry in collect_entries(nix, prefix, category) { - entry - .write_section(&locs, &mut output) - .expect("Failed to write section") - } - - let output = String::from_utf8(output).expect("not utf8"); - - insta::assert_snapshot!(output); -} - -#[test] -fn test_description_of_lib_debug() { - let mut output = Vec::new(); - let src = fs::read_to_string("test/lib-debug.nix").unwrap(); - let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); - let prefix = "lib"; - let category = "debug"; - let desc = retrieve_description(&nix, &"Debug", category); - writeln!(output, "{}", desc).expect("Failed to write header"); - - for entry in collect_entries(nix, prefix, category) { - entry - .write_section(&Default::default(), &mut output) - .expect("Failed to write section") - } - - let output = String::from_utf8(output).expect("not utf8"); - - insta::assert_snapshot!(output); -} - -#[test] -fn test_arg_formatting() { - let mut output = Vec::new(); - let src = fs::read_to_string("test/arg-formatting.nix").unwrap(); - let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); - let prefix = "lib"; - let category = "options"; - - for entry in collect_entries(nix, prefix, category) { - entry - .write_section(&Default::default(), &mut output) - .expect("Failed to write section") - } - - let output = String::from_utf8(output).expect("not utf8"); - - insta::assert_snapshot!(output); -} - -#[test] -fn test_inherited_exports() { - let mut output = Vec::new(); - let src = fs::read_to_string("test/inherited-exports.nix").unwrap(); - let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); - let prefix = "lib"; - let category = "let"; - - for entry in collect_entries(nix, prefix, category) { - entry - .write_section(&Default::default(), &mut output) - .expect("Failed to write section") - } - - let output = String::from_utf8(output).expect("not utf8"); - - insta::assert_snapshot!(output); -} - -#[test] -fn test_line_comments() { - let mut output = Vec::new(); - let src = fs::read_to_string("test/line-comments.nix").unwrap(); - let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); - let prefix = "lib"; - let category = "let"; - - for entry in collect_entries(nix, prefix, category) { - entry - .write_section(&Default::default(), &mut output) - .expect("Failed to write section") - } - - let output = String::from_utf8(output).expect("not utf8"); - - insta::assert_snapshot!(output); -} - -#[test] -fn test_multi_line() { - let mut output = Vec::new(); - let src = fs::read_to_string("test/multi-line.nix").unwrap(); - let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); - let prefix = "lib"; - let category = "let"; - - for entry in collect_entries(nix, prefix, category) { - entry - .write_section(&Default::default(), &mut output) - .expect("Failed to write section") - } - - let output = String::from_utf8(output).expect("not utf8"); - - insta::assert_snapshot!(output); -} diff --git a/src/snapshots/nixdoc__arg_formatting.snap b/src/snapshots/nixdoc__test__arg_formatting.snap similarity index 100% rename from src/snapshots/nixdoc__arg_formatting.snap rename to src/snapshots/nixdoc__test__arg_formatting.snap diff --git a/src/snapshots/nixdoc__description_of_lib_debug.snap b/src/snapshots/nixdoc__test__description_of_lib_debug.snap similarity index 100% rename from src/snapshots/nixdoc__description_of_lib_debug.snap rename to src/snapshots/nixdoc__test__description_of_lib_debug.snap diff --git a/src/snapshots/nixdoc__test__doc_comment.snap b/src/snapshots/nixdoc__test__doc_comment.snap new file mode 100644 index 0000000..fa3a4fd --- /dev/null +++ b/src/snapshots/nixdoc__test__doc_comment.snap @@ -0,0 +1,58 @@ +--- +source: src/test.rs +expression: output +--- +## `lib.debug.nixdoc` {#function-library-lib.debug.nixdoc} + +**Type**: `This is a parsed type` + +nixdoc-legacy comment + +::: {.example #function-library-example-lib.debug.nixdoc} +# `lib.debug.nixdoc` usage example + +```nix +This is a parsed example +``` +::: + +## `lib.debug.rfc-style` {#function-library-lib.debug.rfc-style} + +doc comment in markdown format + + +## `lib.debug.argumentTest` {#function-library-lib.debug.argumentTest} + +doc comment in markdown format + +Example: + +This is just markdown + +Type: + +This is just markdown + + +structured function argument + +: `formal1` + + : Legacy line comment + + `formal2` + + : Legacy Block + + `formal3` + + : Legacy + multiline + comment + + `formal4` + + : official doc-comment variant + + + diff --git a/src/snapshots/nixdoc__test__doc_comment_section_description.snap b/src/snapshots/nixdoc__test__doc_comment_section_description.snap new file mode 100644 index 0000000..1450900 --- /dev/null +++ b/src/snapshots/nixdoc__test__doc_comment_section_description.snap @@ -0,0 +1,8 @@ +--- +source: src/test.rs +expression: output +--- +# Debug {#sec-functions-library-debug} +Markdown section heading + + diff --git a/src/snapshots/nixdoc__inherited_exports.snap b/src/snapshots/nixdoc__test__inherited_exports.snap similarity index 100% rename from src/snapshots/nixdoc__inherited_exports.snap rename to src/snapshots/nixdoc__test__inherited_exports.snap diff --git a/src/snapshots/nixdoc__line_comments.snap b/src/snapshots/nixdoc__test__line_comments.snap similarity index 100% rename from src/snapshots/nixdoc__line_comments.snap rename to src/snapshots/nixdoc__test__line_comments.snap diff --git a/src/snapshots/nixdoc__main.snap b/src/snapshots/nixdoc__test__main.snap similarity index 100% rename from src/snapshots/nixdoc__main.snap rename to src/snapshots/nixdoc__test__main.snap diff --git a/src/snapshots/nixdoc__multi_line.snap b/src/snapshots/nixdoc__test__multi_line.snap similarity index 100% rename from src/snapshots/nixdoc__multi_line.snap rename to src/snapshots/nixdoc__test__multi_line.snap diff --git a/src/test.rs b/src/test.rs new file mode 100644 index 0000000..6d1caea --- /dev/null +++ b/src/test.rs @@ -0,0 +1,172 @@ +use rnix; +use std::fs; + +use std::io::Write; + +use crate::{collect_entries, retrieve_description}; + +#[test] +fn test_main() { + let mut output = Vec::new(); + let src = fs::read_to_string("test/strings.nix").unwrap(); + let locs = serde_json::from_str(&fs::read_to_string("test/strings.json").unwrap()).unwrap(); + let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); + let desc = "string manipulation functions"; + let prefix = "lib"; + let category = "strings"; + + // TODO: move this to commonmark.rs + writeln!( + output, + "# {} {{#sec-functions-library-{}}}\n", + desc, category + ) + .expect("Failed to write header"); + + for entry in collect_entries(nix, prefix, category) { + entry + .write_section(&locs, &mut output) + .expect("Failed to write section") + } + + let output = String::from_utf8(output).expect("not utf8"); + + insta::assert_snapshot!(output); +} + +#[test] +fn test_description_of_lib_debug() { + let mut output = Vec::new(); + let src = fs::read_to_string("test/lib-debug.nix").unwrap(); + let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); + let prefix = "lib"; + let category = "debug"; + let desc = retrieve_description(&nix, &"Debug", category); + writeln!(output, "{}", desc).expect("Failed to write header"); + + for entry in collect_entries(nix, prefix, category) { + entry + .write_section(&Default::default(), &mut output) + .expect("Failed to write section") + } + + let output = String::from_utf8(output).expect("not utf8"); + + insta::assert_snapshot!(output); +} + +#[test] +fn test_arg_formatting() { + let mut output = Vec::new(); + let src = fs::read_to_string("test/arg-formatting.nix").unwrap(); + let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); + let prefix = "lib"; + let category = "options"; + + for entry in collect_entries(nix, prefix, category) { + entry + .write_section(&Default::default(), &mut output) + .expect("Failed to write section") + } + + let output = String::from_utf8(output).expect("not utf8"); + + insta::assert_snapshot!(output); +} + +#[test] +fn test_inherited_exports() { + let mut output = Vec::new(); + let src = fs::read_to_string("test/inherited-exports.nix").unwrap(); + let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); + let prefix = "lib"; + let category = "let"; + + for entry in collect_entries(nix, prefix, category) { + entry + .write_section(&Default::default(), &mut output) + .expect("Failed to write section") + } + + let output = String::from_utf8(output).expect("not utf8"); + + insta::assert_snapshot!(output); +} + +#[test] +fn test_line_comments() { + let mut output = Vec::new(); + let src = fs::read_to_string("test/line-comments.nix").unwrap(); + let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); + let prefix = "lib"; + let category = "let"; + + for entry in collect_entries(nix, prefix, category) { + entry + .write_section(&Default::default(), &mut output) + .expect("Failed to write section") + } + + let output = String::from_utf8(output).expect("not utf8"); + + insta::assert_snapshot!(output); +} + +#[test] +fn test_multi_line() { + let mut output = Vec::new(); + let src = fs::read_to_string("test/multi-line.nix").unwrap(); + let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); + let prefix = "lib"; + let category = "let"; + + for entry in collect_entries(nix, prefix, category) { + entry + .write_section(&Default::default(), &mut output) + .expect("Failed to write section") + } + + let output = String::from_utf8(output).expect("not utf8"); + + insta::assert_snapshot!(output); +} + +#[test] +fn test_doc_comment() { + let mut output = Vec::new(); + let src = fs::read_to_string("test/doc-comment.nix").unwrap(); + let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); + let prefix = "lib"; + let category = "debug"; + + for entry in collect_entries(nix, prefix, category) { + entry + .write_section(&Default::default(), &mut output) + .expect("Failed to write section") + } + + let output = String::from_utf8(output).expect("not utf8"); + + insta::assert_snapshot!(output); +} + +#[test] +fn test_doc_comment_section_description() { + let mut output = Vec::new(); + let src = fs::read_to_string("test/doc-comment-sec-heading.nix").unwrap(); + let nix = rnix::Root::parse(&src).ok().expect("failed to parse input"); + let prefix = "lib"; + let category = "debug"; + let desc = retrieve_description(&nix, &"Debug", category); + writeln!(output, "{}", desc).expect("Failed to write header"); + + for entry in collect_entries(nix, prefix, category) { + entry + .write_section(&Default::default(), &mut output) + .expect("Failed to write section") + } + + let output = String::from_utf8(output).expect("not utf8"); + + insta::assert_snapshot!(output); +} diff --git a/test.nix b/test.nix new file mode 100644 index 0000000..3a2310e --- /dev/null +++ b/test.nix @@ -0,0 +1,37 @@ +/** +Prequel +*/ +{lib}: +{ + + /** + Create a file set from a path that may or may not exist + */ + packagesFromDirectoryRecursive = + # Options. + { + /** + rfc style + + ``` + Path -> AttrSet -> a + ``` + */ + callPackage, + /* + legacy multiline + + ``` + Path + ``` + */ + directory, + # legacy single line + config, + # legacy + # block + # comment + moreConfig, + }: + 1; +} \ No newline at end of file diff --git a/test/doc-comment-sec-heading.nix b/test/doc-comment-sec-heading.nix new file mode 100644 index 0000000..c906d30 --- /dev/null +++ b/test/doc-comment-sec-heading.nix @@ -0,0 +1,4 @@ +/** + Markdown section heading +*/ +{}:{} diff --git a/test/doc-comment.nix b/test/doc-comment.nix new file mode 100644 index 0000000..b50ea8a --- /dev/null +++ b/test/doc-comment.nix @@ -0,0 +1,53 @@ +{ + # not a doc comment + hidden = a: a; + + /* + nixdoc-legacy comment + + Example: + + This is a parsed example + + Type: + + This is a parsed type + */ + nixdoc = {}; + + /** + doc comment in markdown format + */ + rfc-style = {}; + + /** + doc comment in markdown format + + Example: + + This is just markdown + + Type: + + This is just markdown + */ + argumentTest = { + # Legacy line comment + formal1, + # Legacy + # Block + formal2, + /* + Legacy + multiline + comment + */ + formal3, + /** + official doc-comment variant + */ + formal4, + + }: + {}; +} From 4e945f8ba3dad4739fea690cb4baab8a9bf71a70 Mon Sep 17 00:00:00 2001 From: Johannes Kirschbauer Date: Sat, 24 Feb 2024 14:20:51 +0700 Subject: [PATCH 2/5] typos and minor fixup --- CHANGELOG.md | 4 ++-- README.md | 4 ++-- doc/migration.md | 5 +++-- src/format.rs | 16 ++++++++-------- src/main.rs | 16 +++------------- 5 files changed, 18 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1247066..46ffd1b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,7 +10,7 @@ - **Legacy Custom Format:** The custom nixdoc format is now considered a legacy feature. We plan to phase it out in future versions to streamline documentation practices. - We encourage users to transition to the official doc-comment format introduced in this release. -- We will continue to maintain the legacy format, we will not accept new features or enhancements for it. This decision allows for a period of transition to the new documentation practices. +- For now we will continue to maintain the legacy format, but will not accept new features or enhancements for it. This decision allows for a period of transition to the new documentation practices. See [Migration guide](./doc/migration.md) for smooth transition @@ -20,7 +20,7 @@ See [Migration guide](./doc/migration.md) for smooth transition ## 2.7.0 -- Added support to customize the attribute set prefix, which was previously hardcoded to `lib`. +- Added support to customise the attribute set prefix, which was previously hardcoded to `lib`. The default is still `lib`, but you can pass `--prefix` now to use something else like `utils`. By @Janik-Haag in https://github.com/nix-community/nixdoc/pull/97 diff --git a/README.md b/README.md index 978697d..e6126b4 100644 --- a/README.md +++ b/README.md @@ -11,12 +11,12 @@ function set. ## Comment format This tool implements a subset of the doc-comment standard specified in [RFC-145/doc-comments](https://github.com/NixOS/rfcs/blob/master/rfcs/0145-doc-strings.md). -But, it is currently limited to generating documentation for statically analyzable attribute paths only. +But, it is currently limited to generating documentation for statically analysable attribute paths only. In the future, it could be the role of a Nix interpreter to obtain the values to be documented and their doc-comments. It is important to start doc-comments with the additional asterisk (`*`) -> `/**` which renders as a doc-comment. -The content of the doc-comment should be some markdown. ( See [Commonmark](https://spec.commonmark.org/0.30/) specification) +The content of the doc-comment should conform to the [Commonmark](https://spec.commonmark.org/0.30/) specification. ### Example diff --git a/doc/migration.md b/doc/migration.md index fc6e50c..99c5df8 100644 --- a/doc/migration.md +++ b/doc/migration.md @@ -74,7 +74,8 @@ The approach to documenting single arguments has evolved. Instead of individual # Arguments - - **x**: The value to be returned. + `x` (Any) + : The value to be returned. */ id = x: x; @@ -105,7 +106,7 @@ Structured arguments can be documented (described in RFC145 as 'lambda formals') ```nix { /** - * The `add` function calculates the sum of `a` and `b`. + The `add` function calculates the sum of `a` and `b`. */ add = { /** The first number to add. */ diff --git a/src/format.rs b/src/format.rs index 1eece5b..6d9efff 100644 --- a/src/format.rs +++ b/src/format.rs @@ -49,13 +49,13 @@ pub fn handle_indentation(raw: &str) -> Option { /// Shift down markdown headings /// -/// Performs a line-wise matching to ' # Heading ' +/// Performs a line-wise matching to '# Heading ' /// /// Counts the current numbers of '#' and adds levels: [usize] to them /// levels := 1; gives /// '# Heading' -> '## Heading' /// -/// Markdown has 6 levels of headings. Everything beyond that (e.g., H7) may produce unexpected renderings. +/// Commonmark markdown has 6 levels of headings. Everything beyond that (e.g., H7) is not supported and may produce unexpected renderings. /// by default this function makes sure, headings don't exceed the H6 boundary. /// levels := 2; /// ... @@ -78,15 +78,15 @@ pub fn shift_headings(raw: &str, levels: usize) -> String { pub fn handle_heading(line: &str, levels: usize) -> String { let chars = line.chars(); - let mut leading_trivials: String = String::new(); + // let mut leading_trivials: String = String::new(); let mut hashes = String::new(); let mut rest = String::new(); for char in chars { match char { - ' ' | '\t' if hashes.is_empty() => { - // only collect trivial before the initial hash - leading_trivials.push(char) - } + // ' ' | '\t' if hashes.is_empty() => { + // // only collect trivial before the initial hash + // leading_trivials.push(char) + // } '#' if rest.is_empty() => { // only collect hashes if no other tokens hashes.push(char) @@ -100,5 +100,5 @@ pub fn handle_heading(line: &str, levels: usize) -> String { _ => "#".repeat(hashes.len() + levels), }; - format!("{leading_trivials}{new_hashes} {rest}") + format!("{new_hashes}{rest}") } diff --git a/src/main.rs b/src/main.rs index cfa5618..08a8e31 100644 --- a/src/main.rs +++ b/src/main.rs @@ -113,20 +113,10 @@ impl DocItem { } } +/// Returns a rfc145 doc-comment if one is present pub fn retrieve_doc_comment(node: &SyntaxNode, shift_headings_by: Option) -> Option { - // Return a rfc145 doc-comment if one is present - // Otherwise do the legacy parsing - // If there is a doc comment according to RFC145 just return it. - let doc_comment = match node.kind() { - // NODE_IDENT_PARAM: Special case, for backwards compatibility with function args - // In rfc145 this is equivalent with lookup of the lambda docs of the partial function. - // a: /** Doc comment */b: - // NODE_LAMBDA(b:) <- (parent of IDENT_PARAM) - // NODE_IDENT_PARAM(b) - SyntaxKind::NODE_IDENT_PARAM => get_expr_docs(&node.parent().unwrap()), - _ => get_expr_docs(node), - }; - + let doc_comment = get_expr_docs(node); + doc_comment.map(|doc_comment| { shift_headings( &handle_indentation(&doc_comment).unwrap_or(String::new()), From d121347e43fc0d674364d327315dd2bc6c8650ab Mon Sep 17 00:00:00 2001 From: Johannes Kirschbauer Date: Sat, 24 Feb 2024 14:47:20 +0700 Subject: [PATCH 3/5] add more tests --- src/snapshots/nixdoc__test__doc_comment.snap | 9 ++++++-- src/snapshots/nixdoc__test__headings.snap | 22 ++++++++++++++++++++ src/test.rs | 12 ++++++++++- test/doc-comment.nix | 12 +++++++++-- test/headings.md | 17 +++++++++++++++ 5 files changed, 67 insertions(+), 5 deletions(-) create mode 100644 src/snapshots/nixdoc__test__headings.snap create mode 100644 test/headings.md diff --git a/src/snapshots/nixdoc__test__doc_comment.snap b/src/snapshots/nixdoc__test__doc_comment.snap index fa3a4fd..23ce5c9 100644 --- a/src/snapshots/nixdoc__test__doc_comment.snap +++ b/src/snapshots/nixdoc__test__doc_comment.snap @@ -25,11 +25,11 @@ doc comment in markdown format doc comment in markdown format -Example: +### Example (Should be a heading) This is just markdown -Type: +Type: (Should NOT be a heading) This is just markdown @@ -55,4 +55,9 @@ structured function argument : official doc-comment variant +## `lib.debug.foo` {#function-library-lib.debug.foo} + +Comment + + diff --git a/src/snapshots/nixdoc__test__headings.snap b/src/snapshots/nixdoc__test__headings.snap new file mode 100644 index 0000000..f38195e --- /dev/null +++ b/src/snapshots/nixdoc__test__headings.snap @@ -0,0 +1,22 @@ +--- +source: src/test.rs +expression: output +--- +### h1-heading + +#### h2-heading + +##### h3-heading + +##### h4-heading + +Should be h6 + +###### h5-heading + +This should be h6 + +###### h6-heading + +This should be h6 + diff --git a/src/test.rs b/src/test.rs index 6d1caea..f7e4b35 100644 --- a/src/test.rs +++ b/src/test.rs @@ -3,7 +3,7 @@ use std::fs; use std::io::Write; -use crate::{collect_entries, retrieve_description}; +use crate::{collect_entries, format::shift_headings, retrieve_description}; #[test] fn test_main() { @@ -150,6 +150,16 @@ fn test_doc_comment() { insta::assert_snapshot!(output); } +#[test] +fn test_headings() { + let mut output = String::new(); + let src = fs::read_to_string("test/headings.md").unwrap(); + + output = shift_headings(&src, 2); + + insta::assert_snapshot!(output); +} + #[test] fn test_doc_comment_section_description() { let mut output = Vec::new(); diff --git a/test/doc-comment.nix b/test/doc-comment.nix index b50ea8a..07b29ad 100644 --- a/test/doc-comment.nix +++ b/test/doc-comment.nix @@ -23,11 +23,11 @@ /** doc comment in markdown format - Example: + # Example (Should be a heading) This is just markdown - Type: + Type: (Should NOT be a heading) This is just markdown */ @@ -50,4 +50,12 @@ }: {}; + + # Omitting a doc comment from an attribute doesn't duplicate the previous one + /** Comment */ + foo = 0; + + # This should not have any docs + bar = 1; + } diff --git a/test/headings.md b/test/headings.md new file mode 100644 index 0000000..f79eedb --- /dev/null +++ b/test/headings.md @@ -0,0 +1,17 @@ +# h1-heading + +## h2-heading + +### h3-heading + +### h4-heading + +Should be h6 + +#### h5-heading + +This should be h6 + +##### h6-heading + +This should be h6 From 629f1f792a03318b26e9c2a223dd2de10fa13b6e Mon Sep 17 00:00:00 2001 From: Johannes Kirschbauer Date: Sat, 24 Feb 2024 14:57:27 +0700 Subject: [PATCH 4/5] add note & example about definition list --- README.md | 10 ++++++++-- doc/migration.md | 10 ++++++++-- src/main.rs | 2 +- 3 files changed, 17 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index e6126b4..e24b741 100644 --- a/README.md +++ b/README.md @@ -47,14 +47,20 @@ The following is an example of markdown documentation for new and current users # Arguments - - a: The first number - - b: The second number + a + : The first number + + b + : The second number */ add = a: b: a + b; } ```` +> Note: Within nixpkgs the convention of using [definition-lists](https://www.markdownguide.org/extended-syntax/#definition-lists) for documenting arguments has been established. + + ## Custom nixdoc format (Legacy) You should consider migrating to the newer format described above. diff --git a/doc/migration.md b/doc/migration.md index 99c5df8..4293737 100644 --- a/doc/migration.md +++ b/doc/migration.md @@ -50,8 +50,12 @@ filterAttrs = ``` # Arguments - - **pred**: Predicate taking an attribute name and an attribute value, which returns `true` to include the attribute, or `false` to exclude the attribute. - - **set**: The attribute set to filter + + **pred** + : Predicate taking an attribute name and an attribute value, which returns `true` to include the attribute, or `false` to exclude the attribute. + + **set** + : The attribute set to filter */ filterAttrs = pred: @@ -67,6 +71,8 @@ With the introduction of RFC145, there is a shift in how arguments are documente The approach to documenting single arguments has evolved. Instead of individual argument comments, document the function and its arguments together. +> Note: Within nixpkgs the convention of using [definition-lists](https://www.markdownguide.org/extended-syntax/#definition-lists) for documenting arguments has been established. + ```nix { /** diff --git a/src/main.rs b/src/main.rs index 08a8e31..2cfaa95 100644 --- a/src/main.rs +++ b/src/main.rs @@ -116,7 +116,7 @@ impl DocItem { /// Returns a rfc145 doc-comment if one is present pub fn retrieve_doc_comment(node: &SyntaxNode, shift_headings_by: Option) -> Option { let doc_comment = get_expr_docs(node); - + doc_comment.map(|doc_comment| { shift_headings( &handle_indentation(&doc_comment).unwrap_or(String::new()), From 37ad8cd3fc11e0e2678ea3a30374d09e1198e549 Mon Sep 17 00:00:00 2001 From: Johannes Kirschbauer Date: Sat, 24 Feb 2024 15:11:43 +0700 Subject: [PATCH 5/5] Apply some suggestions Co-authored-by: Silvan Mosberger --- doc/migration.md | 3 +- src/format.rs | 10 ++---- src/snapshots/nixdoc__test__doc_comment.snap | 3 -- src/snapshots/nixdoc__test__headings.snap | 8 ++--- test.nix | 37 -------------------- test/headings.md | 12 +++---- 6 files changed, 14 insertions(+), 59 deletions(-) delete mode 100644 test.nix diff --git a/doc/migration.md b/doc/migration.md index 4293737..257e40e 100644 --- a/doc/migration.md +++ b/doc/migration.md @@ -97,8 +97,7 @@ If arguments require more complex documentation consider starting an extra secti # Arguments - ## **x** - + ## **x** (Any) (...Some comprehensive documentation) */ diff --git a/src/format.rs b/src/format.rs index 6d9efff..9ccb028 100644 --- a/src/format.rs +++ b/src/format.rs @@ -64,11 +64,11 @@ pub fn handle_indentation(raw: &str) -> Option { /// pub fn shift_headings(raw: &str, levels: usize) -> String { let mut result = String::new(); - for line in raw.lines() { + for line in raw.split_inclusive('\n') { if line.trim_start().starts_with('#') { - result.push_str(&format!("{}\n", &handle_heading(line, levels))); + result.push_str(&handle_heading(line, levels)); } else { - result.push_str(&format!("{line}\n")); + result.push_str(line); } } result @@ -83,10 +83,6 @@ pub fn handle_heading(line: &str, levels: usize) -> String { let mut rest = String::new(); for char in chars { match char { - // ' ' | '\t' if hashes.is_empty() => { - // // only collect trivial before the initial hash - // leading_trivials.push(char) - // } '#' if rest.is_empty() => { // only collect hashes if no other tokens hashes.push(char) diff --git a/src/snapshots/nixdoc__test__doc_comment.snap b/src/snapshots/nixdoc__test__doc_comment.snap index 23ce5c9..c69cec1 100644 --- a/src/snapshots/nixdoc__test__doc_comment.snap +++ b/src/snapshots/nixdoc__test__doc_comment.snap @@ -20,7 +20,6 @@ This is a parsed example doc comment in markdown format - ## `lib.debug.argumentTest` {#function-library-lib.debug.argumentTest} doc comment in markdown format @@ -33,7 +32,6 @@ Type: (Should NOT be a heading) This is just markdown - structured function argument : `formal1` @@ -60,4 +58,3 @@ structured function argument Comment - diff --git a/src/snapshots/nixdoc__test__headings.snap b/src/snapshots/nixdoc__test__headings.snap index f38195e..78deea6 100644 --- a/src/snapshots/nixdoc__test__headings.snap +++ b/src/snapshots/nixdoc__test__headings.snap @@ -8,15 +8,15 @@ expression: output ##### h3-heading -##### h4-heading +###### h4-heading -Should be h6 +This should be h6 ###### h5-heading -This should be h6 +This should be h6 as well ###### h6-heading -This should be h6 +This should be h6 as well diff --git a/test.nix b/test.nix deleted file mode 100644 index 3a2310e..0000000 --- a/test.nix +++ /dev/null @@ -1,37 +0,0 @@ -/** -Prequel -*/ -{lib}: -{ - - /** - Create a file set from a path that may or may not exist - */ - packagesFromDirectoryRecursive = - # Options. - { - /** - rfc style - - ``` - Path -> AttrSet -> a - ``` - */ - callPackage, - /* - legacy multiline - - ``` - Path - ``` - */ - directory, - # legacy single line - config, - # legacy - # block - # comment - moreConfig, - }: - 1; -} \ No newline at end of file diff --git a/test/headings.md b/test/headings.md index f79eedb..bad1393 100644 --- a/test/headings.md +++ b/test/headings.md @@ -4,14 +4,14 @@ ### h3-heading -### h4-heading +#### h4-heading -Should be h6 +This should be h6 -#### h5-heading +##### h5-heading -This should be h6 +This should be h6 as well -##### h6-heading +###### h6-heading -This should be h6 +This should be h6 as well