Feature request: Generate Rust docs from Doxygen comments

[Ploppz]

I think it would be very useful to be able to tell bindgen to parse Doxygen comments and generate Rust doc comments.

[fitzgen]

@Ploppz, Can you provide some examples for whoever might pick up this issue? Thanks!

[Ploppz]

I'm currently working with this C++ library. Examples: the functions here. At least I think that's Doxygen... The structs don't have doxygen comments though, but I think that should be possible too.

[fitzgen]

@Ploppz we do generate rust doc comments (they aren't necessarily pretty) and I'm not sure what concrete changes you're asking for in this issue. For example:

/**
 * Does the stuff.
 *
 * @returns whatever
 */
bool demboogy(int x, int y);

becomes

/* automatically generated by rust-bindgen */

extern "C" {

    /// Does the stuff.
    ///
    /// @returns whatever

    #[link_name = "_Z8demboogyii"]
    pub fn demboogy(x: ::std::os::raw::c_int, y: ::std::os::raw::c_int)
        -> bool;
}

[Ploppz]

I don't see it in the docs generated by cargo doc --open. I uploaded the code here. Neither the functions nor the structs have documentation comments, it seems.

[thejpster]

See https://docs.rs/nrfxlib-sys/0.1.6/nrfxlib_sys/ for an example of where Doxygen comments don't translate well.

• @brief should be removed.
• @defgroup <group_name> <label> should be used to create groups, or removed.
• @ingroup <group_name> should be used to put items into groups, or removed.
• @param <param_name> <label> should be used to label a function parameter.

As a first pass, it would be helpful to retain the newlines from the original comments, so that these items were each on a line on their own, rather than all thrown into a single line. Actually, I might try doing a post-bindgen regexp in my build.rs as a workaround and see if that helps.

[NickeZ]

Is the direction argument (in/out) like @param[in] supposed to work? It doesn't seem to work for me.

[virtualritz]

Here are some docs generated by bindgen (more or less trying to mirror Sphinx formatting here):

---

AcquireSuite: Option<unsafe extern "C" fn(name: *const c_char, version: c_int, suite: *mut *const c_void) -> SPErr>

Acquires a function suite. Loads the suite if necessary, and increments its reference count. For example: @code SPErr error; SPBasicSuite *sBasic = message->d.basic; AIRandomSuite *sRandom; sBasic->AcquireSuite( kAIRandomSuite, kAIRandomVersion, &sRandom ); @Endcode @param name The suite name. @param version The suite version number. @param suite [out] A buffer in which to return the suite pointer. @see \c #SPSuitesSuite::AcquireSuite()

---

As you can see Doxygen hints (@whatever) and newlines are just ignored.
As @thejpster said: keeping newlines would already help, as a 1st step.
If Doxygen hints were supported too it would look more like this:

---

AcquireSuite: Option<unsafe extern "C" fn(name: *const c_char, version: c_int, suite: *mut *const c_void) -> SPErr>

Acquires a function suite. Loads the suite if necessary, and increments its reference count. For example:

SPErr error;
SPBasicSuite *sBasic = message->d.basic;
AIRandomSuite *sRandom;
sBasic->AcquireSuite( kAIRandomSuite, kAIRandomVersion, &sRandom ); 

name – The suite name.
version – The suite version number.
suite [out] – A buffer in which to return the suite pointer.
See also: SPSuitesSuite::AcquireSuite()

---

[necrose99]

a rs-doxygen would be a plus , ie not use it but be a drop in , when it breaks , pkgs building brreak.

if doxygen breaks , can run .. a compat cli tool. cargo-doxygen anyone ..

[tgross35]

Expanding a little bit to @thejpster's comments for some of the more common commands:

• @briefrepresents a one line summary, so it can be removed. Or, make this the first line in the documentation block (it probably will be the first line anyway). It would be good to insert a line break before any further @xxx attributes, to make sure this goes on its own line
• @copydoc MyClass::myfunction() Copies the documentation from another module, could do the same here
• \@ escaped at symbol, can just remove the slash
• @sa { references } Used to link to other sections. This could somehow influence paths for [attr]
• @todo Maybe just change to TODO:?
• @param is used for each argument, and includes an optional [in], [out], or [in,out]. Example:

 * @param[out] dest The memory area to copy to.
 * @param[in]  src  The memory area to copy from.
 * @param[in]  n    The number of bytes to copy
void memcpy(void *dest, const void *src, size_t n);

The unofficial canonical way to describe arguments is:

/// Arguments:
///
/// * `dest`: The memory area to copy to.
/// * `src`: The memory area to copy from.
/// * `n`: The number of bytes to copy

Which should be easy enough to generate. There is talk under rustdoc to support something for per-argument /// attributes - but this doesn't seem to have gotten far, so likely not worth worrying about for now.

It would be very interesting to optionally use [in]/[in,out] or [out] to determine the mutability of arguments, which would be more clear than including the information in documentation.

The full doxygen command list is here: https://doxygen.nl/manual/commands.html

[pvdrz]

I wonder if we could add some sort of callback to handle comments so bindgen won't have to explicitly support different formats for them.

[tgross35]

Hey @Techie-Pi is working on doxygen-rs - what is your intended relationship with this project? I notice it has the same license, which seems intentional.

I'm wondering if the maintainers here would be open to adding your crate as a dependency, or if you were sort of planning to try to upstream your crate here.

I like @pvdrz's idea about a docs callback in any case, that would be nicely flexible.

[Techie-Pi]

@tgross35 My current plan is to create something (maybe not with the best inner workings), but functional and easy to use (see doxygen-rs-cli).

If the maintainers of bindgen think this is inside the scope of the project, I'll be completely up to working on adding this as a feature of bindgen, fixing the code, moving repos or whatever is needed :)

For example, this is being used on the rust-3ds/ctru-rs repo (PR with latest improvements to doxygen-rs: ref), and it seems to work almost flawlessly. There is still a lot of work, but it is for the most part feasible

PS. The docs related to compatible tags are outdated, see the tracking issue for more details ;p

PSS. If anyone want to use the project and has any troubles, hit me up!

[tgross35]

That's awesome, great to hear it's working well! I'm curious what the maintainers think (pvdrz I think you're involved here, but @emilio what are your thoughts?)

The callback idea seems straightforward enough, with Techie-Pi's work as the default. Only thing is that I don't think this would allow for using doxygen comments to optionally hint mutability like I had suggested earlier - not that this is a pressing feature, but it may be a "nice to have" optional flag some day to ease abstraction design.

[pvdrz]

My only intention with the callback API is not having to add extra dependencies to bindgen to achieve this. Maybe the callback implementation using doxygen could be in the user guide so folks now how to use it if required but I'd prefer not having doxigen-rs as a dependency of bindgen

[Techie-Pi]

I'd prefer not having doxigen-rs as a dependency of bindgen

Yeah, me neither, I think the callback API is the best option here tbh. But if I can help with any Doxygen-related thing or something, I'm up to helping!

[pvdrz]

#2347 would allow users to handle comments however they want. For example:

impl ParseCallbacks for Cb {
    fn process_comment(&self, comment: &str, _indentation_level: usize) -> Option<String> {
        Some(
            comment
                .lines()
                .fold("/// ````c,ignore".to_owned(), |acc, x| acc + "\n/// "+ x)
                + "\n/// ````",
        )
    }
}

can be used to escape all comments inside markdown code blocks. I suppose that after removing the comment delimiters, doxygen-rs should be able to parse the comments.

[tgross35]

Agreed that callback seems like the way to go. Some sort of builtin that does the basics (like remove the annoying \brief) might be nice, but could be added later

Tangential question, is there any reason that bindgen uses the #[doc] macro instead of ///? Couldn’t find any reasoning, and I find the second option significantly easier to read & maintain

[pvdrz]

Agreed that callback seems like the way to go. Some sort of builtin that does the basics (like remove the annoying \brief) might be nice, but could be added later

There are some ongoing discussions about this. If it were my decision I'd just escape everything using quadruple backticks but there might be some issues with that.

Tangential question, is there any reason that bindgen uses the #[doc] macro instead of ///? Couldn’t find any reasoning, and I find the second option significantly easier to read & maintain

The main reason is that quote! takes anything after /// as a doc so we cannot insert docs using that:

fn main() {
    let comment = "hello world!";
    // This is **not** `/// hello world!`
    let x = quote::quote! {
        /// #comment
    };
    println!("{}", x); // Will print `# [doc = r" #comment"]`
}

[tgross35]

There are some ongoing discussions about this. If it were my decision I'd just escape everything using quadruple backticks but there might be some issues with that.

Sorry, what would the triple backticks be for? I see them mentioned here in the context of markdown, but I didn't see the connection to \brief

Thanks for the clarification on quote!, I opened #2348 to discuss a postprocess option there (couldn't find any existing discussion)

[pvdrz]

Sorry, what would the triple backticks be for? I see them mentioned here in the context of markdown, but I didn't see the connection to \brief

Putting all the docstring between quadruple backticks would allow you to show them as a code block in markdown syntax with few chances of having escaping issues.

[pvdrz]

now that #2347 landed you can provide an implementation of ParseCallbacks::process_comments that parses doxygen docs

[Techie-Pi]

now that #2347 landed you can provide an implementation of ParseCallbacks::process_comments that parses doxygen docs

Right now I'm not able to implement this, but in the following weeks I'll add support for this to doxygen-rs

[pvdrz]

@Techie-Pi from the doxygen-rs docs I'd say that something like this from the user side would be enough:

    #[derive(Debug)]
    struct Cb;

    impl ParseCallbacks for Cb {
        fn process_comment(&self, comment: &str) -> Option<String> {
            Some(doxygen_rs::transform(comment))
        }
    }

[oberrich]

I've created another crate that transforms doxygen comments to rustdoc markdown: https://crates.io/crates/doxygen-bindgen
This uses a dead simple design using yap for a bit more convenient parsing (maybe ~60 lines of code). Removing yap with Peekable<char> would be possible here but make the code slightly more verbose.

Advantages to doxygen-rs crate:

• Does not panic
• Supports arbitrary attributes like @param[in, optional, reserved]
• Less strict parsing

Should be almost equivalent to the doxygen-rs crate in output, although it doesn't handle uncommon features like emojis or @{ groups.