How does this magic work? — Recursive ToSchema auto register.

[enricoschaaf]

I tried to understand #1066

but I could not figure out how this works. I am very interested in what to me seems like keeping state inside proc macro calls. I think this could be very helpful for other libraries. I am for example thinking of a message queue where you can at definition time of the message write how the message should be handled and let the macro auto register this message in another place in the codebase without having to basically manually parse the whole codebase to find all messages.

Thanks a lot for creating this library and this interesting pattern!

[juhaku]

In short, there is no global state. And making one would be quite hard actually and not really sustainable (in utoipa's case).

In utoipa the so-called "state" tracking is a chain of definitions. All that needs to be registered needs to be linked to something that is linked to another thing that is eventually linked to an OpenApi. In the example shown in #1066 the Person type has reference to the Account. When ToSchema is executed for the Person struct it will create a schema reference to the Account. Later when the Person type is used in request_body it will create schema references to Person for the path test_collect_schemas. Even later when the path test_collect_schemas is added to the OpenApi via paths(...) or by other means (not manually though) it will call the schema references first for the path test_collect_schemas then for the Person and lastly for the Account.

This is made possible by Rust's amazing trait system. E.g. we can call <Something as MyTrait>::function() anywhere from the code basically. With this in mind at the very root of our definition chain (place where paths(...) is defined) we first define a "state" which is holding the accumulated values over the function calls and when all of the calls has been called we add the accumulated state to the OpenAPI.

[enricoschaaf]

Thanks for the explanation, much appreciated and indeed that makes a lot of sense. ☺️ Sad to know that global state is still hard to impossible, would be really nice for auto registering http routes, background processing fns etc. I guess I keep doing it manually or via reading the files and detecting usage this way.

[juhaku]

Yeah, for example there is a library called utoipauto that does it. It processes the project files or appointed files across crates and registers found items for utoipa. While it works, it is more resource intensive and slows down the compilation.

This is just the nature of the Rust language and effect of it being statically typed, so no runtime evaluation or awareness of available types and methods within the project.

While the global state is possible in some sense e.g. using lazy_static crate and most of its functionality has already landed into Rust itself. But if it needs to work across crates and threads and somehow to be synchronized by the order of which macros can and should be executed, there is no way to guarantee the order and by that the fact that the state stays correct throughout the macro execution.