Add tutorials

[francois-rozet]

Description

The documentation should provide tutorials for common use cases, such as creating a custom (coupling/autoregressive) flow, training with the forward and backward KL, performing importance sampling, adding preprocessing transformations, and maybe training a CNF with a flow-matching loss.

These tutorials can be Jupyter notebooks and can either be included in the documentation with myst-nb or linked from the repository.

[francois-rozet]

Hey @simonschnake, do you think of another tutorial that could be added?

[simonschnake]

Hey @francois-rozet,
yes, I would start by porting the two moons example from nflows
https://github.com/bayesiains/nflows/blob/master/examples/conditional_moons.ipynb.
It is relative straight forward and was a good starting point for myself.

I shortly looked into the examples of normflows and they look very nice.
https://github.com/VincentStimper/normalizing-flows/tree/master/examples

I am happy to help you in writing tutorials.
What format do you target?

[francois-rozet]

These notebooks are good bases, but I think more explanations are necessary, at least for the "first" tutorial that would explain the core Zuko concepts. For instance, something like https://github.com/francois-rozet/lampe/blob/master/docs/tutorials/npe.ipynb . I'll write this first tutorial, and then we can add the others with a similar format. I will gladly accept your help!

Also, I am planning to refactor the flows module into sub-modules (see #24). I was wondering if we should rename the DistributionModule and TransformModule to DistributionFactory and TransformFactory (or Recipe). WDYT?

[simonschnake]

I think you are right. The way in which the forward pass of the TransformModule gives you a Transform is something not very common. A normal torch user would expect to get a tensor back.

I don't know how backward compatible you want to stay. I think a clear naming split between the DistributionModule and TransformModule, and the dedicated DistributionFactory and TransformFactory, would be nice.

In mathematics, a function that returns a function is called a functional. Maybe we could take that and call them Transformal and Distributional. The nice thing about including Factory at the end is that it is absolutely clear what the thing is doing.

[francois-rozet]

Let's go for Factory, I also think it is clearer! I'll merge PR #24 soon.

[francois-rozet]

Hello @simonschnake, I finally had the time to write the first tutorials. There has been a lot of changes (including transferring to the probabilists organization) since the last release. I think after we merge the tutorials PR (#27) we are ready to bump Zuko to version 1.0.0 🔥

If you have the time, could you read the tutorials (https://zuko.readthedocs.io/en/tutorials/tutorials.html) and give some feedback? 🙏