[Feature] Pyqtorch - First Order Adjoint Differentiation

[dominikandreasseitz]

Fixes #97

This MR adds a "adjoint" differentiation mode to qadence which is a implementation of Algorithm 1 of https://arxiv.org/pdf/2009.02823.pdf.

Limitations:

1. The adjoint method is only available in the pyqtorch backend
2. Parametric observables are not supported
3. Multiple observables are not supported
4. Higher order derivatives are not natively supported by the adjoint method but torch.autograd can be used on the gradients returned by the adjoint method.
5. Only expectation values can be differentiated, not wave functions

Additional changes:

• Removal of gradient checkpointing (raising a UserWarning if requested)
• Utility methods for the pyqtorch backends to convert/validate qadence/ pyqtorch states
• Many new tests comparing gradients between ad, adjoint and gpsr
• Rework of _set_backend_config
• Added section for adjoint differentiation to examples/quantum_model.py

Future Todos:

• Optimize HamEvo derivatives for time_evo and generators (currently done via finite differencing)

[nmheim]

Are we moving adjoint diff from pyqtorch to qadence?

[dominikandreasseitz]

@nmheim, cant reply to your comment directly. we need our own qadence adjoint because we need to handle custom backward passes for nested circuits, and scaleblocks

[nmheim]

• are we preventing user-level parameters somehow? I remember us talking about concerns about correctness in that case?

what are "user-facing" parameters to you? expression level? if yes, then yes, we need gate-level parameters.

just wondering how the gate-level params are enforced, is that done automatically when choosing the diff_mode?

[madagra]

First part of the review. Overall, it looks great to me but there is one very important point missing here: documentation. I would at least add a small example in the getting started tutorial + a script in the examples/ folder.

[dominikandreasseitz]

• are we preventing user-level parameters somehow? I remember us talking about concerns about correctness in that case?

what are "user-facing" parameters to you? expression level? if yes, then yes, we need gate-level parameters.

just wondering how the gate-level params are enforced, is that done automatically when choosing the diff_mode?

@nmheim here https://github.com/pasqal-io/qadence/blob/ds/adjoint/qadence/extensions.py#L97

[madagra]

Thanks, @dominikandreasseitz. I am pre-approving this but I would add proper documentation before merging.

[RolandMacDoland]

Very nice @dominikandreasseitz. It is much clearer for me after our discussion. Very minor comment for the sake of discussion. Otherwise LGTM.