Operator bases which are not outer product "KetBra" bases

[akirakyle]

I've been thinking a lot about all the issues related to bases here (#25, #26, #27, #32, #33, #34). To help me think through these issues, I've been iterating through a bunch of prototype abstract interfaces for Basis, OperatorBasis, and SuperOperatorBasis (discussed in #26) so I can get a feeling for how well different design tradeoffs in the abstract interface work out in practice. I'll hopefully submit that for some discussion here soon. One thing I've run into, which warrants a separate issue so that I have something to reference, is PauliBasis.

Currently PauliBasis is not actually used in anywhere in any dependent packages I could find aside from being exported by QuantumOpticsBase. Here's the current definition

struct PauliBasis{S,B} <: Basis
    shape::S
    bases::B
    function PauliBasis(num_qubits::T) where {T<:Integer}
        shape = [2 for _ in 1:num_qubits]
        bases = Tuple(SpinBasis(1//2) for _ in 1:num_qubits)
        return new{typeof(shape),typeof(bases)}(shape, bases)
    end
end

There's the test_pauli in QuantumOpticsBase which shows some examples of it being used such as

q2 = PauliBasis(2)
CNOT = DenseOperator(q2, q2, diagm(0 => [1,1,0,0], 1 => [0,0,1], -1 => [0,0,1]))

But this is not the CNOT gate in the pauli basis, but rather in the "standard" (assuming a given tensor product layout) computational basis. In the Pauli basis CNOT can be represented as (taken from Gottesman's textbook, the other elements can be found by acting by conjugation by other Paulis)

$$\begin{align*} X \otimes I &\to X \otimes X\\\ Z \otimes I &\to Z \otimes I\\\ I \otimes X &\to I \otimes X\\\ I \otimes Z &\to Z \otimes Z \end{align*}$$

Thus in order to meaningfully define an operator in the Pauli basis, one needs to fix some canonical ordering of the (complex) coefficients for $I,X,Z,Y$, which is really the whole point of the stabilizer tableau.

Currently every subtype of AbstractOperator{BL,BR} specifies a left hilbert space and right hilbert space basis which basically implicitly assumes a "KetBra" style operator basis in which basis elements for the operator are all outer products of basis elements for the Operators left and right Hilbert spaces. In general this need not be the case, and the Pauli basis is as example of a unitary operator basis for which the operator basis elements cannot be represented as outer products. Another example of operator bases are the Heisenberg-Weyl (sometimes called clock and shift) basis for density operators on arbitrary dimensional hilbert spaces. The Gaussian state formalism being developed in Gabs I think can be considered as working in a CV analogue of the PauliBasis which suggests that in fact the GaussianState should actually subtype AbstractOperator instead of StateVector given that it can represent mixed as well as pure states.

@Krastanov @apkille
Hopefully this makes some sense, but it's fine to hold off commenting until I have converged my prototyping work on something that I feel solves these issues at the interface level...

[Krastanov]

Good catch. I do not have anything useful to say yet, but I did some git archeology.

This functionality was moved to QuantumInterface from QuantumOpticsBase. It was already present in QuantumOpticsBase when it was split from QuantumOptics.

Here is the initial PR that added it to QuantumOptics qojulia/QuantumOptics.jl#251

[Krastanov]

I think there might be some confusion about what PauliBasis is supposed to be. I have not read through all of the code, but looking at the docstring and the constructor:

Basis for an N-qubit space where `num_qubits` specifies the number of qubits.
The dimension of the basis is 2²ᴺ.

To me this does not look like a basis of matrices for the vector space of unitary matrices. To me this looks like a basis of kets for the Hilbert space ℂ²ⁿ.

I.e. it kinda makes sense to say that PauliBasis(n) == SpinBasis(1//2)^n.

Is there something deeper in that PR that might be at odds with my interpretation?

[akirakyle]

Yes I think the way it's specified and used it looks like it's just meant to be a shorthand for a CompositeBasis consisting of spin 1/2 bases. I just think it's really confusing to call this the Pauli basis since that typically refers to the unitary operator basis consisting of pauli matrices and identity. Also the tensor product machinery won't always work with this since many functions explicitly use CompositeBasis for their type signature. It might make sense to have specialized objects which represent tensor product spaces but don't obey the CompositeBasis rules. Because it can be useful to treat a CompositeBasis as just another hilbert space without preexisting tensor product structure. This is something I need for the Kraus representation PR so to it is something I'm trying to figure out how to add, but in a more general way.

The larger point of this issue is to to discuss how to incorporate the actual Pauli basis into QuantumInterface since that would be what I think should be subtyped in QuantumCliffords and QuantumSymbolics where relevant. Such unitary operator bases won't fit into an AbstractOperator which separates a left and right hilbert basis, since such a concept only makes sense for "KetBra" style operator bases formed from outer products of Hilbert space bases. In other words a generic orthornormal (under Hilbert-Schmidt norm) operator basis for $\mathcal{H}_A \to \mathcal{H}_A$ with $d= \dim\mathcal{H}_A$ has $d^2$ elements but there isn't necessarily a way in which $d$ of those elements correspond to a basis for only $\mathcal{H}_A$.

[Krastanov]

We can start making a list of deprecations. The name PauliBasis is indeed confusing, but what it implements here is pretty useful. Some notion of equivalence between PauliBasis and SpinBasis^2 would also be useful.

I do not think that the entity currently called QuantumIntefrace.PauliBasis has anything to do with operator bases, right? If that is true, let's keep two separate issue reports: one about problems with PauliBasis and one about the need for more general operator bases. Right now this issue report refers to both, which would lead to significant confusion.

For when we start discussing general operator bases: do you currently have an example problem that needs an operator basis that can not be represented in terms of outer products of basis vectors of the two Hilbert spaces?

[akirakyle]

I do not think that the entity currently called QuantumIntefrace.PauliBasis has anything to do with operator bases, right?

Nope it doesn't, I opened a different issue for that.

For operator bases. This is mainly an interface design question. Given that every operator (and superoperator) can always be represented in outer product bases, that is a valid design choice to just restrict all objects to ultimately be represented as such. However two current examples where making the api of QuantumInterface general enough to capture the general notion of an orthornormal operator basis are the QuantumOpticsBase.PauliTransferMatrix and all the subtypes of QuantumSymbolics.Abstract{Single,Two}QubitGate.

A Pauli transfer matrix describes a channel in terms of how Paulis map to Paulis and so is a "KetBra" or outer product style superoperator basis where the left and right bases are the Pauli operator basis. Of course given we have PauliTransferMatrix as a distinct type, we can just hold distinction this "in our head" and be careful to define conversion to objects defined in an outer product basis.

In the case of QuantumSymbolics, in one sense the operators which are defined need not be basis dependent, since only their algebraic relations codified via symbolic rules matter. However I think pretty much every symbolic object one would care to define will be in terms of a basis element. Thus the XBasisState should really introduce a different basis than SpinBasis(1//2) since we typically consider the latter to be the computational basis which ZBasisStates are elements of. Similarily all of {X,Y,Z}Gate are basis elements of the Pauli basis. While PauliM and PauliM are elements of the "KetBra" basis. Meanwhile the clifford gates can only be written a superpositions of Paulis, however given that they are the normalizer of the Pauli group, their representation as a Pauli transfer matrix is simply as permutation matrices.

Related to this design question is what the objects in QuantumClifford and Gabs should subtype. Given each takes advantage of specific group structure to efficiently represent certain classes of states, the underlying data is never in a "KetBra" type basis. They're really in a symplectic basis, and I'm wondering whether QuantumInterface should try to capture that notion somehow.

[Krastanov]

In the case of QuantumSymbolics, in one sense the operators which are defined need not be basis dependent, since only their algebraic relations codified via symbolic rules matter. However I think pretty much every symbolic object one would care to define will be in terms of a basis element. Thus the XBasisState should really introduce a different basis than SpinBasis(1//2) since we typically consider the latter to be the computational basis which ZBasisStates are elements of. Similarily all of {X,Y,Z}Gate are basis elements of the Pauli basis. While PauliM and PauliM are elements of the "KetBra" basis. Meanwhile the clifford gates can only be written a superpositions of Paulis, however given that they are the normalizer of the Pauli group, their representation as a Pauli transfer matrix is simply as permutation matrices.

I think my mental model of QuantumSymbolics is very different, which might be leading to miscommunication. No matter which design ends up better (or simply implemented if not better), we should definitely document this more clearly.

In my mind, we should have basis(::XBasisState)==basis(::ZBasisState)==basis(::YBasisState)==SpinBasis(1//2)==QuantumInterface.Deprecated.PauliBasis(1) because due to historic reason we conflate "basis" with "space" and all of these are indeed in the ℂ² Hilbert space.

Similarly basis(::XGate)==basis(::YGate)==basis(::ZGate)==basis(::AbstractSingleQubitGate)==SpinBasis(1//2) because basis on a square operator gives the space in which the operator acts, not the space to which the operator belongs.

I do not think we currently have a way to query "what is the space of operators to which this one operator belongs". Currently I am not sure we need it.

Concerning Gabs, I think it should just use FockBasis(Inf) (again, with the annoying conflating of space and basis). States in Gabs should be subtypes of Ket or AbstractKet; operators should be subtypes of AbstractOperator.

These are mostly my thoughts in the context of balancing good API design, simplicity, practicality, and minimizing breaking changes.

[akirakyle]

I think my mental model of QuantumSymbolics is very different

I think one motivation for my thinking here is that I have some ambitions to teach QuantumSymbolics about index groups (in the context of nice error-bases - see https://errorcorrectionzoo.org/c/knill#defterm-Nice_20Xerror_20Xbasis) in which case having a more fine grained understanding of the relation between operator bases and ket-bra bases is potentially beneficial for using method dispatch in an idiomatic way to make operations on objects compatible between the two. Of course it's hard to say abstractly without digging into the details :)

I do not think we currently have a way to query "what is the space of operators to which this one operator belongs". Currently I am not sure we need it.

I think length(b::Basis) is basically that since every finite Hilbert space along with separable Hilbert space can just be identified by its dimensionality.

Concerning Gabs, I think it should just use FockBasis(Inf) (again, with the annoying conflating of space and basis). States in Gabs should be subtypes of Ket or AbstractKet; operators should be subtypes of AbstractOperator.

I would advocate that states in Gabs should be subtypes of AbstractOperator while channels are subtypes of AbstractSuperOperator given that there's isn't a simplified representation of pure states in the Gaussian formalism the way there's the simplified representation of pure states as kets because their density matrices are rank one.

PS, in case you haven't seen it I would say #34 may be the most immediate basis design issue since I think there's the potential for subtle correctness issues that the basis system is intended to correct but doesn't currently. Fixing this has been the main focus of my prototyping efforts around revising the way bases work.

[Krastanov]

I would advocate that states in Gabs should be subtypes of AbstractOperator while channels are subtypes of AbstractSuperOperator given that there's isn't a simplified representation of pure states in the Gaussian formalism

Yup, that makes sense.

[apkille]

I would advocate that states in Gabs should be subtypes of AbstractOperator while channels are subtypes of AbstractSuperOperator given that there's isn't a simplified representation of pure states in the Gaussian formalism the way there's the simplified representation of pure states as kets because their density matrices are rank one.

This is a good point, I agree as well. I can implement these changes once the parametrization of these abstract types are changed/settled in QuantumInterface.

Concerning Gabs, I think it should just use FockBasis(Inf) (again, with the annoying conflating of space and basis).

I don't think I agree with this. What I believe @akirakyle is referring to, particularly about bases in Gabs, is that there's an additional case of symplectic bases for Gaussian states and operators, which is based on how you order field operators for an N mode system. In Weedbrook et. al, everything is kept in a paired quadrature basis, however, you could certainly rearrange your quadrature operators. For example, they're also commonly placed in a block form (Serafini does this in his textbook "Quantum Continuous Variables"; the block symplectic form is also the first example on wikipedia: https://en.wikipedia.org/wiki/Symplectic_matrix). In Gabs, I followed the convention of Weedbrook, however, it would be good to have support for different symplectic bases. As I mentioned in #33 (comment), this could work nicely with the AbstractOperator changes that @akirakyle mentioned. I don't see any large benefits to having Gabs types be assigned to FockBasis or CoherentStateBasis. Really the underlying basis structure that's important for Gaussian states is the symplectic representation. I should mention, of course, in the future I plan to have numerical conversions to Fock state representations in QuantumOptics.

[Krastanov]

Concerning Gabs, I think it should just use FockBasis(Inf) (again, with the annoying conflating of space and basis).

I don't think I agree with this. What I believe @akirakyle is referring to, particularly about bases in Gabs, is that there's an additional case of symplectic bases for Gaussian states and operators, which is based on how you order field operators for an N mode system. In Weedbrook et. al, everything is kept in a paired quadrature basis, however, you could certainly rearrange your quadrature operators.

I think this is another example of the issues that come from currently conflating basis and space in this API. For me it is important to have a way to ask "in what space does this object live". For the objects from Gabs, these would be HarmonicOscillatorSpace^number_of_modes. The only way we can currently express this is by saying basis(gabs_object) == FockBasis(Inf)^nsubsystems(gabs_object). This is an abuse of the word basis, but that abuse is fairly established in this codebase.

Thus, if we are to take for granted that basis actually means "space", would you agree that Gabs should use FockBasis(Inf)?

Now there is the separate question of whether we should fix this abuse of the word "basis". Maybe one way to do that would be to introduce space() which is similar to basis(). Or maybe we should just completely drop (deprecate) basis and rename it to hilbert_space.

To clarify future discussions, could you specify whether when you say basis you mean "'basis' as currently implemented in QuantumOptics, which is significantly conflated with 'Hilbert space'" or you mean "'basis' and 'space' are two distinct notions that I will not be conflating, although this requires severe changes to QuantumOptics"

[apkille]

Thus, if we are to take for granted that basis actually means "space", would you agree that Gabs should use FockBasis(Inf)?

Yes. However, this would make things confusing if we were to have this feature and support for symplectic bases in Gabs. In this case, I would probably define a "symplectic basis" system in Gabs, while maintaining the current conflation between "basis" and "Hilbert space". So we would have something like basis(gabs_obj) == FockBasis(Inf)^nsubsystems(gabs_obj) and symplecticbasis(gabs_obj) = SomeSymplecticBasis.

To clarify future discussions, could you specify whether when you say basis you mean "'basis' as currently implemented in QuantumOptics, which is significantly conflated with 'Hilbert space'" or you mean "'basis' and 'space' are two distinct notions that I will not be conflating, although this requires severe changes to QuantumOptics"

When I've referred to "basis" in previous responses in this issue, I meant a notion separate from "space". My thinking was that I could define abstract type SymplecticBasis <: Basis, and use that basis type when subtyping Gabs types with AbstractOperator. Although maybe this is too clumsy of a workaround, as it seems both you and @akirakyle define Basis as a Hilbert space basis, and it doesn't address your need of knowing what space the object is in. In an ideal world, I think the best thing for Gabs would be to say basis(gabs_obj) == SomeSymplecticBasis and space(gabs_obj) == HarmonicOscillatorSpace^nmodes. But, as you've mentioned, this doesn't follow the much more established design of QuantumOptics. I currently don't think this ideal world is enough to make such severe changes to QuantumOptics. I think it's great and productive to discuss all of the different possibilities for the interface design, but I think more concrete examples are needed to argue that substantial parts of QuantumOptics should be redefined mainly for reasons of elegance. To make these discussions more organized, perhaps we could document the examples we do have in a separate issue?

[Krastanov]

I concur with that last message.

We should indeed get around to documenting all of this somewhere. A few more points to be added in this discussion:

• Currently basis is most valuable when it comes to "incompatible basis checks" which really should just be called "incompatible spaces checks"
• The FockBasis is probably the one that has the most significant distinction between basis and space (and regrettably it serves both roles). It having lower and upper cutoff makes it viable as a way to talk about different (incomplete, sparse) bases of the same space.
• Currently, QuantumClifford and QuantumSymbolics just use SpinBasis(1//2)^n, i.e. they use it as a "space", not as a "basis"
• The symplectic basis setup of Gabs should probably start as something independent from the current "basis, but not really, actually a space" conventions of QuantumOptics, as to not slow stifle the development of Gabs. After the Gabs teams figures out what is best for them, we can have a discussion of what would be the most fruitful way to start making breaking changes to QuantumOptics/Interface. Until then I am all on board with the less-severely-breaking changes that Akira is suggesting.

[akirakyle]

Currently basis is most valuable when it comes to "incompatible basis checks" which really should just be called "incompatible spaces checks"

As far as QuantumOpticsBase is concerned, I think it's both, right? Even though multiplying a bra defined on SpinBasis(1//2) and a ket defined on FockBasis(1) is fine as far as Hilbert dimensions go, I think we want to catch this as a user error? This is precisely the issue I want to discuss here since it's the operator and superoperator version of this.

To be explicit, let's focus on the PauliTransferMatrix since the whole super operator interface is what is motivating me to work on all of this anyways. Currently there's no way to apply a PauliTransferMatrix to an Operator without first converting it to a SuperOperator. This is certainly easy to define just via defining an appropriate method dispatch. However the question is in what representation do I multiply these objects and in which representation do I return them? An Operator is represented in a "KetBra" operator basis, while a PauliTransferMatrix is represented with a left and right Pauli operator basis, thus I must convert one or the other before doing normal matrix multiplication. There's always a choice when the representations differ, similar to rules behind type promotion. But when they're the same representation, there's no need for conversion. Thus I think it would be good to have a concept of an Operator represented in a Pauli basis, especially if my PauliTransferMatrix preserves sparsity in such a representation, while the "KetBra" representation is dense.