Option for orthogonal pointers in a vocabulary

[bjkomer]

The old nengo.spa had the option to do spa.Vocabulary(randomize=False) which forced all of the semantic pointers to be axis aligned and orthogonal (e.g. [1,0,0], [0,1,0] etc). I found this useful for debugging, but I don't see an option to do it in nengo_spa. Is there a reason it was removed, and it is straightforward to add?

[jgosmann]

It was removed because SPA (and even more so the implementation in this repository) is heavily based on the assumption of uniformly and randomly distributed unit vectors. Axis aligened, orthogonal vetors break this assumption and lead to unexpected behaviour in different places (see nengo/nengo#925). For this reason, we decided to remove that support, but might reconsider adding it in the future. That, however, will require some careful thinking and API design.

[jgosmann]

I'm starting to think about ways to implement support of different binding operations in Nengo SPA (#69). That might allow to address this issue too, but currently I do not understand the use case well enough. (I think I never wanted axis-aligned vectors.)

• Is the axis-alignment required or would orthogonal (non-axis aligned) vectors do?
• Axis-aligned vectors do not work particularly well with circluar convolution as binding operation (at least there are some pitfalls). Should axis-aligned vectors therefore go along with a different binding operation? How would such a binding operation be defined?
• Is changing binding operations and the structure of nengo_spa.State modules to be better suited for those vectors a problem in your debugging situations?

---

Also note that despite the removal of the feature and mentioned problems, if you really want those axis-aligned orthogonal vectors with the current Nengo SPA and default circular-convolution, you can add those manually to you vocabulary:

vocab = spa.Vocabulary(d)
for k, v in zip(sp_names, np.eye(d)):
    vocab.add(k, v)

[jgosmann]

@bjkomer?

[bjkomer]

For most of my use cases orthogonal (non-axis aligned) vectors would do. The goal was to make the similarity of all vectors zero to debug whether issues in the model were due to vectors being too close, or something else.
Another place I sometimes use this would be to quickly generate a bunch of one-hot encoded control vectors.

Often I would have these vectors be inputs to non-spa networks, so having them axis aligned would just help when viewing the value plots later in the network, without needing to feed the output into a spa.State.

I believe the code snippet you wrote would be perfect for the majority of my use cases :)