API & Documentation about ESP32-S3's SIMD/AI instruction? (IDFGH-6063)

[huming2207]

Is your feature request related to a problem? Please describe.

I'm wondering will there be some kind of API and documentations for ESP32-S3's SIMD/AI instruction available soon?

Currently I've only seen you guys made a esp-dl library for now, but it is proprietary. I fully understand your needs to protect your AI algorithms. But could you please provide a SIMD low-level instruction documentation and/or API? This may be helpful for some of our projects for display waveform manipulation, data parsing as well as UI rendering. Thanks!

Describe the solution you'd like

Something like these (for ARM Neon): https://github.com/thenifty/neon-guide

Or these (for x86 AVX): https://www.intel.com/content/www/us/en/develop/documentation/cpp-compiler-developer-guide-and-reference/top/compiler-reference/intrinsics/intrinsics-for-intel-advanced-vector-extensions/intrinsics-for-load-and-store-operations-1/mm256-loadu-ps.html

Describe alternatives you've considered

Probably some kind of instruction manual should be okay for me as well.

Additional context

N/A

[huming2207]

Oh oops, I've got what I need in the esp-dsp library instead. Sorry my bad...

[igrr]

@huming2207 I think the plan is to include this information into the ESP32-S3 technical reference manual, sorry that it's not there yet.

[huming2207]

@huming2207 I think the plan is to include this information into the ESP32-S3 technical reference manual, sorry that it's not there yet.

Yep thanks @igrr .

In our use cases we have a few massive unsigned integer (uint8 or uint32) comparisons, additions and subtractions. It would be really great if there are some dedicated instructions to do so, even if it's not perfect.

Do we have any other documentation or demo code that currently available, except the esp-dsp? Or do we just need to wait for the TRM for now?

[igrr]

As you have probably figured out from esp-dsp code, there are 6x 128-bit "Q" registers at your disposal, plus an accumulator register. There are 128-bit loads and stores, element-wise additions and subtractions (although, annoyingly, only signed and saturated ones...) and comparisons (signed) as well, among other things. Element-wise operations exist in 8, 16, and 32-bit flavors.

A few pointers while we are waiting for the docs:

• https://github.com/espressif/xtensa-overlays/blob/master/xtensa_esp32s3/binutils/bfd/xtensa-modules.c is a fairly verbose representation of instruction descriptions used by binutils. It has basic information about instruction opcodes and operands, although it doesn't tell anything about their behavior. Searching this file for a part of instruction mnemonic (e.g. VLD) should give some pointers about its operands.
• EE.VCMP.{EQ,LT,GT}.S{8,16,32} are signed comparisons.
• EE.VADDS, EE.SUBS are additions and subtractions.
• EE.*LD and EE.*ST are loads and stores between RAM and Q registers. They come in many flavours, probably esp-dsp code is the best reference for these, for the time being.

I've also asked my documentation colleagues whether there is some draft reference that can be shared.

[huming2207]

@igrr thanks!

As you have probably figured out from esp-dsp code, there are 6x 128-bit "Q" registers at your disposal, plus an accumulator register. There are 128-bit loads and stores, element-wise additions and subtractions (although, annoyingly, only signed and saturated ones...) and comparisons (signed) as well, among other things. Element-wise operations exist in 8, 16, and 32-bit flavors.

I've seen some of those instructions you mentioned on esp-dsp but indeed that's for signed integer. We probably need to waste 8 bit for each array element (e.g. uint8 -> int16, or three uint8 -> int32??). I don't know if that's actually feasible if I just do it in a for-loop instead, as I need to do the conversion in advance.

I've also asked my documentation colleagues whether there is some draft reference that can be shared.

Thanks, I will wait for that while do some quick PoC if possible.

[Joy-hao]

Hi @huming2207 I already sent you the draft Chinese version of the Processor Instruction Extensions through the email address in your profile, please check. Hope this can help!

[huming2207]

@Joy-hao thanks! I'm reading it now.

Looks like it may not perfectly suit for some of our display waveform manipulation purposes due to the alignment issue I mentioned above. However the GPIO instructions looks pretty good. I hope it can speed up the bit-bang for my SWD programmer project.

I will arrange a time for some PoC if possible, and provide further feedback later on.

[huming2207]

However the GPIO instructions looks pretty good. I hope it can speed up the bit-bang for my SWD programmer project.

That is probably just the same as dedicated GPIO on ESP32-S2...? But anyway I should try it out later.