Yocto build: optee-os 3.8.0: ModuleNotFoundError: No module named 'Cryptodome'

[RajeshRavi-brcm]

I 'm trying to build optee_os 3.8.0 thorugh Yocto on Ubuntu 18.04 LTS build machine.
I see error: ModuleNotFoundError: No module named 'Cryptodome'

Build components' details

optee_os version: 3.8.0
Build machine: Ubuntu 18.04 LTS
Yocto poky verison: 3.0
python3-pycryptodome version: 3.9.4:
meta-openembedded/meta-python/recipes-devtools/python/python3-pycryptodome_3.9.4.bb

I included following lines in my Yocto optee_os.bb recipe to handle dependecies:

inherit deploy autorev-pv python3native

DEPENDS += "python3-pyelftools-native"
DEPENDS += "python3-pycryptodome-native"
##DEPENDS += "python3-pycrypto-native"

Error Log

SIGN    out/arm-plat-bcm/ta/avb/023f8f1a-292a-432b-8fc4-de8471358067.ta
| Traceback (most recent call last):
|   File "out/arm-plat-bcm/export-ta_arm64/scripts/sign_encrypt.py", line 258, in <module>
|     main()
|   File "out/arm-plat-bcm/export-ta_arm64/scripts/sign_encrypt.py", line 131, in main
|     from Cryptodome.Signature import pss
| ModuleNotFoundError: No module named 'Cryptodome'
| ta/arch/arm/link.mk:100: recipe for target 'out/arm-plat-bcm/ta/avb/023f8f1a-292a-432b-8fc4-de8471358067.ta' failed
| make: *** [out/arm-plat-bcm/ta/avb/023f8f1a-292a-432b-8fc4-de8471358067.ta] Error 1
| make: *** Waiting for unfinished jobs....
| + die oe_runmake failed
| + bbfatal_log oe_runmake failed

[RajeshRavi-brcm]

Interestingly, Cryptodome module is found in python counter part of the recipe: python-pycryptodomex_3.9.4.bb.

Log

find tmp/ -path '*/Cryptodome/init.py'
tmp/work/x86_64-linux/python-pycryptodomex-native/3.9.4-r0/pycryptodomex-3.9.4/lib/Cryptodome/init.py
tmp/work/x86_64-linux/python-pycryptodomex-native/3.9.4-r0/pycryptodomex-3.9.4/build/lib.linux-x86_64-2.7/Cryptodome/init.py
tmp/work/x86_64-linux/python-pycryptodomex-native/3.9.4-r0/image/home/build/ns3/poky/build/tmp/work/x86_64-linux/python-pycryptodomex-native/3.9.4-r0/recipe-sysroot-native/usr/lib/python2.7/site-packages/Cryptodome/init.py
tmp/work/x86_64-linux/python-pycryptodomex-native/3.9.4-r0/sysroot-destdir/home/build/ns3/poky/build/tmp/work/x86_64-linux/python-pycryptodomex-native/3.9.4-r0/recipe-sysroot-native/usr/lib/python2.7/site-packages/Cryptodome/init.py
tmp/sysroots-components/x86_64/python-pycryptodomex-native/usr/lib/python2.7/site-packages/Cryptodome/init.py

[jforissier]

OP-TEE wants Python 3, it seems you have the Cryptodome module only for Python 2 (2.7).

[vchong]

Not sure if this will work, but also try below. Note the 'x' after 'pycryptodome'.

DEPENDS += "python3-pycryptodomex-native"

[RajeshRavi-brcm]

Thanks a lot jforissier & vchong.

There is no such recipe as python3-pycryptodomex-native for python3.

As per my understanding, python3-pycryptodome is somehwat superset of python3-pycrypto
python-pycryptodomex provides only cryptodome functionality but not pycrypto functionality.

Log

bitbake -v python3-pycryptodomex-native
Loading cache: 100% |##############################################################################################################################################################################| Time: 0:00:02
Loaded 12039 entries from dependency cache.
Parsing recipes: 100% |############################################################################################################################################################################| Time: 0:00:26
Parsing of 2743 .bb files complete (2669 cached, 74 parsed). 12261 targets, 1069 skipped, 4 masked, 0 errors.
ERROR: Nothing PROVIDES 'python3-pycryptodomex-native'. Close matches:
python3-pycryptodome-native
python-pycryptodomex-native
python3-pycrypto-native

Summary: There was 1 ERROR message shown, returning a non-zero exit code.

[vchong]

Maybe these can help:

https://www.mail-archive.com/[email protected]//msg11149.html
https://www.mail-archive.com/[email protected]//msg11151.html

Seems like they've created a python3-pycryptodomex-native recipe that includes python-pycryptodomex.inc.

[RajeshRavi-brcm]

Thanks a lot vchong. This is very helpful.

[RajeshRavi-brcm]

Following patches worked finally for me:
python3-pycryptodomex changes from the above links worked but optee_os patch did not work because it removed inherit python3native and

inherit deploy autorev-pv python3native
DEPENDS += "python3-pycrypto-native"
DEPENDS += "python3-pyelftools-native"

Following are the python dependency contents of optee_os recipe which worked for me:

optee_os.bb:

DEPENDS += "python3-pycrypto-native"
DEPENDS += "python3-pyelftools-native"
DEPENDS += "python3-pycryptodomex-native"

Following are patches for new files for python3-pycryptodomex recipe:

diff --git a/meta-python/recipes-devtools/python/python-pycryptodomex.inc b/meta-python/recipes-devtools/python/python-pycryptodomex.inc
new file mode 100644
index 000000000..d00a07647
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python-pycryptodomex.inc
@@ -0,0 +1,29 @@
+SUMMARY = "Cryptographic library for Python"
+DESCRIPTION = "PyCryptodome is a self-contained Python package of low-level\
+ cryptographic primitives."
+HOMEPAGE = "http://www.pycryptodome.org"
+LICENSE = "PD & BSD-2-Clause"
+LIC_FILES_CHKSUM = "file://LICENSE.rst;md5=6dc0e2a13d2f25d6f123c434b761faba"
+
+SRC_URI[md5sum] = "46ba513d95b6e323734074d960a7d57b"
+SRC_URI[sha256sum] = "22d970cee5c096b9123415e183ae03702b2cd4d3ba3f0ced25c4e1aba3967167"
+
+inherit pypi
+
+RDEPENDS_${PN} += " \
+    ${PYTHON_PN}-io \
+    ${PYTHON_PN}-math \
+"
+
+RDEPENDS_${PN}-tests += " \
+    ${PYTHON_PN}-unittest \
+"
+
+PACKAGES =+ "${PN}-tests"
+
+FILES_${PN}-tests += " \
+    ${PYTHON_SITEPACKAGES_DIR}/Cryptodome/SelfTest/ \
+    ${PYTHON_SITEPACKAGES_DIR}/Cryptodome/SelfTest/__pycache__/ \
+"
+
+BBCLASSEXTEND = "native nativesdk"
diff --git a/meta-python/recipes-devtools/python/python3-pycryptodomex_3.9.4.bb b/meta-python/recipes-devtools/python/python3-pycryptodomex_3.9.4.bb
new file mode 100644
index 000000000..be66430e1
--- /dev/null
+++ b/meta-python/recipes-devtools/python/python3-pycryptodomex_3.9.4.bb
@@ -0,0 +1,2 @@
+require python-pycryptodomex.inc
+inherit setuptools3

[RajeshRavi-brcm]

optee_os.bb:
DEPENDS += "python3-pycrypto-native"
DEPENDS += "python3-pyelftools-native"
DEPENDS += "python3-pycryptodomex-native"

Attached is the patch for adding
python3-pycryptodomex.patch.txt
new python3-pycryptodomex recipe renamed as .txt file.