OOTB generated python CF resource cannot be used with cfn test

[carpnick]

Repro:

• Install latest cloudformation-cli
• Install master branch of this repo
• Generate a new resource type - in my example I used python3.9
• run cfn submit --dry-run
• run sam build
• in new terminal run sam local start-lambda
• run just one contract test -cfn test -- -k contract_create_delete
• Error from cryptography library:
• Unable to import module 'dd_dd_test1.handlers': cannot import name 'ObjectIdentifier' from 'cryptography.hazmat.bindings._rust' (unknown location)

Environment:

• cfn version - cfn 0.2.28
• SAM version - SAM CLI, version 1.66.0
• Operating System - macOS Monterey - 12.3.1
• Architecture - Apple M1 Pro (arm64)
• plugin version - cloudformation-cli-python-plugin @ git+https://github.com/aws-cloudformation/cloudformation-cli-python-plugin.git@fad3b0740a76c7bad0be18b08cb46f6e22973bde (master as of 01/17/2023)
• requirements.txt (generated) - cloudformation-cli-python-lib>=2.1.9
• Resolved cryptography library - cryptography 39.0.0

Research:

• User having issues with python3.8 in lambda environment - https://stackoverflow.com/questions/75129142/aws-lambda-cannot-import-name-objectidentifier-from-cryptography-hazmat-b
• aws-encryption-sdk only requires 3.4.0 or above currently. This is used here. And defined here

[carpnick]

Full trace:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/var/task/dd_dd_test1/handlers.py", line 3, in <module>
    from cloudformation_cli_python_lib import (
  File "/var/task/cloudformation_cli_python_lib/__init__.py", line 4, in <module>
    from .hook import Hook  # noqa: F401
  File "/var/task/cloudformation_cli_python_lib/hook.py", line 9, in <module>
    from .cipher import Cipher, KmsCipher
  File "/var/task/cloudformation_cli_python_lib/cipher.py", line 4, in <module>
    import aws_encryption_sdk  # type: ignore
  File "/var/task/aws_encryption_sdk/__init__.py", line 19, in <module>
    from aws_encryption_sdk.caches.local import LocalCryptoMaterialsCache  # noqa
  File "/var/task/aws_encryption_sdk/caches/__init__.py", line 25, in <module>
    from ..internal.formatting.encryption_context import serialize_encryption_context
  File "/var/task/aws_encryption_sdk/internal/formatting/__init__.py", line 14, in <module>
    from .serialize import serialize_header
  File "/var/task/aws_encryption_sdk/internal/formatting/serialize.py", line 17, in <module>
    import aws_encryption_sdk.internal.defaults
  File "/var/task/aws_encryption_sdk/internal/defaults.py", line 16, in <module>
    import aws_encryption_sdk.identifiers
  File "/var/task/aws_encryption_sdk/identifiers.py", line 18, in <module>
    from cryptography.hazmat.primitives.asymmetric import ec, padding, rsa
  File "/var/task/cryptography/hazmat/primitives/asymmetric/ec.py", line 10, in <module>
    from cryptography.hazmat._oid import ObjectIdentifier
  File "/var/task/cryptography/hazmat/_oid.py", line 7, in <module>
    from cryptography.hazmat.bindings._rust import (
ImportError: cannot import name 'ObjectIdentifier' from 'cryptography.hazmat.bindings._rust' (unknown location)

[carpnick]

Found out issue:

1st issue:
Template assumes X86. If you are on an Apple Mac M1, you have to modify the SAM template to use arm64 using the Architectures property. User error.

Enhancement Request - With cfn init ask user architecture and support arm64 and x86/AMD64 and modify templating of new resources as required.

2nd issue:
Fundamentally we have a broken workflow I think. Any python package that requires native install cannot support the copysource workflow that SAM supports since it doesnt copy over all files. It excludes certain files we need for native packages. Broken because of this line more than likely.

Confirmed:

2023-01-17 20:04:34,335 | File (_padding.abi3.so) is in ignored set, skipping it
2023-01-17 20:04:34,335 | File (_openssl.abi3.so) is in ignored set, skipping it

This fundamentally breaks native zipping with SAM CLI to allow contract testing locally. Posting workaround below.

AWS team - is there any plans to make it so we can ship native docker container images as an integration instead? If not, how will we overcome the native dependency install? SAM CLI Change? aws-lambda-builders change? In this case we cannot say not supported since cryptography is in the cloudformation-cli-python-plugin dependency tree, let alone customers having their own dependencies.

As I see it - this completely blocks cfn test use cases running locally with sam local start-lambda using the documented way of running sam build and the templated default that comes from cfn init.

---

---

Complete workaround:

• run cfn submit --dry-run
• Change generated template CodeUri line to CodeUri: src/. This allows the sam build call to rebuild for the SAM context.
• instead of running sam build run sam build -u -m requirements.txt
• This will reinstall the requirements but within the SAM context.
• It also still copies the source over.
• This is more of a hack workaround workflow to get it working properly. It was over 2 days of effort to get to the point I could run the hello world contract tests for python locally and start making code changes.