lambda: jsonschema import fails with lambda using aws-cdk 2.87.0

[tzven0]

Describe the bug

Our lambda uses jsonschema, installed through an requirements.txt file at deployment time into a lambda layer which gets attached to our lambda.
We realized the following behaviour after the update to 2.87.0 of aws-cdk stated in our CloudWatch logs of the lambda:

START RequestId: 28ee9b79-79ab-4d7a-bbe9-3c0cbe078be9 Version: $LATEST
--
[ERROR] Runtime.ImportModuleError: Unable to import module 'index': No module named 'rpds.rpds'Traceback (most recent call last):
[AWS Parameters and Secrets Lambda Extension] 2023/07/10 04:30:31 INFO ready to serve traffic
END RequestId: 28ee9b79-79ab-4d7a-bbe9-3c0cbe078be9

We narrowed down the issue to the import of jsonschema. It seems to be working if we switch back to aws-cdk 2.86.0.

Expected Behavior

import jsonschema in lambda will not give error.

Current Behavior

Error of

[ERROR] Runtime.ImportModuleError: Unable to import module 'index': No module named 'rpds.rpds'Traceback (most recent call last):

without trace given as soon as jsonschema is getting imported in our lambda.

Reproduction Steps

lambda_requirements.txt:

requests
boto3
jsonschema

cdk code for lambda with layer from lambda_requirements.txt:

from aws_cdk.aws_lambda import LayerVersion
from aws_cdk.aws_lambda_python_alpha import PythonLayerVersion, PythonFunction

lambda_layer = PythonLayerVersion(
	self,
	id=f"lambda-layer",
	layer_version_name=f"lambda-layer",
	# Python3.9, ARM-64 should be used, but there is a bug in AWS CDK currently, hence this is disabled
	compatible_runtimes=[aws_lambda.Runtime.PYTHON_3_8],
	# compatible_architectures=[aws_lambda.Architecture.ARM_64],
	entry=os.path.join(<path to requirements file>, "lambda_requirements"),
)

lambda = PythonFunction(
	self,
	id=f"lambda",
	function_name=f"lambda",
	entry=<path to lambda code>,
	index="index.py",
	role=lambda_role,
	# Python3.9, ARM-64 should be used, but there is a bug in AWS CDK currently, hence this is disabled
	architecture=aws_lambda.Architecture.ARM_64,
	runtime=aws_lambda.Runtime.PYTHON_3_8,
	memory_size=1024,
	layers=[
		lambda_layer,
	],
	log_retention=aws_logs.RetentionDays.SIX_MONTHS,
	timeout=Duration.minutes(1),
	environment={
		"some_env_var": "value",
	},
)

Possible Solution

Switching back to aws-cdk 2.86.0 resolves this issue.

Additional Information/Context

From pipeline using aws-cdk 2.87.0 (failing):

Downloading aws_cdk_lib-2.87.0-py3-none-any.whl
Downloading constructs-10.2.69-py3-none-any.whl
Downloading aws_cdk.aws_lambda_python_alpha-2.87.0a0-py3-none-any.whl
Downloading jsonschema-4.18.0-py3-none-any.whl

From pipeline using aws-cdk 2.86.0 (working):

Downloading aws_cdk_lib-2.86.0-py3-none-any.whl
Downloading constructs-10.2.69-py3-none-any.whl
Downloading aws_cdk.aws_lambda_python_alpha-2.87.0a0-py3-none-any.whl
Downloading jsonschema-4.18.0-py3-none-any.whl

CDK CLI Version

2.87.0

Framework Version

No response

Node.js Version

16

OS

linux

Language

Python

Language Version

No response

Other information

No response

[pahud]

Thank you for your report. I just can't find any root cause off the top of my head.

As I can't see your lambda code, are you able to create a mini repo that I can clone and reproduce this error in my account?

[nicolavs]

I found that if you use 2.87.0 of aws-cdk
and keep jsonschema to version jsonschema==4.17.3

it still works ok

[BernardVas]

I found that if you use 2.87.0 of aws-cdk and keep jsonschema to version jsonschema==4.17.3

it still works ok

Same here!

[darioackermann]

Also see this issue: python-jsonschema/jsonschema#1117.
The authors are well-aware that their minor upgrade broke things for users, so you'll likely have to pin this dependency for longer ever

[Leo10Gama]

I'm unable to replicate the bug, so I'm assuming the previous commits and lack of comments means the issue has been fixed, and it can be safely closed. For reference, this was the code I used, in CDK 2.140.0:

import os
from aws_cdk import Duration, Stack
from aws_cdk.aws_lambda import Runtime, Architecture
from aws_cdk.aws_lambda_python_alpha import PythonLayerVersion, PythonFunction
from aws_cdk.aws_logs import RetentionDays
from constructs import Construct

class IssueJsonschemaStack(Stack):

    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        lambda_layer = PythonLayerVersion(
            self,
            id=f"lambda-layer",
            layer_version_name=f"lambda-layer",
            compatible_runtimes=[Runtime.PYTHON_3_8],
            entry=os.path.join("lambda"),
        )

        _lambda = PythonFunction(
            self,
            id=f"lambda",
            function_name=f"lambda",
            entry="lambda",
            index="index.py",
            handler="lambda_handler",
            architecture=Architecture.ARM_64,
            runtime=Runtime.PYTHON_3_8,
            memory_size=1024,
            layers=[
                lambda_layer,
            ],
            log_retention=RetentionDays.SIX_MONTHS,
            timeout=Duration.minutes(1),
            environment={
                "some_env_var": "value",
            },
        )

If the problem persists, please feel free to open another issue.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.