Added langchain-aws package.

[3coins]

Description

Adds langchain-aws package.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[andrii-i]

I built sagemaker-distribution based on this PR with aws-langchain included and ran a newly-added test cases with python -m pytest -n auto -vv -rs -k "langchain-aws" --local-image-version 1.8.0, they pass:

Note irregularities with python src/main.py create-minor-version-artifacts --base-patch-version=$BASE_PATCH_VERSION --force command that is supposed to be run after adding {cpu/gpu}.additional_packages_env.in into new version folder accordingly to the docs. It checks if new version folder exists and if it does, checks for existence of {cpu/gpu}.additional_packages_env.in file and pulls new packages into {cpu/gpu}.env.in. But for me it sometimes works incorrectly and overwrites new version folder without checking for {cpu/gpu}.additional_packages_env.in or pulling packages from it.

[andrii-i]

Note that if all tests are ran, langchain-aws tests pass but tests for 5 other packages fail. For the sake of experiment I tried building 1.8.0 without adding langchain-aws or anything else and I’m getting the same errors.

FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_gpu[keras.test.Dockerfile-required_packages0] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_gpu[scipy.test.Dockerfile-required_packages4] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_gpu[pandas.test.Dockerfile-required_packages7] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_gpu[sm-python-sdk.test.Dockerfile-required_packages8] - assert 1 == 0
FAILED test/test_dockerfile_based_harness.py::test_dockerfiles_for_gpu[pytorch.examples.Dockerfile-required_packages9] - assert 1 == 0
===================================================================================== 5 failed, 20 passed, 2 skipped, 4 warnings in 7016.70s (1:56:56) =====================================================================================
Traceback (most recent call last):
File “/local/home/aieroshe/Docs/sagemaker-distribution/src/main.py”, line 432, in
parse_args(get_arg_parser())
File “/local/home/aieroshe/Docs/sagemaker-distribution/src/main.py”, line 428, in parse_args
args.func(args)
File “/local/home/aieroshe/Docs/sagemaker-distribution/src/main.py”, line 193, in build_images
_test_local_images(image_ids, args.target_patch_version)
File “/local/home/aieroshe/Docs/sagemaker-distribution/src/main.py”, line 219, in _test_local_images
assert exit_code == 0, f”Tests failed with exit codes: {exit_codes} against: {image_ids}”
^^^^^^^^^^^^^^
AssertionError: Tests failed with exit codes: [] against: []

All packages for tests of which errors are thrown except sm-python-sdk are listed in v1.8.0/gpu.env.out. At the same time sm-python-sdk is also not listed in v1.7.1/gpu.env.out .

[balajisankar15]

Can we add more tests

[andrii-i]

Thank you for giving this PR a look @balajisankar15. Is this blocking / could we add more tests in the follow-up PR?

[3coins]

@balajisankar15
We can add more tests around importing other sub-modules to test their importability. Anything other than that requires an elaborate setup with boto3 credentials and service endpoint infrastructure (bedrock, kendra, vectorstores etc.) to test each of these components, which we don't plan to do for this library.
We will update the PR with these and open for a review. Thanks.

[3coins]

@balajisankar15
Added more import tests to the test file. Can you take a look. Thanks.

[andrii-i]

@balajisankar15 @claytonparnell
Rebased this PR on main and updated the path of {cpu/gpu}.additional_packages_env.in to build_artifacts/v1/v1.9/v1.9.0/. I also built an image and ran langchain-aws tests, both successfully. Please give this another look.

[3coins]

@claytonparnell @balajisankar15
Is there a plan to merge this PR, is there an ETA that you can provide?