Load Session from Zipfile

[kojiromike]

When attempting to use botocore / boto3 in AWS EMR or PySpark, it's necessary to load from a zipfile. However, botocore does not support this.

+ PYTHONPATH=botocore.zip:boto3.zip
+ .venv/bin/python -c 'import boto3; client=boto3.client("rds")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/boto3.zip/boto3/__init__.py", line 91, in client
  File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/boto3.zip/boto3/session.py", line 258, in client
  File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 827, in create_client
  File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 700, in _get_internal_component
  File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 924, in get_component
  File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/session.py", line 163, in create_default_resolver
  File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/loaders.py", line 132, in _wrapper
  File "/private/var/folders/4f/2ps6fjrj2kn4klgq638844bc0000gq/T/tmp.TKx8uF60/botocore.zip/botocore/loaders.py", line 424, in load_data
botocore.exceptions.DataNotFoundError: Unable to load data for: endpoints

This is a re-submission of #1969 with permission from @gliptak, the original author.

[codecov-commenter]

Codecov Report

Merging #2437 (8e6384f) into develop (7203964) will increase coverage by 2.01%.
The diff coverage is 100.00%.

❗ Current head 8e6384f differs from pull request most recent head 07dbf44. Consider uploading reports for the commit 07dbf44 to get more accurate results

@@             Coverage Diff             @@
##           develop    #2437      +/-   ##
===========================================
+ Coverage    95.00%   97.02%   +2.01%     
===========================================
  Files           60       59       -1     
  Lines        12237    11193    -1044     
===========================================
- Hits         11626    10860     -766     
+ Misses         611      333     -278     

Impacted Files	Coverage Δ
botocore/__init__.py	100.00% <ø> (ø)
botocore/loaders.py	99.28% <100.00%> (+<0.01%)	⬆️
botocore/configprovider.py	93.46% <0.00%> (-0.99%)	⬇️
botocore/validate.py	99.44% <0.00%> (-0.56%)	⬇️
botocore/session.py	96.61% <0.00%> (-0.17%)	⬇️
botocore/stub.py	97.80% <0.00%> (-0.16%)	⬇️
botocore/response.py	92.64% <0.00%> (-0.13%)	⬇️
botocore/exceptions.py	99.50% <0.00%> (-0.05%)	⬇️
botocore/parsers.py	99.42% <0.00%> (-0.04%)	⬇️
botocore/client.py	98.68% <0.00%> (-0.04%)	⬇️
... and 35 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7203964...07dbf44. Read the comment docs.

[kojiromike]

Hello! I'll try to make sure this continues to get attention until someone engages in a real conversation about it. This is September 2021. The original PR was from Feb 8, 2020.

[kojiromike]

Hi, friends, it's now October 2021. Trick or treat, please merge this PR 👻 🍫 🍬 !

[alete89]

Can't wait for this!

[kojiromike]

Hello, it's January 2022. Maybe someone can look at this this year?

[kojiromike]

Hello, February! I hope the events of this month turn out to be less harmful than they seem right now 🇺🇦.

I also hope that maybe this PR will not be forgotten.

[gliptak]

boto3/botocore might also benefit from aws/aws-cli#6828

[kojiromike]

Hi, it's May. May I merge this some day?

[kojiromike]

Hi, we continue to occasionally have engineers run into this problem. It's a chore to have to remember to explicitly exclude boto3 from our EMR job zipfiles to avoid what appears to us to be a completely unnecessary problem.

[kojiromike]

Hello from April, 2023, I hope you've been having a great year so far. Any chance someone from AWS could address this?

[kojiromike]

Hello from June, 2023. It'd be great if AWS could take a look at this.

[kojiromike]

Hello from April 18, 2024! How's everyone doing this year? Well enough to look at this PR, maybe? :D

[RyanFitzSimmonsAK]

Thanks for the pull request, and for your patience. A more comprehensive solution with additional testing is required here. In a more recent PR (#2708) the team started to explore options here, but that pull request was insufficient, and ultimately this is something that would need to be addressed in a new major version. We'll keep the feature request open for tracking purposes, and I've marked it as needs-major-version. Thank you for contributing.