Move JSONFileCache to botocore

[JordonPhillips]

This moves the file cache to botocore and allows users to set their
own cache via create_credential_resolver.

Future work: allow setting this via the config file and/or some other
easy method of configuration.

Closes #1157

[codecov-io]

Codecov Report

Merging #1338 into develop will increase coverage by <.01%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop    #1338      +/-   ##
===========================================
+ Coverage    98.13%   98.13%   +<.01%     
===========================================
  Files           46       46              
  Lines         7701     7732      +31     
===========================================
+ Hits          7557     7588      +31     
  Misses         144      144

Impacted Files	Coverage Δ
botocore/credentials.py	98.39% <100%> (+0.06%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f1de3d7...b915ca3. Read the comment docs.

[kyleknap]

Looks fine. I just had a question about the default location of the cache.

[kyleknap]

I think we should consider using a different directory than what the CLI uses. I have not thought through whether there will be edge cases to this, but it may be surprising if you are running a script to learn that your temp credentials were being cached under a cli directory. I wonder if it makes sense to call it boto or python instead?

[JordonPhillips]

sounds reasonable

[JordonPhillips]

done

[kyleknap]

Looks good to me. Don't forget to add an enhancement changelog. 🚢

[JordonPhillips]

Will do. Also gonna put up a PR for the cli to switch to using this.

[et304383]

Can we get a quick explanation of how to use this? Is it possible to cache the MFA globally yet?

[mixja]

@et304383 - here is an example of leveraging the AWS CLI cache. Note you could create your own custom cache location if you wanted.

from botocore import credentials
import botocore.session
import boto3
import os

# By default the cache path is ~/.aws/boto/cache
cli_cache = os.path.join(os.path.expanduser('~'),'.aws/cli/cache')

# Construct botocore session with cache
session = botocore.session.get_session()
session.get_component('credential_provider').get_provider('assume-role').cache = credentials.JSONFileCache(cli_cache)

# Create boto3 client from session
client = boto3.Session(botocore_session=session).client('ecs')

[bgabrhelik]

If you need to specify a profile you have to specify it in botocore.Session. It's too late to specify it in boto3.Session as AssumeRoleProvider instantiates with wrong 'default' profile.
So the code above will be

from botocore import credentials
import botocore.session
import boto3
import os

# By default the cache path is ~/.aws/boto/cache
cli_cache = os.path.join(os.path.expanduser('~'),'.aws/cli/cache')

# Construct botocore session with cache
session = botocore.session.Session(profile='prod')
session.get_component('credential_provider').get_provider('assume-role').cache = credentials.JSONFileCache(cli_cache)

# Create boto3 client from session
client = boto3.Session(botocore_session=session, profile_name = 'prod').client('ecs')

[mixja]

@bgabrhelik - it's generally better practice to rely on the external environment to configure your credentials, rather than specify an AWS profile name in code. This approach has benefit of allowing the code to work in EC2/ECS environments where you may be using EC2 instance profile or ECS task role.

My code snippet works in a local desktop environment assuming you have set profile name as environment variable - e.g. export AWS_PROFILE=prod

[mikehas]

Does anyone know if there's a feature request to implement this as a configuration option, like @JordonPhillips mentioned in initial description?

Future work: allow setting this via the config file and/or some other
easy method of configuration.