-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AWS error response when request date does not match signing key date #10
Comments
GitHub issue: #10 See issue report and docstrings for detailed discussion of new functionality. Package changes: * Added StrictAWS4Auth and PassiveAWS4Auth classes * Added exceptions module with new exceptions: RequestsAWS4AuthException, DateMismatchError, NoSecretKeyError, DateFormatError * Now only tested on Pythons 2.7 and 3.3 and up, to match Requests supported versions (dropped support for Python 3.2) AWS4Auth changes: * Added regenerate_signing_key() method, to allow regeneration of current signing key with parameter overrides * Added methods for checking and extracting dates from requests: get_request_date(), parse_date(), handle_date_mismatch() * __call__() now checks for a date header in the request and attempts to automatically regenerate the signing key with the request date if request date differs from the signing key date * Can now supply a date to the constructor * Changed default included sig headers to include 'date' header if present * Docstring improvements AWS4SigningKey changes: * Added new store_secret_key instantiation parameter which allows control of whether the secret key is stored in the instance * Deprecated the amz_date property in favour of just 'date' * Spelling typo fix in AWS4AuthSigningKey module docstring. Thanks to jhgorrell Test changes: * Many new tests for the above functionality * Added tests for generating canonical path, including test for fix added in 0.7 for percent encoding of paths * Added tests for generating canonical querystrings
Code changes to implement this commited to the 0.8_dev branch. Take a look and let me know if you have any problems or thoughts. There are more details on the implementation in the updated module docstrings. I'll leave this out there for testing and comment until 30th Dec, then I'll merge to master and release on PyPi. |
Just gave this a try and it seems to be working. Thank you for all the work. |
Version 0.8 has been published to PyPi, the 0.8_dev branch has been merged to master and tagged 0.8. |
setup.py: The "requests-aws4auth" package has been updated to the 0.8 version. The details can be found here: * tedder/requests-aws4auth#10. * tedder/requests-aws4auth#3.
Can any one tell me how to add data in the request. I am getting signature mismatch while adding data |
Hi @rahulmuraliqllabscom, are you still having trouble with this? Can you post some sample code? |
Hi Sam I just went through the aws signing tutorial and wrote a code snippet myself. Thanks for showing concern. |
GitHub issue: #10 See issue report and docstrings for detailed discussion of new functionality. Package changes: * Added StrictAWS4Auth and PassiveAWS4Auth classes * Added exceptions module with new exceptions: RequestsAWS4AuthException, DateMismatchError, NoSecretKeyError, DateFormatError * Now only tested on Pythons 2.7 and 3.3 and up, to match Requests supported versions (dropped support for Python 3.2) AWS4Auth changes: * Added regenerate_signing_key() method, to allow regeneration of current signing key with parameter overrides * Added methods for checking and extracting dates from requests: get_request_date(), parse_date(), handle_date_mismatch() * __call__() now checks for a date header in the request and attempts to automatically regenerate the signing key with the request date if request date differs from the signing key date * Can now supply a date to the constructor * Changed default included sig headers to include 'date' header if present * Docstring improvements AWS4SigningKey changes: * Added new store_secret_key instantiation parameter which allows control of whether the secret key is stored in the instance * Deprecated the amz_date property in favour of just 'date' * Spelling typo fix in AWS4AuthSigningKey module docstring. Thanks to jhgorrell Test changes: * Many new tests for the above functionality * Added tests for generating canonical path, including test for fix added in 0.7 for percent encoding of paths * Added tests for generating canonical querystrings
GitHub issue: #10 See issue report and docstrings for detailed discussion of new functionality. Package changes: * Added StrictAWS4Auth and PassiveAWS4Auth classes * Added exceptions module with new exceptions: RequestsAWS4AuthException, DateMismatchError, NoSecretKeyError, DateFormatError * Now only tested on Pythons 2.7 and 3.3 and up, to match Requests supported versions (dropped support for Python 3.2) AWS4Auth changes: * Added regenerate_signing_key() method, to allow regeneration of current signing key with parameter overrides * Added methods for checking and extracting dates from requests: get_request_date(), parse_date(), handle_date_mismatch() * __call__() now checks for a date header in the request and attempts to automatically regenerate the signing key with the request date if request date differs from the signing key date * Can now supply a date to the constructor * Changed default included sig headers to include 'date' header if present * Docstring improvements AWS4SigningKey changes: * Added new store_secret_key instantiation parameter which allows control of whether the secret key is stored in the instance * Deprecated the amz_date property in favour of just 'date' * Spelling typo fix in AWS4AuthSigningKey module docstring. Thanks to jhgorrell Test changes: * Many new tests for the above functionality * Added tests for generating canonical path, including test for fix added in 0.7 for percent encoding of paths * Added tests for generating canonical querystrings
Description
Reported by @ipartola in pull request #3.
When the date of a request, as given in either the
Date
or theX-Amz-Date
header, is different from the date in theAWS4Auth
object's signing key scope, then an invalid signature is generated and AWS sends an authentication error in response to the request.This issue affects
request-aws4auth
versions 0.7 and earlier.As descibed by @ipartola:
"The bug is that the timestamp for the signing key is generated when the object is instantiated. Once we cross a date boundary it seems AWS requires that the signing key's scope date match the x-amz-date header. The solution I propose is to continually set the current date for the signing key to match the x-amz-date header."
Discussion
The AWS documentation for date handling in signing key scopes is here: http://docs.aws.amazon.com/general/latest/gr/sigv4-date-handling.html
It could be argued that this isn't a bug in requests-aws4auth, since it should be up to the calling code to ensure that the scope of the signing key is kept up to date by generating a new
AWS4Auth
instance. But it's obvious that this is a common situation and that it would be a very useful feature if requests-aws4auth could handle this automatically.Allowing the key to automatically regenerate raises problems for sharing an AWS4Auth instance between multiple threads, since if the key can be regenerated automatically at any time then it could be modified by another thread while it is being used to sign a request.
It also means that the AWS secret key will need to be stored in the signing key, so that the key can be regenerated without having to escape back out to the code generating the request.
Proposed Solution
As suggested by @ipartola, the key needs to be regenerated with a date that matches the date in the request.
As this is likely to be a common situation, and a bit of a WTF moment when new users hit it, it is proposed that automatic key regeneration is made the default behaviour for new instances of
AWS4Auth
. However, the behaviour needs to be made configurable, for situations where people want to share instances between threads, or don't want the key stored in the instance, or don't want automatic regeneration for other reasons. Retaining a way of duplicating the current behaviour (i.e. signing the request with a bad signature) is also desirable for backward compatibility if needed.The proposal is to modify the
AWS4Auth
class to check a request'sX-Amz-Date
orDate
headers against the scope date, and if it is different then automatically regenerate the signing key with a scope that uses the new date. This means when new instances ofAWS4Auth
are created, the AWS secret key will need to be supplied.A new method will be added to
AWS4Auth
: `regenerate_signing_key()``. Calling this will trigger an immediate regeneration of the instance's signing key. Parameters can be supplied to this method to set new values for the key scope parameters: region, service and date. This method will be used when a automatic key regeneration is needed, but can also be called by other code if a manual regeneration is needed.Two new subclasses of
AWS4Auth
will be added:StrictAWS4Auth
andPassiveAWS4Auth
.StrictAWS4Auth
will not regenerate the signing key when a date mismatch is encountered, it will instead raise a new exception,DateMismatchError
. This will allow the calling code to handle the situation itself if desired.PassiveAWS4Auth
will do no date checking, and will sign the request and send it even if the request date does not match the scope date. This mimics the current behaviour of v0.7AWS4Auth
.Further, a new parameter will be added to
AWS4SigningKey
:store_secret_key
. If this is set toTrue
, which will be the default, the secret key is stored in the instance, not thrown away as it is in v0.7. By creating instances ofAWS4Auth
or its subclasses usingAWS4SigningKey
instances created withstore_secret_key
set toFalse
, you can control whether the secret key is stored in anAWS4Auth
instance or not.Examples
Regular usage, usual case with no date change,
X-Amz-Date
header is automatically added to the request by the AWS4Auth signing process, using today's date, since Requests doesn't automatically add a date header. This is the same usage and behaviour as current 0.7 version when not crossing a date boundary:Next this shows regular usage, but crossing a date boundary and AWS4Auth automatically regenerating the signing key. This is the main use case for this fix. (In this case the
X-Amz-Date
header is again added automatically by the AWS4Auth authentication using the current date. It is added before the scope/request date check is made, and can be different to the scope date):Next is an example with
StrictAWS4Auth
. With this subclass the key is not automatically regenerated, instead aDateMismatchError
is raised when the request date does not match the signing key scope date:The key can now be manually regenerated to match the new day's date:
Example again using
StrictAWS4Auth
, but also where the secret key is not stored in the auth instance:Here the manual regeneration fails because there is no stored secret key. To regenerate, the secret key must be passed to
regenerate_signing_key()
:The key is not stored in the new key either, because the new key uses the value of
store_secret_key
in the old key, which isFalse
in this case.Any feedback on these proposed changes gratefully received.
The text was updated successfully, but these errors were encountered: