Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade kappa to 0.7.0 #1518

Closed
wants to merge 1 commit into from
Closed

Upgrade kappa to 0.7.0 #1518

wants to merge 1 commit into from

Conversation

n3il
Copy link

@n3il n3il commented May 27, 2018

Description

In the latest Kappa (0.7.0), there have been many improvements.
https://github.com/garnaat/kappa/releases/tag/0.7.0

These mostly include keeping up with enhancements in AWS' capabilities and in effect new Boto3 functions.

In my example, I was trying to add multiple S3 Event Notifications on the same bucket. This is limited in 0.6.0 because get_bucket_notification only returns one configuration. Whereas get_bucket_notification_configuration returns all the events set up for a bucket. This was limiting in my use-case, and actually had to not use Zappa because of it.

The fix in this pull request not only allows Zappa to register multiple S3 Event Notifications on a single bucket, but also takes advantage of the rest of the features brought forth by Kappa 0.7.0 - which to my knowledge include similar enhancements in Kinesis, SNS, and SQS.

GitHub Issues

#684
#688
#1319
#1461
#1442
#1545

Please tag duplicates as this is a catch-all to kappa issues - AWS Lambda resource management.
And correct if I tagged non relevant issues. This PR is scoped to mine, but upgrading may hopefully solve others.

This is the sample configuration that doesn't work without this PR, and Kappa 0.7.0.

{
  "dev": {
    "aws_region": "us-east-1",
    "profile_name": "default",
    "project_name": "test-app",
    "runtime": "python3.6",
    "apigateway_enabled": false,
    "events": [
       {
         "function": "app.test_handler",
         "event_source": {
           "key_filters": [ { "type": "prefix", "value": "test-dir/" } ],
           "arn":  "arn:aws:s3:::key-filter-test",
           "events": [ "s3:ObjectCreated:*" ]
         }
       },
       {
         "function": "app.another_test_handler",
         "event_source": {
           "key_filters": [ { "type": "prefix", "value": "another-test-dir/" } ],
           "arn":  "arn:aws:s3:::key-filter-test",
           "events": [ "s3:ObjectCreated:*" ]
         }
       }
     ]
  }
}

@coveralls
Copy link

coveralls commented May 27, 2018

Coverage Status

Coverage decreased (-15.6%) to 58.461% when pulling 3b518a0 on n3il:master into 2d1f253 on Miserlou:master.

4 similar comments
@coveralls
Copy link

Coverage Status

Coverage decreased (-15.6%) to 58.461% when pulling 3b518a0 on n3il:master into 2d1f253 on Miserlou:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-15.6%) to 58.461% when pulling 3b518a0 on n3il:master into 2d1f253 on Miserlou:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-15.6%) to 58.461% when pulling 3b518a0 on n3il:master into 2d1f253 on Miserlou:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-15.6%) to 58.461% when pulling 3b518a0 on n3il:master into 2d1f253 on Miserlou:master.

@coveralls
Copy link

coveralls commented May 27, 2018

Coverage Status

Coverage decreased (-0.008%) to 73.863% when pulling 006901c on n3il:master into 12ac82e on Miserlou:master.

@matt-land
Copy link

Any updates to this? Having same problem, would like to see this merge.

@n3il n3il closed this Sep 3, 2018
@n3il n3il reopened this Sep 3, 2018
@n3il
Copy link
Author

n3il commented Sep 3, 2018

I'm not familiar with Zappa's final review process, but as far as I can tell this is ready to go (I accidentally closed and reopened this to rebase and fix conflicts).

To note, this extends Kappa's S3EventResource, but that can be removed as shown in this commit https://github.com/n3il/Zappa/commit/b915b16e11b09bea038dba8d304959c1a3f732a0. Which is dependent on merging garnaat/kappa#120.
That said, with or without garnaat/kappa#120, this will work seamlessly with a random id, as the notification id is internal to kappa and won't be displayed in AWS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants