Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[request] support for DynamoDB #183

Closed
nickveenhof opened this issue Apr 29, 2015 · 47 comments
Closed

[request] support for DynamoDB #183

nickveenhof opened this issue Apr 29, 2015 · 47 comments
Labels
task/feature Requests for new features in Kong

Comments

@nickveenhof
Copy link

Cassandra is quite a beast to setup and maintain. If Kong runs in EC2,it would be great if it could also write it's data to DynamoDB (very similar to Cassandra) and use that as a service. This allows engineers to have less maintenance on Cassandra and focus more on Kong.

Great project!
@thibaultcha
Copy link
Member

We are aware of this, and support for other databases is on the roadmap, but we have a lot of things to do before that.

@thibaultcha thibaultcha changed the title Add support for DynamoDB instead of Cassandra Add support for DynamoDB Apr 29, 2015
@thibaultcha thibaultcha changed the title Add support for DynamoDB [request] support for DynamoDB Apr 29, 2015
@subnetmarco subnetmarco added the dao label May 1, 2015
@jakehow
Copy link

jakehow commented May 14, 2015

👍

@DavidTPate
Copy link

For me, supporting another database which I can have managed (see RDS/DynamoDB/etc) would be awesome.

@sonicaghi
Copy link
Member

btw.. @DavidTPate @nickveenhof you can have Cassandra in the cloud through https://www.instaclustr.com/

@jasonfill
Copy link

+1

@berkay
Copy link

berkay commented Jun 13, 2015

+1
IMO, Cassandra dependency is a major problem. It would be unwise for anyone to build such a critical service (API) with a dependency on a data store that one does not fully grok but have to manage. DynamoDB would be a good fit.

@sonicaghi sonicaghi mentioned this issue Jun 14, 2015
Closed
@desmondmorris
Copy link

+1

5 similar comments
@glkz
Copy link

glkz commented Jun 29, 2015

👍

@owahlen
Copy link

owahlen commented Jun 30, 2015

+1

@vmattos
Copy link

vmattos commented Jun 30, 2015

👍

@kenwarner
Copy link

+1

@skippy
Copy link

skippy commented Jul 2, 2015

+1

@udleinati
Copy link

+1

@vwal
Copy link

vwal commented Aug 10, 2015

+1

While PostgreSQL support would be great, support for DynamoDB would be even more critical for AWS applications. If KONG is used on Auto-Scaling Group instances, DynamoDB access would allow much lighter instances to be used as Cassandra is a resource hog.

@sonicaghi
Copy link
Member

@vwal that is true. and we're def going to do both. We just prioritized Postgres at the moment given more community demand for it.

@seiffert
Copy link

seiffert commented Sep 1, 2015

👍

@gankkank
Copy link

gankkank commented Oct 2, 2015

👍 Glad to hear that

@harlow
Copy link

harlow commented Oct 3, 2015

While PostgreSQL support would be great, support for DynamoDB would be even more critical for AWS applications. If KONG is used on Auto-Scaling Group instances, DynamoDB access would allow much lighter instances to be used as Cassandra is a resource hog.

Agreed. 👍 for DynamoDB.

@jtconnor
Copy link

+1 for DynamoDB

@RussellLuo
Copy link

I think it will be awesome if KONG provides the extension/plugin support for different databases.

@eric1iu
Copy link

eric1iu commented Dec 10, 2015

+1

3 similar comments
@forhot2000
Copy link

+1

@truman-misfit
Copy link

+1

@exquisite2007
Copy link

+1

@nhuray
Copy link

nhuray commented Jan 16, 2016

+1

@lindo-jmm
Copy link

Installing Cassandra is a complete nightmare.

@sonicaghi
Copy link
Member

@lindo-jmm https://www.instaclustr.com/products/kong/

@lindo-jmm
Copy link

I saw! Starting at $299/mo is a little steep for us unfortunately. I spun up a few Azure Hosts in Rancher w/ 128GB SSDs so I’ll just hope the pager doesn’t go off in the middle of the night.

@sonicaghi
Copy link
Member

@lindo-jmm maybe try w/ three nodes at $20/month? (dev edition)

@lindo-jmm
Copy link

Do you think those are production ready, in your opinion?

On Fri, Jan 29, 2016 at 2:25 PM -0800, "Augusto Marietti" <[email protected]mailto:[email protected]> wrote:

@lindo-jmmhttps://github.com/lindo-jmm maybe try w/ three nodes at $20/month?

Reply to this email directly or view it on GitHubhttps://github.com//issues/183#issuecomment-176997348.

@AndrewDryga
Copy link

+1

@SGrondin
Copy link
Contributor

@lindo-jmm There's also the $80/month option here: https://www.instaclustr.com/products/developers/ Feel free to contact [email protected] if you have questions. From what I can tell, the "dev" plans are simply smaller EC2 instances, they're not "lower quality".

@jkwong
Copy link

jkwong commented Feb 14, 2016

+1

@slater-ben
Copy link

Hello all,

I'm Chief Product Officer at Instaclustr so I thought I could provide some info on suitability of our dev offerings for Kong. SGrondin is correct, they are not "lower quality" and are very reliable for many small use cases. The main things to be aware of are:

  • Both dev offerings are based on t2 class AWS instances (t2.small and t2.medium). Both of these are CPU credit limited so if you run them too hard for too long they will run out of credits and drastically slow down. Not a problem when you're getting started but you want to keep an eye on it once your API starts to take off (in particular, I wouldn't recommend the t2.smalls for any consistent load - but OK if you're only getting a few calls an minute).
  • The dev offerings don't come with an SLA (not because they are less reliable, just because they're cheap).
  • The dev offerings don't support SSL connections to Cassandra (due to resource limitations) so we'd recommend using VPC peering for security (you can set this up through our console).

If you do have any questions then you can start a chat with our support staff from support.instaclustr.com or you can email [email protected].

Cheers
Ben

@thibaultcha thibaultcha mentioned this issue Feb 28, 2016
Closed
@thibaultcha thibaultcha mentioned this issue Mar 8, 2016
Closed
@BrianHutchison
Copy link

+1 for DynamoDB support. Please.

@Mgutjahr
Copy link

+1

@DavidTPate
Copy link

I've gone some more thought to this recently. Not sure what the stance of the Mashape team is, but I'm not sure that DynamoDB is a good fit unfortunately. The reason why is that you have a lot more to manage with something like Kong that DynamoDB just doesn't do for you and could cause negative impacts to the usage of Kong (and likely perceived as an issue with Kong when really it's due to the limitations of DynamoDB).

For example, in order to properly use DynamoDB for Kong you're going to need to implement some sort of auto-scaling for your DynamoDB table(s) which will monitor their usage and increase their provisoning prior to it being an issue. This comes into play for Kong because it is going to need to Read/Write to DynamoDB and it will be constrained by how throughput works in DynamoDB. The issue here is that throughput might be exceeded and the request will fail (and Kong will have to throttle itself somehow for requests to DynamoDB). This sounds like a very poor fit to me because of the need to do all of this.

I wonder if instead utilizing a database like PostgreSQL with the JSON type (or other implementation) would be a better fit.

@grantcarver
Copy link
Contributor

grantcarver commented Oct 25, 2016 •
edited
Loading

@DavidTPate I disagree a bit with that. I don't see any difference between having to manage a Cassandra cluster, Postgres servers, or DynamoDB. In every case you are responsible for ensuring the data layer's ability to manage the load you are intending to put onto it. In addition those wanting to use DynamoDB as a datastore for Kong are likely to be aware of any limitations as they are familiar with the platform (and not with Cassandra which is why Postgres support was such a welcome addition!). A note in the docs is sufficient in my opinion - us users are pretty capable.

@nickveenhof
Copy link
Author

Completely agree with grantcarver. We've switched an application from Cassandra to DynamoDB with great success. The fact that we don't have to manage the beast that cassandra is, was such a relieve for us. That said, obviously we still need to tune how we access that data, in the same was as we had to do with Cassandra.

The Throguhput exceeding issue can be easily resolved with a tool like https://github.com/sebdah/dynamic-dynamodb so that's a non-argument. Postgres is indeed a good addition, as this can actually be started as a saas product that removes some of the maintenance burden from small teams. I still think DynamoDB is a great fit for Kong and it would, in my opinion, also increase its adoption by a fair amount given that it greatly increases the ease of installation.

@lbadger
Copy link

lbadger commented Sep 9, 2017

Any progress on supporting DynamoDB?

1 similar comment
@HitDaCa
Copy link

HitDaCa commented Sep 24, 2017

Any progress on supporting DynamoDB?

@hishamhm hishamhm removed the area/DAO label Oct 18, 2017
@vvondra
Copy link

vvondra commented Nov 29, 2017

This is now even much more interesting with Global DynamoDB tables - you can have regionally distributed Kong deployments as with Cassandra

@aelesbao
Copy link

Having DynamoDB support would be amazing!

bungle added a commit that referenced this issue Sep 22, 2020
@bungle
chore(deps) bump luacheck (used for tests) from 0.23 to 0.24
b592a3a 
### Summary

#### Changes

* Caching now uses files in a global directory instead of local
  `.luacheckcache` file. Default cache directory is
  `%LOCALAPPDATA%\Luacheck\Cache` on Windows,
  `~/Library/Caches/Luacheck` on OS X/macOS, and
  `$XDG_CACHE_HOME/luacheck` or `~/.config/luacheck` on other systems.

#### Fixes

* Fixes for using Luacheck with Lua 5.4:
  - Parse 5.4 attributes in `local` declarations (but ignores them for now)
  - Luacheck itself can also run with Lua 5.4
* Fixed `randomize` missing from `busted` set of standard globals (#183).
* Added additional `table` and `thread` definitions for `ngx_lua`.
* Added additional `match` definition for `busted`.

#### Miscellaneous

* Upgraded Windows binary components: Lua 5.3.4 -> 5.3.5,
  LuaFileSystem 1.6.3 -> 1.7.0.
@bungle bungle mentioned this issue Sep 22, 2020
Merged
bungle added a commit that referenced this issue Sep 23, 2020
@bungle
chore(deps) bump luacheck (used for tests) from 0.23 to 0.24 (#6381)
fa75725 
### Summary

#### Changes

* Caching now uses files in a global directory instead of local
  `.luacheckcache` file. Default cache directory is
  `%LOCALAPPDATA%\Luacheck\Cache` on Windows,
  `~/Library/Caches/Luacheck` on OS X/macOS, and
  `$XDG_CACHE_HOME/luacheck` or `~/.config/luacheck` on other systems.

#### Fixes

* Fixes for using Luacheck with Lua 5.4:
  - Parse 5.4 attributes in `local` declarations (but ignores them for now)
  - Luacheck itself can also run with Lua 5.4
* Fixed `randomize` missing from `busted` set of standard globals (#183).
* Added additional `table` and `thread` definitions for `ngx_lua`.
* Added additional `match` definition for `busted`.

#### Miscellaneous

* Upgraded Windows binary components: Lua 5.3.4 -> 5.3.5,
  LuaFileSystem 1.6.3 -> 1.7.0.
@kikito kikito added the task/feature Requests for new features in Kong label Nov 24, 2020
@bungle
Copy link
Member

bungle commented May 19, 2021

I am closing this as we have not pursued it, and I don't see it happening soon. If it happens, we keep you updated.

@bungle bungle closed this as completed May 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
task/feature Requests for new features in Kong
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests