Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clustering, cache invalidation and event bus #729

Closed
wants to merge 10 commits into from
Closed

Clustering, cache invalidation and event bus #729

wants to merge 10 commits into from

Conversation

subnetmarco
Copy link
Member

Note: Running this PR requires having serf installed into /usr/local/bin/serf.

This PR implements an invalidation strategy (#15), and for doing so it also implements the following features/improvements:

  • Disables the time based expiration cache.
  • Exposes a /cache/ endpoint for retrieving elements stored in the in-memory cache of a node.
  • Exposes a /cluster/ endpoint used to add/remove/list members of the cluster, and also used internally for data propagation.
  • CLI refactoring to better handle the services that are started by Kong (and error scenarios).
  • CLI support for kong cluster <command>.
  • CLI supports an additional kong status command that checks the status of all the Kong services.
  • serf integration for clustering (Caching/Invalidation strategy for APIs, Consumers and Plugins #15).
  • Even bus with both local and cluster wide propagation (partially Hook / Event System #690).
  • Fixes a few documented and undocumented bugs, including Updating a Basic Authentication credential invalidates the password #726.
  • The OAuth 2.0 plugin now also exposes /oauth2_tokens/ with the possibility to retrieve, update or delete OAuth 2.0 access tokens.
  • Adds marshall_event function as an option to the schemas used to define which properties to send to the cluster. Required to improve the performance (smaller data means greater network performance) and because there is a hard limit of 512 bytes to the payload that can be sent across the cluster (because the data needs to fit inside a UDP packet).
  • Add a new nodes table for the auto-join feature.
  • Various fixes and tests (even for previous code).

TODO

  • Lots of documentation needs to be updated on the website.
  • I am not happy by the output of the CLI, I will need to do a minor enhancement for this (also to disable it from the tests).
  • Cluster auto-join feature, by storing the active machines in a Cassandra table and listen to Serf events to add/remove the nodes from that table (as long as the nodes point to the same datastore they will auto-join themselves). Maybe this feature could be enabled/disabled by an autojoin: true property in the configuration.

@subnetmarco subnetmarco added area/admin api pr/wip A work in progress PR opened to receive feedback labels Nov 19, 2015
@subnetmarco subnetmarco self-assigned this Nov 19, 2015
@subnetmarco subnetmarco added this to the 0.6.0 milestone Nov 19, 2015
subnetmarco
subnetmarco reviewed Nov 19, 2015
View reviewed changes
kong/cli/cmds/config.lua
--["cluster"] = {
-- ["bind"] = "0.0.0.0:7947",
-- ["rpc-addr"] = "127.0.0.1:7374"
--},
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is currently a problem because of the YAML library we use. Related to lubyk/yaml#2.

@ahmadnassri
Copy link
Contributor

Cluster auto-join feature, by storing the active machines in a Cassandra table and listen to Serf events to add/remove the nodes from that table (as long as the nodes point to the same datastore they will auto-join themselves). Maybe this feature could be enabled/disabled by an autojoin: true property in the configuration.

I like this as a the default approach, especially useful to ensure nodes of the same cluster AND data store only can join together (to avoid conflicts when multiple clusters of different data stores are available)

@subnetmarco
Copy link
Member Author

Auto-join would be also useful because the deployment method would be the same as Kong < 0.6.0 - so nothing new to learn.

@nijikokun nijikokun mentioned this pull request Nov 21, 2015
Closed
@subnetmarco
Copy link
Member Author

There is a problem with the autojoin feature, which is that we don't know what's the public IP/address of the machine. The table that contains all the nodes added, needs to also contain the address that the other nodes will use when joining the cluster, and we currently don't know that.

@ahmadnassri
Copy link
Contributor

there are ways to discover the machine's own IP ... but if course not accurately and not always successfully.

we can start with that? and if not successful, requires manual configuration?

@subnetmarco
Copy link
Member Author

auto-join has been implemented.

@subnetmarco subnetmarco mentioned this pull request Nov 25, 2015
Closed
@subnetmarco subnetmarco force-pushed the feat/invalidations branch 2 times, most recently from 898c686 to 56609df Compare December 1, 2015 21:55
This was referenced Dec 2, 2015
Closed
Closed
@subnetmarco subnetmarco force-pushed the feat/invalidations branch 7 times, most recently from f6b99a8 to 13a53f5 Compare December 27, 2015 09:22
@thibaultcha thibaultcha changed the title Invalidations Clustering and cache invalidation Jan 6, 2016
@thibaultcha thibaultcha changed the title Clustering and cache invalidation Clustering, cache invalidation and event bus Jan 6, 2016
thibaultcha added a commit that referenced this pull request Jan 7, 2016
@thibaultcha
docs: add changes from #729
68161d1
@thibaultcha @subnetmarco
docs(readme) add more development instructions
6c31ec5 
The different test suites and the way test/development running Kong need
a different configuration file can be confusing at first, so this adds
some more insight as to what is needed and why.

Of course, this is the current system but nothing prevents it from being
improved in the future.
thibaultcha and others added 8 commits January 20, 2016 20:30
@thibaultcha @subnetmarco
docs(readme) update getkong links to https
32b9945
@thibaultcha @subnetmarco
fix(api) allow to delete APIs and Consumers by name
5feacd1 
Cassandra from 3 to 2 during an update and a delete, it removed the
possibility of updating and deleting APIs and Consumers by their name,
because that change made it so the base_dao's underlying `delete()` and
`update()` were directly used instead of first querying the entity.
Querying the entity first allowed to retrieve its PRIMARY KEY fields,
and hence `delete()` and `update()` never complained. But by removing
this initial retrieving, we removed this feature, and **there were not
tests** for it!

- This adds test for PATCH/DELETE APIs and Consumers by name/username
- Adds an argument to the base_dao `update()` and `delete()` to select
  by any field rather than only the PRIMARY KEY fields.
@subnetmarco
refactor(req-transformer) append, replace of header/querystring
20d4b72 
Fix for issue #393 for response-transformer motivated fix for this plugin too. New config append/replace added to support adding new value to existing header/querystring and
replcaing  existing header/querystring with new value.
@subnetmarco
fix(resp-tranformer) handle already existing headers
9f1ffb3 
Now add will only add header/json if it does not exist, config `append` added to support adding new value to existing header/json. If header does not exist, a new one will be added. config `replace added to replace the value of existing header/json with new value
@subnetmarco
Adding cache API - CLI Rewrite - Serf integration/Clustering - Hooks/…
4ea5b2b 
…Events - Invalidations
@thibaultcha @subnetmarco
docs(readme) add more development instructions
738cd8e 
The different test suites and the way test/development running Kong need
a different configuration file can be confusing at first, so this adds
some more insight as to what is needed and why.

Of course, this is the current system but nothing prevents it from being
improved in the future.
@thibaultcha @subnetmarco
docs(readme) update getkong links to https
cf4ea3a
@thibaultcha @subnetmarco
fix(api) allow to delete APIs and Consumers by name
e7ce5d6
@subnetmarco subnetmarco force-pushed the feat/invalidations branch 5 times, most recently from feb6e0e to 5ab02aa Compare January 22, 2016 02:37
@subnetmarco
Copy link
Member Author

Merged with 4720b7b

@subnetmarco subnetmarco deleted the feat/invalidations branch March 9, 2016 00:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@subnetmarco subnetmarco

Labels
pr/changes requested Changes were requested to this PR by a maintainer. Please address them and ping back once done.
Projects
None yet
Milestone
Jan 2016
Development

Successfully merging this pull request may close these issues.
3 participants
@subnetmarco @ahmadnassri @thibaultcha