How to distribute (release process) Kong? #2

thibaultcha · 2015-01-30T22:34:15Z

Find a way to ship Kong and an easy way to install/run Cassandra for test purposes.

Ideas to experiment:

See if Kong can be bundled in Openresty
Deb/RPM packages
Chef cookbook?
Bash script for installation
Use the Makefile
Docker/Vagrant
Ship a Cassandra test cluster in Docker/Vagrant

Things required to run Kong:

Lua
Luarocks
Openresty
Cassandra
Our luarocks dependencies to install
A Cassandra cluster

An idea could be that on the website, the user lands on a Download page asking him what is his distribution, he selects, we display the way to install Kong on his distribution (Deb vs RPM for example) as a list of commands he can copy/paste.

ahmadnassri · 2015-01-30T22:39:35Z

easiest way for Debian packaging is to setup a ppa package through
launchpad.net

https://github.com/LMMS/lmms/wiki/Packaging-Ubuntu-PPA

nijikokun · 2015-02-06T23:14:20Z

Package / Chef / Puppet / Docker / Vagrant (Not all at once, but eventually should support these)

subnetmarco · 2015-02-20T05:28:25Z

Take a look at the various Dockerfiles to see exactly what dependencies Kong needs.

montanaflynn · 2015-02-24T22:10:11Z

I'll just throw these in for consideration:

ahmadnassri · 2015-02-25T00:49:53Z

👍 fpm, throw that in Travis, package it and store it back in github releases.

ahmadnassri · 2015-02-25T00:51:26Z

also, github releases in itself is a distribution method, compile versions for different environments and store in github releases ...

https://github.com/Mashape/kong/releases/tag/0.0.1-beta

all can be done in Travis, with proper tagging and setup.

thibaultcha · 2015-02-25T00:54:54Z

We still need an environment-specific way to install on a server. Aka copy-paste of commands in an ssh session, using different package managers for dependencies.

Putting stuff in GH releases doesn't really help, we are not shipping a binary, but a Luarocks module, so... It's install from source or Luarocks.

montanaflynn · 2015-02-25T21:05:32Z

When Niji suggested an installation shell script yesterday I remembered that's how Boundary supports a ton of platforms with the same quick installation snippet. Here's their script: https://github.com/boundary/boundary_scripts/blob/master/setup_meter.sh

thibaultcha · 2015-02-25T21:06:46Z

The problem with installation scripts is that you have to read it to know what it's going to do on your machine. When giving the user a list of commands feels safer and less time consuming (if you care about your setup).

We could still provide the script for people who don't really care though. It is after all, suggested in the list above ^

nijikokun · 2015-02-25T21:38:03Z

only if you're extremely paranoid, 100% sure you don't read every brew ruby file, or every installation shell script you use.

thibaultcha · 2015-02-25T21:40:45Z

It's not about being paranoid, but knowing how and where it installs stuff. I know where my package managers install stuff. It's more about setup. And I understand better what it's doing, especially if we just have a .deb etc.

We could replace an install.sh with a simple package manager install(from apt-get and others)

nijikokun · 2015-02-25T21:42:43Z

Its actually fine, because it's in plain text so you can read what its doing 👍

thibaultcha · 2015-02-27T01:39:46Z

I think we can close this issue for now. Install process has been discussed and is mocked.

For release

Platform-specific install method from source
Docker

In the future

Install script (preferred way)
Package managers (per platform)
Platform-specific install method from source
Docker

- Cluster is only spawned when inside if ngx_lua (when starting Kong). - Ensure LuaSocket is used by the driver by changing the return value of `ngx_stub.get_phase()` to `init`. - Ensure lua-cassandra does not output logs when in the `init` phase (which is the case because of change #2) and inside of the specs. Which is why `ngx.stub` is a new property.

### Summary Before this commit: ``` patching file LuaJIT-2.1-20190507/src/lj_tab.c patching file LuaJIT-2.1-20190507/src/lj_asm_arm.h patching file LuaJIT-2.1-20190507/src/lj_api.c patching file LuaJIT-2.1-20190507/src/lj_arch.h patching file LuaJIT-2.1-20190507/src/lj_cconv.c patching file LuaJIT-2.1-20190507/src/lj_obj.h patching file LuaJIT-2.1-20190507/src/lj_state.c patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/resty/core.lua patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/ngx/balancer.lua patching file lua-resty-websocket-0.07/lib/resty/websocket/client.lua patching file nginx-1.15.8/src/http/ngx_http_upstream.c Hunk #2 succeeded at 1691 (offset -3 lines). Hunk #4 succeeded at 1768 (offset -7 lines). patching file nginx-1.15.8/src/http/ngx_http_special_response.c patching file nginx-1.15.8/src/stream/ngx_stream_proxy_module.c Hunk #2 succeeded at 802 (offset 20 lines). patching file ngx_lua-0.10.15/src/ngx_http_lua_socket_tcp.c Hunk #5 succeeded at 1556 (offset 48 lines). Hunk #6 succeeded at 1626 (offset 48 lines). Hunk #7 succeeded at 1726 (offset 48 lines). Hunk #8 succeeded at 1749 (offset 48 lines). Hunk #9 succeeded at 1760 (offset 48 lines). Hunk #10 succeeded at 1775 (offset 48 lines). Hunk #11 succeeded at 1816 (offset 48 lines). Hunk #12 succeeded at 1827 (offset 48 lines). Hunk #13 succeeded at 1849 (offset 48 lines). Hunk #14 succeeded at 1868 (offset 48 lines). Hunk #15 succeeded at 1878 (offset 48 lines). Hunk #16 succeeded at 2008 (offset 48 lines). Hunk #17 succeeded at 6101 (offset 62 lines). patching file ngx_lua-0.10.15/src/ngx_http_lua_socket_tcp.h patching file ngx_lua-0.10.15/src/ngx_http_lua_socket_tcp.c Hunk #1 succeeded at 1647 (offset 48 lines). patching file ngx_lua-0.10.15/src/ngx_http_lua_socket_tcp.c Hunk #3 succeeded at 1579 (offset 48 lines). Hunk #4 succeeded at 1644 (offset 48 lines). Hunk #5 succeeded at 1668 (offset 48 lines). patching file ngx_lua-0.10.15/src/ngx_http_lua_socket_tcp.c Hunk #1 succeeded at 1756 (offset 48 lines). patching file ngx_lua-0.10.15/src/ngx_http_lua_balancer.c patching file ngx_lua-0.10.15/src/ngx_http_lua_common.h patching file ngx_lua-0.10.15/src/ngx_http_lua_module.c patching file ngx_lua-0.10.15/src/ngx_http_lua_string.c patching file ngx_stream_lua-0.0.7/src/ngx_stream_lua_control.c patching file ngx_stream_lua-0.0.7/src/ngx_stream_lua_variable.c patching file ngx_stream_lua-0.0.7/src/ngx_stream_lua_common.h patching file ngx_stream_lua-0.0.7/src/ngx_stream_lua_util.c patching file ngx_stream_lua-0.0.7/src/api/ngx_stream_lua_api.h ``` vs. (after this commit) ``` patching file LuaJIT-2.1-20190507/src/lj_tab.c patching file LuaJIT-2.1-20190507/src/lj_asm_arm.h patching file LuaJIT-2.1-20190507/src/lj_api.c patching file LuaJIT-2.1-20190507/src/lj_arch.h patching file LuaJIT-2.1-20190507/src/lj_cconv.c patching file LuaJIT-2.1-20190507/src/lj_obj.h patching file LuaJIT-2.1-20190507/src/lj_state.c patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/resty/core.lua patching file lua-resty-core-0.1.17/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.17/lib/ngx/balancer.lua patching file lua-resty-websocket-0.07/lib/resty/websocket/client.lua patching file nginx-1.15.8/src/http/ngx_http_upstream.c patching file nginx-1.15.8/src/http/ngx_http_special_response.c patching file nginx-1.15.8/src/stream/ngx_stream_proxy_module.c patching file ngx_lua-0.10.15/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.15/src/ngx_http_lua_socket_tcp.h patching file ngx_lua-0.10.15/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.15/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.15/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.15/src/ngx_http_lua_balancer.c patching file ngx_lua-0.10.15/src/ngx_http_lua_common.h patching file ngx_lua-0.10.15/src/ngx_http_lua_module.c patching file ngx_lua-0.10.15/src/ngx_http_lua_string.c patching file ngx_stream_lua-0.0.7/src/ngx_stream_lua_control.c patching file ngx_stream_lua-0.0.7/src/ngx_stream_lua_variable.c patching file ngx_stream_lua-0.0.7/src/ngx_stream_lua_common.h patching file ngx_stream_lua-0.0.7/src/ngx_stream_lua_util.c patching file ngx_stream_lua-0.0.7/src/api/ngx_stream_lua_api.h ```

) ### Summary ``` patching file LuaJIT-2.1-20200102/src/lj_api.c patching file LuaJIT-2.1-20200102/src/lj_arch.h Hunk #1 succeeded at 241 (offset 4 lines). patching file LuaJIT-2.1-20200102/src/lj_cconv.c patching file LuaJIT-2.1-20200102/src/lj_obj.h Hunk #1 succeeded at 831 (offset 1 line). patching file LuaJIT-2.1-20200102/src/lj_state.c patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/resty/core.lua Hunk #1 succeeded at 19 with fuzz 1 (offset -2 lines). patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/ngx/balancer.lua patching file lua-resty-core-0.1.19/lib/ngx/balancer.lua Hunk #1 succeeded at 45 (offset 7 lines). Hunk #2 succeeded at 348 (offset 137 lines). patching file lua-resty-core-0.1.19/lib/resty/core.lua patching file lua-resty-core-0.1.19/lib/resty/core/request.lua patching file lua-resty-core-0.1.19/lib/resty/core/utils.lua patching file lua-resty-websocket-0.07/lib/resty/websocket/client.lua patching file nginx-1.17.8/src/http/ngx_http_upstream.c Hunk #2 succeeded at 1688 (offset -3 lines). Hunk #3 succeeded at 1719 (offset -3 lines). Hunk #4 succeeded at 1765 (offset -3 lines). patching file nginx-1.17.8/src/http/ngx_http_special_response.c patching file nginx-1.17.8/src/stream/ngx_stream_proxy_module.c patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.h patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.17/src/ngx_http_lua_balancer.c patching file ngx_lua-0.10.17/src/ngx_http_lua_common.h patching file ngx_lua-0.10.17/src/ngx_http_lua_module.c patching file ngx_lua-0.10.17/src/ngx_http_lua_balancer.c patching file ngx_lua-0.10.17/src/ngx_http_lua_balancer.c patching file ngx_lua-0.10.17/src/ngx_http_lua_common.h patching file ngx_lua-0.10.17/src/ngx_http_lua_balancer.c patching file ngx_stream_lua-0.0.8/src/ngx_stream_lua_util.c Hunk #1 succeeded at 1965 with fuzz 2 (offset 10 lines). patching file ngx_stream_lua-0.0.8/src/api/ngx_stream_lua_api.h ``` vs. ``` patching file LuaJIT-2.1-20200102/src/lj_api.c patching file LuaJIT-2.1-20200102/src/lj_arch.h patching file LuaJIT-2.1-20200102/src/lj_cconv.c patching file LuaJIT-2.1-20200102/src/lj_obj.h patching file LuaJIT-2.1-20200102/src/lj_state.c patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/resty/core.lua patching file lua-resty-core-0.1.19/lib/resty/core/socket_tcp.lua patching file lua-resty-core-0.1.19/lib/ngx/balancer.lua patching file lua-resty-core-0.1.19/lib/ngx/balancer.lua patching file lua-resty-core-0.1.19/lib/resty/core.lua patching file lua-resty-core-0.1.19/lib/resty/core/request.lua patching file lua-resty-core-0.1.19/lib/resty/core/utils.lua patching file lua-resty-websocket-0.07/lib/resty/websocket/client.lua patching file nginx-1.17.8/src/http/ngx_http_upstream.c patching file nginx-1.17.8/src/http/ngx_http_special_response.c patching file nginx-1.17.8/src/stream/ngx_stream_proxy_module.c patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.h patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.17/src/ngx_http_lua_socket_tcp.c patching file ngx_lua-0.10.17/src/ngx_http_lua_balancer.c patching file ngx_lua-0.10.17/src/ngx_http_lua_common.h patching file ngx_lua-0.10.17/src/ngx_http_lua_module.c patching file ngx_lua-0.10.17/src/ngx_http_lua_balancer.c patching file ngx_lua-0.10.17/src/ngx_http_lua_balancer.c patching file ngx_lua-0.10.17/src/ngx_http_lua_common.h patching file ngx_lua-0.10.17/src/ngx_http_lua_balancer.c patching file ngx_stream_lua-0.0.8/src/ngx_stream_lua_util.c patching file ngx_stream_lua-0.0.8/src/api/ngx_stream_lua_api.h ```

) This PR fixes a bug when worker consuming dynamic log level setting event and using a wrong reference for notice logging ``` 2023/05/18 14:29:06 [error] 23010#0: *3 [lua] callback.lua:98: do_handlerlist(): worker-events: event callback failed; source=debug, event=log_level, wid=0 error='./kong/runloop/handler.lua:937: bad argument #2 to 'log' (expected table to have __tostring metamethod) stack traceback: [C]: in function 'log' ./kong/runloop/handler.lua:937: in function <./kong/runloop/handler.lua:925> [C]: in function 'xpcall' ...uild/kong-dev/openresty/lualib/resty/events/callback.lua:83: in function 'do_handlerlist' ...uild/kong-dev/openresty/lualib/resty/events/callback.lua:130: in function 'do_event' .../build/kong-dev/openresty/lualib/resty/events/worker.lua:68: in function 'do_event' .../build/kong-dev/openresty/lualib/resty/events/worker.lua:239: in function <.../build/kong-dev/openresty/lualib/resty/events/worker.lua:221>', data={"timeout":60,"log_level":8}, context: ngx.timer ```

### Summary #### bug fixes - **\*:** fix typos and add error check for new_of/dup_of ([#2](fffonion/lua-resty-openssl#2)) [aa6ad47](fffonion/lua-resty-openssl@aa6ad47) #### features - **tests:** add performance test ([#112](fffonion/lua-resty-openssl#112)) [100b4e4](fffonion/lua-resty-openssl@100b4e4) - **x509.store:** add store:check_revocation and add flag to skip check CRL for store:add ([#1](fffonion/lua-resty-openssl#1)) [1a5a4c8](fffonion/lua-resty-openssl@1a5a4c8) Signed-off-by: Aapo Talvensaari <[email protected]>

### Summary Without this I get: ``` Hunk #1 succeeded at 121 (offset 8 lines). Hunk #2 succeeded at 143 (offset 8 lines). ``` When applying the `ldp_stp_fusion` patch. Signed-off-by: Aapo Talvensaari <[email protected]>

### Summary Before: ``` patching file bundle/LuaJIT-2.1-20230410/src/lj_asm_arm64.h Hunk #1 succeeded at 1133 (offset 26 lines). Hunk #2 succeeded at 1142 (offset 26 lines). ``` After: ``` patching file bundle/LuaJIT-2.1-20230410/src/lj_asm_arm64.h ``` Signed-off-by: Aapo Talvensaari <[email protected]>

### Summary Before: ``` patching file bundle/LuaJIT-2.1-20230410/src/lj_asm_arm64.h Hunk #1 succeeded at 1133 (offset 26 lines). Hunk #2 succeeded at 1142 (offset 26 lines). ``` After: ``` patching file bundle/LuaJIT-2.1-20230410/src/lj_asm_arm64.h ``` Signed-off-by: Aapo Talvensaari <[email protected]> (cherry picked from commit dda623d)

Without this I get: ``` Hunk #1 succeeded at 121 (offset 8 lines). Hunk #2 succeeded at 143 (offset 8 lines). ``` When applying the `ldp_stp_fusion` patch. Signed-off-by: Aapo Talvensaari <[email protected]>

In 4059a31 (#13843) we added a plugin-like interface for Wasm filters. We now have 3 sources for plugins: Lua, External, and Wasm Filters. When a plugin is enabled or configured, the plugin system follows a resolution order for looking up the plugin handler and schema: 1. Lua => `require kong.plugins.<name>.{handler,schema}` 2. External => `kong.runloop.plugin_servers.load_{handler,schema}(<name>)` 3. Wasm Filters => `kong.runloop.wasm.plugins.load_{handler,schema}(<name>)` When a user configures Kong with a "bad" entry in `pluginserver_names` (read: a plugin server that is not actually installed), step #2 of the plugin resolution process throws an exception, because the external plugin subsystem attempts to query a plugin server that does not exist. Importantly, *this exception is not encountered when the user has only configured/enabled Lua plugins,* because we never reach beyond step #1 of the plugin resolution process. A side effect of adding the Wasm filter plugin interface is that discovered Wasm filters are added to the global plugins table (`kong.configuration.loaded_plugins`) when Wasm is enabled. This means that, if Wasm is enabled, and any Wasm filters are installed, we _always_ step through step #2 of the plugin resolution process, triggering an exception if the user has any badly-configured plugin server. A future change will likely render this scenario unreachable by performing deeper validation of `pluginserver_names` at startup. For now, a simple fix is just to change the resolution order such that Wasm filters are loaded _before_ we query the external plugin subsystem: 1. Lua 2. Wasm Filters 3. External

thibaultcha added this to the v0.1 milestone Jan 30, 2015

thibaultcha modified the milestones: v1.0.0 (release), v0.1 (MVP) Jan 30, 2015

thibaultcha added the TODO label Jan 30, 2015

mashapedeployment changed the title ~~Distribute Kong~~ How to Distribute Kong? Jan 30, 2015

thibaultcha changed the title ~~How to Distribute Kong?~~ How to distribute Kong? Jan 30, 2015

thibaultcha changed the title ~~How to distribute Kong?~~ How to distribute (release process) Kong? Feb 6, 2015

thibaultcha closed this as completed Feb 27, 2015

montanaflynn mentioned this issue Apr 28, 2015

Document/benchmark #172

Merged

thibaultcha mentioned this issue Dec 18, 2015

fix(dao) do not spawn cluster when outside of ngx_lua #812

Merged

JnMik mentioned this issue Jul 11, 2016

allow for combining request_path and request_host for matching API rules #845

Closed

chris-hailstorm mentioned this issue Sep 14, 2016

Assertion error in 0.9.1 / AWS Marketplace edition #1632

Closed

JnMik mentioned this issue Dec 15, 2016

[request] Additional Metadata stored w/ the Consumer #550

Closed

robert2020 mentioned this issue Dec 26, 2016

Mac os 10.12.1 build kong test error #1927

Closed

JnMik mentioned this issue Feb 15, 2017

Metadata insertion plugin #1962

Closed

thibaultcha mentioned this issue May 2, 2017

feat(cors) match configured origins as a regular expression #2482

Merged

arunpk-zageno mentioned this issue Apr 12, 2023

KONG not handles null value in the attributes of JSON payload. #10658

Closed

1 task

chobits mentioned this issue Feb 5, 2024

perf(proxy): use more reasonable keepalive default value for Nginx tuning #12223

Merged

3 tasks

chronolaw added a commit that referenced this issue Sep 29, 2024

clean sync/rpc.lua #2

313c342

chronolaw added a commit that referenced this issue Oct 12, 2024

clean sync/rpc.lua #2

2c2d3ae

dndx pushed a commit that referenced this issue Oct 16, 2024

clean sync/rpc.lua #2

b8f3bd4

flrgh mentioned this issue Nov 26, 2024

fix(plugins): load wasm filter plugins before external plugins #13931

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to distribute (release process) Kong? #2

How to distribute (release process) Kong? #2

thibaultcha commented Jan 30, 2015

ahmadnassri commented Jan 30, 2015

nijikokun commented Feb 6, 2015

subnetmarco commented Feb 20, 2015

montanaflynn commented Feb 24, 2015

ahmadnassri commented Feb 25, 2015

ahmadnassri commented Feb 25, 2015

thibaultcha commented Feb 25, 2015

montanaflynn commented Feb 25, 2015

thibaultcha commented Feb 25, 2015

nijikokun commented Feb 25, 2015

thibaultcha commented Feb 25, 2015

nijikokun commented Feb 25, 2015

thibaultcha commented Feb 27, 2015

How to distribute (release process) Kong? #2

How to distribute (release process) Kong? #2

Comments

thibaultcha commented Jan 30, 2015

ahmadnassri commented Jan 30, 2015

nijikokun commented Feb 6, 2015

subnetmarco commented Feb 20, 2015

montanaflynn commented Feb 24, 2015

ahmadnassri commented Feb 25, 2015

ahmadnassri commented Feb 25, 2015

thibaultcha commented Feb 25, 2015

montanaflynn commented Feb 25, 2015

thibaultcha commented Feb 25, 2015

nijikokun commented Feb 25, 2015

thibaultcha commented Feb 25, 2015

nijikokun commented Feb 25, 2015

thibaultcha commented Feb 27, 2015

For release

In the future