Revamp configuration loading #270

mikz · 2017-02-24T09:33:14Z

closes #230

Breaking Change

Accidentally previous APIcast would try to lazy load configuration even in the production mode.

That means that newly started APIcast would fetch configuration for service 1.
When receiving request for service 1 it would use the downloaded configuration.
But when receiving configuration of non existing service, it would try to lazy lookup the configuration and it would access the remote API.

This is "fixed" in this branch and when APIcast is in boot mode it won't lazy load configuration, ever.

This could break @3scale/qe workflow when publishing production service and immediately trying production APIcast. Because it would not have to be reloaded in up to 5 minutes.

Environment variables

This PR removes:

AUTO_UPDATE_INTERVAL
AUTO_UPDATE_INTERVAL
AUTO_UPDATE_INTERVAL

And adds:

APICAST_CONFIGURATION_CACHE
APICAST_CONFIGURATION_LOADER

APICAST_CONFIGURATION_LOADER

Can be set either to boot or lazy.
Boot will load configuration on boot or exit if there are any errors (but empty configuration will work).
Lazy will try to load configuration on incoming request.

APICAST_CONFIGURATION_CACHE

Controls for how long (in seconds) the configuration will be stored. Can be set to 0 for not caching it at all.
Default is empty value which will cache it forever.

octobot · 2017-02-24T09:37:58Z

	1 Warning
⚠️	Big PR

Proselint found issues

README.md

Line	Message	Severity

doc/parameters.md

Line	Message	Severity

Spell Checker found issues

README.md

Line	Typo
11	This Dockerfile creates a [3scale](http://www
11	self according to your 3scale params.
32	* THREESCALE_PORTAL_ENDPOINT
32	* THREESCALE_PORTAL_ENDPOINT****
32	* THREESCALE_PORTAL_ENDPOINT****
43	* THREESCALE_CONFIG_FILE
43	* THREESCALE_CONFIG_FILE****
43	* THREESCALE_CONFIG_FILE****
55	vice.json)). The optional APICAST_SERVICES environment variab
55	on)). The optional APICAST_SERVICES**** environment variable allows
82	enable this by adjusting the APICAST_CONFIGURATION_CACHE (seconds)
82	this by adjusting the APICAST_CONFIGURATION_CACHE (seconds) to some value
82	ting the APICAST_CONFIGURATION_CACHE (seconds) to some value great
82	s) to some value greater than 60:
88	This variable is set to 0 by default.
102	d on NGINX core and including LuaJIT and Lua modules. Follow the [
104	- LuaRocks - the
106	For Mac OS X the following Homebrew formula can
122	- redis in-memory
122	for the OAuth flow require a redis instance running on `localhos
170	l REGISTRY value, defaults to quay.io):

doc/parameters.md

Line	Typo
3	APIcast v2 has a number of parameters co
3	variables that can modify the behavior of the gateway. The following
5	e that when deploying APIcast v2 with OpenShift, some of thee
12	Default: stderr
17	nfo
17	warn
31	Default: "apicast"
69	Values: a number > 60**
70	Default: 0
72	r. The value should be set to 0 or more than 60. For example,
72	ould be set to 0 or more than 60. For example, if `APICAST_CON
72	ONFIGURATION_CACHE` is set to 120, the gateway will reload the
72	eload the configuration every 2 minutes (120 seconds).
72	onfiguration every 2 minutes (120 seconds).
75	Default: "127.0.0.1"
77	ning Redis instance for OAuth 2.0 flow. `REDIS_HOST` parameter
80	Default: 6379
82	ning Redis instance for OAuth 2.0 flow. `REDIS_PORT` parameter
86	pty, the DNS resolver will be autodiscovered.

Spell Checker found issues

README.md

Line	Typo
11	This Dockerfile creates a [3scale](http://www
11	self according to your 3scale params.
32	* THREESCALE_PORTAL_ENDPOINT
32	* THREESCALE_PORTAL_ENDPOINT****
32	* THREESCALE_PORTAL_ENDPOINT****
43	* THREESCALE_CONFIG_FILE
43	* THREESCALE_CONFIG_FILE****
43	* THREESCALE_CONFIG_FILE****
55	vice.json)). The optional APICAST_SERVICES environment variab
55	on)). The optional APICAST_SERVICES**** environment variable allows
82	enable this by adjusting the APICAST_CONFIGURATION_CACHE (seconds)
82	this by adjusting the APICAST_CONFIGURATION_CACHE (seconds) to some value
82	ting the APICAST_CONFIGURATION_CACHE (seconds) to some value great
82	s) to some value greater than 60:
88	This variable is set to 0 by default.
102	d on NGINX core and including LuaJIT and Lua modules. Follow the [
104	- LuaRocks - the
106	For Mac OS X the following Homebrew formula can
122	- redis in-memory
122	for the OAuth flow require a redis instance running on `localhos
170	l REGISTRY value, defaults to quay.io):

doc/parameters.md

Line	Typo
3	APIcast v2 has a number of parameters co
3	variables that can modify the behavior of the gateway. The following
5	e that when deploying APIcast v2 with OpenShift, some of thee
12	Default: stderr
17	nfo
17	warn
31	Default: "apicast"
69	Values: a number > 60**
70	Default: 0
72	r. The value should be set to 0 or more than 60. For example,
72	ould be set to 0 or more than 60. For example, if `APICAST_CON
72	ONFIGURATION_CACHE` is set to 120, the gateway will reload the
72	eload the configuration every 2 minutes (120 seconds).
72	onfiguration every 2 minutes (120 seconds).
75	Default: "127.0.0.1"
77	ning Redis instance for OAuth 2.0 flow. `REDIS_HOST` parameter
80	Default: 6379
82	ning Redis instance for OAuth 2.0 flow. `REDIS_PORT` parameter
86	pty, the DNS resolver will be autodiscovered.

Generated by 🚫 Danger

mayorova · 2017-02-24T09:49:33Z

apicast/src/configuration_store.lua

@@ -16,9 +15,18 @@ local mt = { __index = _M }

 function _M.new()
  return setmetatable({
+    -- services hashed by id, example: {
+    --   ["16"] = service1
+    -- }


This kind of comments are very helpful :) 👍

Yep. Wanted to do it just for sake of having it written when I was changing the data structure :)
Will try to do that more in complicated cases.

jcechace · 2017-03-02T13:29:37Z

Generally speaking we should be fine with the configuration not being immediate as long as there is a mechanism to trigger the immediate reload manually -- waiting up to 5 minutes for the configuration to be reloaded doesn't really make sense in tests.

I will let @mijaros confirm this since I'm still a bit out of picture after rejoining the team. though

mikz · 2017-03-02T14:03:07Z

Ok. We could:

trigger reload by calling POST /boot on the management api (would not work with multiple Pods though)
set lower refresh rate
trigger deploy on OpenShift

print which configuration loader is being used

with APICAST_CONFIGURATION_CACHE

by APICAST_CONFIGURATION_LOADER

rather return stale configuration than nothing if services takes longer than expected to refresh it is better to return stale config than an error

into own module for reuse

wait for up to 15 seconds to load configuration for the same host

octobot assigned mikz Feb 24, 2017

octobot added the T-obux label Feb 24, 2017

mikz added B-current in progress and removed B-current labels Feb 24, 2017

mayorova reviewed Feb 24, 2017

View reviewed changes

mikz force-pushed the revamp-configuration branch 3 times, most recently from e24d78e to 6bdd136 Compare February 24, 2017 11:45

mikz removed the in progress label Feb 28, 2017

mikz force-pushed the revamp-configuration branch from e6db6c5 to 1cbc85b Compare March 2, 2017 12:43

mikz added 11 commits March 2, 2017 18:44

revamp configuration loading

a8b5103

[nginx] split nginx configuration

2a9c803

[config] more debugging info

dde435c

print which configuration loader is being used

[t] test boot crashing apicast

b829041

[t] test configuration loading

b82e4cd

[config] store configuration with ttl

214f273

[travis] limit what artifacts to upload

bfd1f3f

[t] fix oauth tests when many resolvers are defined

fc4a305

[env] replace AUTO_UPDATE_INTERVAL

b55a4b4

with APICAST_CONFIGURATION_CACHE

[env] replace APICAST_RELOAD_CONFIG

3012f35

[env] replace APICAST_MISSING_CONFIGURATION

01877bc

by APICAST_CONFIGURATION_LOADER

mikz force-pushed the revamp-configuration branch from 1cbc85b to 01877bc Compare March 2, 2017 18:09

mikz added 4 commits March 2, 2017 19:18

[readme] fix typo

95a6ab4

[changelog] update changelog with env changes

f7447be

[configuration] return stale configuration

50e094a

rather return stale configuration than nothing if services takes longer than expected to refresh it is better to return stale config than an error

[resty.synchronization] extract de-duplication code

b3a662a

into own module for reuse

mikz added 2 commits March 3, 2017 08:25

[resolver] use new resty.synchronization

ea80183

[configuration] de-duplicate configuration loading

af54f33

wait for up to 15 seconds to load configuration for the same host

mikz added this to the On-premise ER4 release milestone Mar 3, 2017

mikz added the B-current label Mar 3, 2017

mikz merged commit cf840cc into master Mar 3, 2017

mikz deleted the revamp-configuration branch March 3, 2017 07:50

mikz removed B-current labels Mar 3, 2017

mikz mentioned this pull request Mar 7, 2017

Store configuration with TTL #229

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revamp configuration loading #270

Revamp configuration loading #270

mikz commented Feb 24, 2017 •

edited

Loading

octobot commented Feb 24, 2017 •

edited

Loading

mayorova Feb 24, 2017

mikz Feb 24, 2017

jcechace commented Mar 2, 2017

mikz commented Mar 2, 2017 •

edited

Loading

Revamp configuration loading #270

Revamp configuration loading #270

Conversation

mikz commented Feb 24, 2017 • edited Loading

Breaking Change

Environment variables

APICAST_CONFIGURATION_LOADER

APICAST_CONFIGURATION_CACHE

octobot commented Feb 24, 2017 • edited Loading

Proselint found issues

Spell Checker found issues

Spell Checker found issues

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcechace commented Mar 2, 2017

mikz commented Mar 2, 2017 • edited Loading

mikz commented Feb 24, 2017 •

edited

Loading

octobot commented Feb 24, 2017 •

edited

Loading

mikz commented Mar 2, 2017 •

edited

Loading