Cannot use external ElasticSearch db with Docker

[martinzuern]

I tried to connect api-umbrella v0.14 with a managed elasticsearch db on AWS (version 2.3).

Custom Configuration:

services:
  - general_db
  - router
  - web
elasticsearch:
  hosts:
    - "https://search-XXXXXXXXXXXXX.us-east-1.es.amazonaws.com:443"

When trying to access analytics from the admin dashboard, web-puma always throws an error 500, as per log:

2017-03-23T14:34:30.74831 Started GET "/admin/stats/search.json?start_at=2017-02-22&end_at=2017-03-23&interval=day&query=%7B%22condition%22%3A%22AND%22%2C%22rules%22%3A%5B%7B%22field%22%3A%22gatekeeper_denied_code%22%2C%22id%22%3A%22gatekeeper_denied_code%22%2C%22input%22%3A%22select%22%2C%22operator%22%3A%22is_null%22%2C%22type%22%3A%22string%22%2C%22value%22%3Anull%7D%5D%7D&search=&beta_analytics=false" for 127.0.0.1 at 2017-03-23 14:34:30 +0000
2017-03-23T14:34:30.75053 Processing by Admin::StatsController#search as JSON
2017-03-23T14:34:30.75059   Parameters: {"start_at"=>"2017-02-22", "end_at"=>"2017-03-23", "interval"=>"day", "query"=>"{\"condition\":\"AND\",\"rules\":[{\"field\":\"gatekeeper_denied_code\",\"id\":\"gatekeeper_denied_code\",\"input\":\"select\",\"operator\":\"is_null\",\"type\":\"string\",\"value\":null}]}", "search"=>"", "beta_analytics"=>"false"}
2017-03-23T14:34:31.58045 GET https://search-XXXXXXXX.us-east-1.es.amazonaws.com:443/api-umbrella-logs-2017-02,api-umbrella-logs-2017-03/_search?allow_no_indices=true&ignore_unavailable=missing&search_type=count&size=0&timeout=90s [status:200, request:0.822s, query:0.001s]
2017-03-23T14:34:31.61735   Rendered admin/stats/search.rabl (19.9ms)
2017-03-23T14:34:31.61770 Completed 500 Internal Server Error in 867ms
2017-03-23T14:34:31.61947 
2017-03-23T14:34:31.61951 ActionView::Template::Error (undefined method `[]' for nil:NilClass):
2017-03-23T14:34:31.61951     1: object false
2017-03-23T14:34:31.61952     2: 
2017-03-23T14:34:31.61952     3: extends "admin/stats/_hits_over_time"
2017-03-23T14:34:31.61952     4: 
2017-03-23T14:34:31.61952     5: node :stats do
2017-03-23T14:34:31.61952     6:   {
2017-03-23T14:34:31.61953   app/models/log_result/base.rb:22:in `hits_over_time'
2017-03-23T14:34:31.61953   app/views/admin/stats/_hits_over_time.rabl:4:in `block in eval_source'
2017-03-23T14:34:31.61953   app/views/admin/stats/search.rabl:3:in `_app_views_admin_stats_search_rabl__1804047830935439177_47248133041760'
2017-03-23T14:34:31.61953   app/controllers/application_controller.rb:103:in `set_time_zone'
2017-03-23T14:34:31.61954   app/controllers/application_controller.rb:160:in `set_userstamp'
2017-03-23T14:34:31.61954   app/controllers/application_controller.rb:88:in `block in use_locale'
2017-03-23T14:34:31.61954   app/controllers/application_controller.rb:87:in `use_locale'

nginx log:

2017-03-23T14:24:59.06945 starting nginx...
2017-03-23T14:26:49.91219 2017/03/23 14:26:49 [error] 126#0: *14 [lua] elasticsearch_setup.lua:124: timed out waiting for eleasticsearch before setup, rerunning..., context: ngx.timer
2017-03-23T14:28:46.28969 2017/03/23 14:28:46 [error] 126#0: *14 [lua] elasticsearch_setup.lua:124: setup(): timed out waiting for eleasticsearch before setup, rerunning..., context: ngx.timer
2017-03-23T14:30:37.71610 2017/03/23 14:30:37 [error] 126#0: *14 [lua] elasticsearch_setup.lua:124: setup(): timed out waiting for eleasticsearch before setup, rerunning..., context: ngx.timer
2

However, the elasticsearch db is accessible:

curl https://search-XXXXXX.us-east-1.es.amazonaws.com:443/_cluster/health 

{"cluster_name":"312707417434:api-umbrella-test014","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":1,"active_shards":1,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":1,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":50.0} 

Using the bundled elasticsearch db, everything works fine.

[martinzuern]

Maybe I should add that I'm using the latest docker image:

➜  ~ docker images nrel/api-umbrella  
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
nrel/api-umbrella   latest              96883b6c4607        4 weeks ago         835 MB

[martinzuern]

After some more investigation, I found the following line in the logs:

2017-05-05T07:51:04.81271 2017/05/05 07:51:04 [notice] 10233#0: *13 [lua] elasticsearch_setup.lua:29: wait_for_elasticsearch(): failed to fetch cluster health from elasticsearch (this is expected if elasticsearch is starting up at the same time): 20: unable to get local issuer certificate, context: ngx.timer

which lead me to to this documentation: https://github.com/openresty/lua-nginx-module#lua_ssl_trusted_certificate

With adding a root CA, everything is working fine. Would you accept a PR for adding this option?
cc @GUI @brylie

[GUI]

This is part of v0.14.2 which now released.

[martinzuern]

@GUI nice. Do you have plans to release this to the Docker repository as well? :)

[GUI]

@martinzuern: Oops, sorry about that! The Docker image for v0.14.2 has been released. Thanks for the heads up.