AJAX requests and nginx proxy

[Siraris]

So I setup an Angular app leveraging elastic-service to try to interact with my ES server. Since I used your fantastic cookbook, it installed the nginx proxy to wall off ES from external requests without a username and PW. Upon trying to search using the elastic-service, it fails due to failed authorization because the AJAX request is external to the server.

Is there an easy way around this? I'm guessing what I need to do is setup IP restrictions, so that only certain IP's can POST/DEL/PUT and whitelist GET requests, and require authorization for any external POST/DEL/PUT requests.

[karmi]

the AJAX request is external to the server

This is weird, ES tries hard to support CORS so you shouldn't run into these issues, the cookbook adds the support as well -- if that's what you mean.

See elastic/elasticsearch#2238 -- can you provide similar info about what's not working?

(You can always use JSONP instead of regular Ajax call, to work around the same origin problems.)

[Siraris]

Well, it would work fine if you're interacting with your local machine (localhost:9200), but if you're trying to interact with an application on a remote server, (like I am) then you can't use localhost:9200 with an Angular app, since it tries to connect to the client machine you are on, and not the instance on the server.

So I need to connect to http://my_ip:8080 (since ES is behind the nginx proxy) in order to have the Angular app in conjunction with elastic-service work properly. In doing so, I get the following response:

OPTIONS http://my_ip:8080/index/type/_search 401 (Unauthorized) angular.min.js:99
XMLHttpRequest cannot load http://my_ip:8080/index/tyoe/_search. Origin http://my_ip is not allowed by Access-Control-Allow-Origin.

So the server is returning a 401, not authorized. The reason is, in order to access ES externally, I need to input a username and password as defined by the auth_basic property from your cookbook. If I comment out the last two lines (auth_basic, and auth_basic_user_file), reboot nginx and try to use my Angular app, it works flawlessly.

So by default, no one could use this Cookbook in conjunction with any kind of AJAX request, unless they are able to pass the username/password (I don't know if this is possible, or safe).

My only thought is that rules would need to be added on a case-by-case basis. I assume OPTION and GET would need white list access, while POST/DEL and PUT would need more nuanced rules (only allow from certain IP's and internally on the server).

I hope I explained this well, and I hope I'm not missing something :)

[karmi]

Thanks, that's better. So, the moment you use the Nginx proxy with username/password, all requests must be authorized. Of course it would be possible to allow OPTIONS request, but then, how much sense does it make? You would have to allow GET anyway, and serialize queries into the source param, etc.

Depending on the app you're doing, it might be best to deploy it as an Elasticsearch site plugin, see http://www.elasticsearch.org/tutorials/javascript-web-applications-and-elasticsearch/ for explanation/examples, and also for an example with smart proxying based on cookie authentication.

It is possible in general to use HTTP Auth in Ajax (http://api.jquery.com/jQuery.ajax/#jQuery-ajax-settings), but then again, if your application is public, everybody can see it.

[Siraris]

It's 2 am and I need to get some sleep, but I'll look into things more and reply. I'm sure other people down the line might encounter this issue, so including something in the Cookbook to account for this might be useful. As of now, I just have to imagine that the only reasonable solution is accounting for requests and handling security per requests as the only viable solution. The application will be public facing, but that doesn't mean I want anyone to be sending POST and DEL requests to my ES instance :)

[karmi]

The simplest solution is then to change the Nginx config template in your wrapper cookbook (see “Create a wrapper cookbook with your custom recipes and attributes” at http://dougireton.com/blog/2013/02/16/chef-cookbook-anti-patterns/) and just put OPTIONS and GET requests outside of the authorization block.

[Siraris]

So I've done a lot of researching/learning over the past few days, and I have one issue I hope you can help with.

I stuck with the nginx proxy, and after a lot of banging my head against the wall with their dics, I found the geo and map directives, which allow me to assign granularity to who can send POST/PUT/DELETE etc. requests. FANTASTIC, such a relief, and I finally feel like I'm making progress. Now I can say "Anyone inside our company IP range is able to put and delete and post documents, and everyone else can piss off" :)

Unfortunately, I have one last issue, and that's with the elastic.js service from FullScale. For some reason, they are doing a POST when executing search requests. This means that if I want to do a search on ES from their library, I need to allow ANYONE to POST to ES. I asked on the elastic.js git, and the author said that not all clients can send a request body using GET, so they use POST instead. I'm confused by this since everything I know about RESTful API's says that you use GET to make requests, and POST to push data.

Do you have a suggestion of what to do? I obviously cannot allow anyone to send a POST request to my ES instance, as there could be malicious intent there. Should I pursue a different route than the elastic.js library, or is there some recourse that I can take and continue using the excellent library they created?

[karmi]

With JS/jQuery there's no (easy) way to send a body with HTTP GET request; I have alway used POST as well.

That said, it's possible to encode the JSON search definition into a source URL parameter and send it as a GET request to Elasticsearch. You can either extend the Elastic.js library or ask the authors to allow this "mode" of operation.

For an example, these two requests are functionally identical (the URL parameters version will have some performance overhead, usually not your worry in an JS app, can be also easily cached externally because everything is in the URL):

curl localhost:9200/_search -d '{
  "query" : {
    "match" : {
      "title" : "foo"
    }
  }
}'

echo;echo '---'

curl localhost:9200/_search?source=$(ruby -r uri -e "print URI.escape('{
  \"query\" : {
    \"match\" : {
      \"title\" : \"foo\"
    }
  }
}')")

[karmi]

@Siraris Hi, any update on this?

[Siraris]

@karmi Sorry I didn't update this! I implemented a way to whitelist IP's in the nginx proxy using the geo module and deny_request directive. This will work in the short term, but in the long term I'll most likely need to implement a token generation system. But for the time being, it's simple and easy to just throw the white listed IP's into the proxy.conf.

[karmi]

@Siraris Cool!, how did you change the cookbook's Nginx proxy template? With Chef? Using a "wrapper" cookbook? Is there anything the cookbook could do to make it easier, such as using an attribute for the template path?