Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Very small percentage of requests were failing to be logged in the analytics #213

Closed
GUI opened this issue Apr 6, 2015 · 1 comment
Closed

Comments

@GUI
Copy link
Member

GUI commented Apr 6, 2015

In the spirit of tracking down bizarre corner-case issues, I discovered that a very small percentage of requests had failed to make their way into our analytics database. This only affected around 30 requests in the past month, so the percentages are quite small, but it's nonetheless concerning and something we should address.

The issue cropped up when the first request in a given month contained a query string parameter like '?something=2015-01-01. Due to how we index the query string parameters in our analytics database, this triggered Elasticsearch to index that as a special date type field. In general, all other requests would log fine, but where things broke was if someone then made a request for?something=2015-1-0(an invalid date) or?something=foo` (the same field, but with a string value). Since elasticsearch was treating that field as a date field (based on the first detected value), these subsequent requests would fail to log, since they contained invalid dates.

So while the number of affected requests that haven't been logged because of this issue have been very small, it's a very reproducible problem, and could become more of a problem if different APIs happened to use the same query string parameter for date vs non-dates.

@GUI GUI added this to the Sprint 18 (3/23-4/3) milestone Apr 6, 2015
@GUI GUI self-assigned this Apr 6, 2015
@GUI
Copy link
Member Author

GUI commented Apr 6, 2015

Fixed by NREL/api-umbrella-router@5f406b4

And additional tests added for this specific scenario, where the first seen requests contained date fields, but then subsequent requests passed in other values.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant