Why are so many Horizon client requests "undefined"?

[ire-and-curses]

What did you do?

Our internal Kibana dashboard shows a very large number of "undefined" clients hitting Horizon.

What did you expect to see?

Since all major SDKs implemented tracking headers (#725) this is surprising (and unhelpful). I expect to see bars corresponding almost entirely to SDKs with headers.

This issue is to look into any possible errors or oversights that might mean we are not tracking this usage accurately. Some questions to ask:

• do we log headers correctly in all cases (e.g. on 404)?
• is there any way to tell if these are from a source like curl?
• do we track streaming correctly?
• do we log the headers correctly in terms of start and end in the logs?

[leighmcculloch]

is there any way to tell if these are from a source like curl?

Curl sends the User-Agent header in the form User-Agent: curl/7.64.0. It should be possible to graph based on that in Kibana.

[bartekn]

do we log headers correctly in all cases (e.g. on 404)?
do we track streaming correctly?

I believe so but someone else can check the code too. Are you sure you are filtering by horizon_msg:"Finished request"? Otherwise the chart will include all the log entries where client_name can be undefined.

is there any way to tell if these are from a source like curl?

We don't store User-Agent in Horizon logs. But this can be checked in ELB access log.

do we log the headers correctly in terms of start and end in the logs?

I don't understand.

[ire-and-curses]

Filtering on Finished request makes no difference (I tried this). If you filter you see that 90% are undefined.

I don't understand.

I meant is it possible that in Horizon logging code we don't pass on headers to one or other of these log messages under some circumstances? But I just tried filtering on "Starting request" and get 88.4%, so the difference is not large (although why is there a difference?).

[bartekn]

Yeah, it's strange. I can see one request coming from Account Viewer ("stellar-sdk": "^2.0.1" so should send client headers, right?) but the client_name is empty. We can check this out. Maybe some browsers are not sending non-standard headers? Horizon bug possible too.