collector.histogram is slow

[ochaton]

collector histogram is just bunch of counters, it is too bad for it to be slow. I've slightly rewrite collectors/histogram to speed it up:

With constant labels for each observe

127.0.0.1:3301> clock.bench(function() local lbls = {method='api.users.get',ok='true'} local prev = clock.time() for _ = 1, 1e6 do api_timings_hist:observe(clock.time()-prev, lbls) prev = clock.time() end end)
---
- - 16.211009
...

127.0.0.1:3301> clock.bench(function() local lbls = {method='api.users.get',ok='true'} local prev = clock.time() for _ = 1, 1e6 do api_timings_fast:observe(clock.time()-prev, lbls) prev = clock.time() end end)
---
- - 1.884366
...

Without labels on observe

127.0.0.1:3301> clock.bench(function() local prev = clock.time() for _ = 1, 1e6 do api_timings_hist:observe(clock.time()-prev) prev = clock.time() end end)
---
- - 10.5773
...
127.0.0.1:3301> clock.bench(function() local prev = clock.time() for _ = 1, 1e6 do api_timings_fast:observe(clock.time()-prev) prev = clock.time() end end)
---
- - 1.810581
...

Source code of FastHistogram is available on my gist: https://gist.github.com/ochaton/305e4668b41e2721b82b92217d74c9ba

Initialisation

local metrics = require 'metrics'
local metrics_fast_hist = require 'metrics.fast'

metrics.set_global_labels({
	app_name = 'bench',
	instance_name = 'bench_001',
})

local api_timings_fast = metrics_fast_hist:new('api_call', 'Histogram of API call',nil, {})
local api_timings_hist = metrics.histogram('api_call_hist', 'Histogram of API call',nil, {})

rawset(_G, 'api_timings_hist', api_timings_hist)
rawset(_G, 'api_timings_fast', api_timings_fast)

@yngvar-antonsson please take a look, and me you'll suggest better approach to increase perfomance of metrics/histogram

[ochaton]

In my opinion 500K observations per second is still very-very slow

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days