Add basic prometheus alerts

[sfrique]

Some basic alerts to help others like:
increase of 4xx errors
Increase of waiting connections.
spikes on requests. ( could be an attack)

This is just some ideas, there is probably more that make sense!

If you have any, please add it here or do a PR.

I probably will work on something soon with this, so I will have some alerts to add it, but if someone already have soem it would be great!

Thanks

[sysulq]

Grafana 4.x is recommended which supports alerting

[sfrique]

Can't we have it both?
My alerting is centralized at prometheus. But I am looking into grafana for alerting but it won't be anytime soon.

[sysulq]

Hi @sfrique , I would like to add some prometheus alert rules, and PR is welcome.