MIQ(MiqUiWorker::Runner#log_long_running_requests) Long running http(s) request: '/'

[cronosy]

Hello guys.
So, I share with all of you the next warning output that is affecting my customers experience.
MIQ(MiqUiWorker::Runner#log_long_running_requests) Long running http(s) request: '/'
I've increase the UI worker in my web servers ui but still appearing the message:
My distribution is:
2 UI servers.
2 Worker servers
2 database server act-stb

I would like to hear ideas to identify why is appearing the warning.

Kind regards.

[jrafanie]

@cronosy sorry for the delay in responding. This typically happens if there's a really slow request or there's a timing issue between threads. It should log the pid and thread id of the request. I would suggest you grep the production log / journal for that pid+thread id to see the request. Does it ever finish? How long does it take? If it never finishes, look at the full production.log when it's started, are there other new "Started" requests roughly at the same time this one started?

[cronosy]

Hello @jrafanie no worries. Thanks for your help. Following the suggest i could determine the next:
My database server is not working well due high consumption of CPU and memory. I had to modify the PostgreSQL memory parameters (read https://www.manageiq.org/docs/reference/latest/deployment_planning_guide/) there are an suggestion about how and how many increase the memory. These enhance the behavior. However, apparently recover the VMware metrics when the cluster is big, impact in the performance; Other solution was limit the access to some cluster's resources(datastores).

kind regards,

[jrafanie]

Hi @cronosy CPU could be missing index, leading to table scans or maybe your autovacuum/analyze process isn't maintaining the database. It could be many things but those are the things that come to mind.

You can try something like this #22665 to monitor it ongoing.

Or you can run something similar manually:

select psut.relname, psut.schemaname, psut.last_vacuum, psut.last_autovacuum, psut.last_analyze, psut.last_autoanalyze, pg_class.reltuples from pg_stat_user_tables psut JOIN pg_class on psut.relid = pg_class.oid order by reltuples desc ;

Your last_autovacuum and last_autoanalyze times should be pretty recent for larger row count / size tables. It's sorted by relpules so it's generally the largest row count tables first. If it's not running recently, you'll need to go into the postgresql.log to see if it's logging the autovacuum daemon activity.

It's possible you're hitting shared memory limits, causing the autovacuum to fail before finishing.

Let me know what you find.

[cronosy]

this are my changes to postgres:
shared_buffers = 4GB # 25% of physical memory