Logs UI GraphQL data source request takes about 40 seconds

[cpmoore]

Kibana version: 7.6.0
Elasticsearch version: 7.6.0
Server OS version: Red Hat 7.7
Browser version: Chrome 78.0.3904.70
Browser OS version: Windows 7
Original install method: RPM from Yum repo

When accessing the logs ui page in version 7.6.0 the initial data source graphql request takes about 40 seconds to complete.

This is the payload of the request

{
  "operationName": "LogEntries",
  "variables": {
    "sourceId": "default",
    "countBefore": 200,
    "countAfter": 200,
    "timeKey": {
      "time": 1582223089284,
      "tiebreaker": 0
    },
    "filterQuery": null
  },
  "query": "query LogEntries($sourceId: ID = \"default\", $timeKey: InfraTimeKeyInput!, $countBefore: Int = 0, $countAfter: Int = 0, $filterQuery: String) {\n source(id: $sourceId) {\n id\n logEntriesAround(key: $timeKey, countBefore: $countBefore, countAfter: $countAfter, filterQuery: $filterQuery) {\n start {\n ...InfraTimeKeyFields\n }\n end {\n ...InfraTimeKeyFields\n }\n hasMoreBefore\n hasMoreAfter\n entries {\n ...InfraLogEntryFields\n }\n }\n }\n}\n\nfragment InfraTimeKeyFields on InfraTimeKey {\n time\n tiebreaker\n}\n\nfragment InfraLogEntryFields on InfraLogEntry {\n gid\n key {\n time\n tiebreaker\n }\n columns {\n ... on InfraLogEntryTimestampColumn {\n columnId\n timestamp\n }\n ... on InfraLogEntryMessageColumn {\n columnId\n message {\n ... on InfraLogMessageFieldSegment {\n field\n value\n }\n ... on InfraLogMessageConstantSegment {\n constant\n }\n }\n }\n ... on InfraLogEntryFieldColumn {\n columnId\n field\n value\n }\n }\n}\n"
}

Our cluster contains about 4.8TB of data (some of that is replicas). Here is a screen shot of our cluster specs from the monitoring page

I've attached a video of the error as well, as long as showing the reference between how long the discover page takes to load versus the logs ui. I had to zip it in order to attach it.
logs-ui-error.zip

The logs ui has always been slower than discovery, but it's wasn't nearly this slow in 7.5.1
I really hope that the speed of the logs ui will improve as it's much easier to view logs in this traditional format, and log aggregation is our main use case for elasticsearch.

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[weltenwort]

our plan to mitigate this is #53138 (the issue mentions the metrics UI but it applies to the logs UI as well)

[weltenwort]

With #58553 having been merged and released in 7.6.1 can you confirm that the situation has improved?

[cpmoore]

@weltenwort Sorry, been out of the country on vacation.
I installed the update on my test cluster before I left and didn't see any issues there, however the cluster size is much smaller. I'm planning on upgrading production tomorrow (03/20/2020) night. I will update you with my results then.

[cpmoore]

@weltenwort I upgraded our production cluster tonight to 7.6.1 and I can verify that the datasource request takes about 3-4 seconds now.

[weltenwort]

Thanks for confirming. 4 seconds is still not great, but not too bad considering the size of your cluster. There are other improvements in the works.

I'll close this for now. Feel free to file a new one or re-open if you think this is in error.