Sorting field in Discover is not updated on index pattern changes

[fdartayre]

Kibana version: 7.14.0

Elasticsearch version:

Server OS version:

Browser version:

Browser OS version:

Original install method (e.g. download page, yum, from source, etc.):

Describe the bug:
Sorting field is not reset to the default time field when switching from an index pattern to another. Not sure if this is a bug or the expected behavior, but this seems to defeat the purpose of having a default time field.

Steps to reproduce:
Create 2 indices, and 2 index patterns: timefield is the time field of my_index and timefield_2 is the time field of my_index_2. In my_index_2 also add timefield to the mapping but it won't contain data:

PUT my_index
{
  "mappings": {
    "properties": {
      "timefield": {
        "type": "date"
      }
    }
  }
}

POST my_index/_bulk
{ "index": {} }
{ "timefield": "2021-08-05" }
{ "index": {} }
{ "timefield": "2021-08-04" }
{ "index": {} }
{ "timefield": "2021-08-03" }

PUT my_index_2
{
  "mappings": {
    "properties": {
      "timefield": {
        "type": "date"
      },
      "timefield_2": {
        "type": "date"
      }
    }
  }
}

POST my_index_2/_bulk
{ "index": {} }
{ "timefield_2": "2021-08-05" }
{ "index": {} }
{ "timefield_2": "2021-08-04" }
{ "index": {} }
{ "timefield_2": "2021-08-03" }

Now in Discover, when switching from my_index to my_index_2, Discover sees the current sorting field (timefield) exists in my_index_2 and doesn’t change it. Since there is no data in this field in my_index_2, it shows up as unsorted.

Expected behavior:
Sort on default time field by default.

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[kertal]

so when you switch index pattern without having changed the default sorting, the sorting field of the new index pattern is used. But when you have changed the sorting, and the sorted field is also available in the new index pattern, the last user sorting is used. So this is intended. However we could check if the user change of sorting was applied to the index pattern timefield, and therefore switch to the next index pattern timefield. Wonder if there are cases when the current behavior would be preferred.

[ThunderM]

Hi @kertal,

The problem we face, which is described by Frederic, does not completely match your description.

so when you switch index pattern without having changed the default sorting, the sorting field of the new index pattern is used.

The default sorting field only changes if the default sorting field of the first index pattern does not exist in the new index pattern.

In our case we have two types of index patterns:

• pattern1: default sorting of timestampfield1, but does not have timestampfield2.
• pattern2: default sorting of timestampfield2, timestampfield1 exists, but only for <1% of the documents

So in our case starting on pattern2, gives default sorting timestampfield2.
After switching to pattern1, the default sorting is changed to timestampfield1, as timestampfield2 does not exist in pattern1.
Thusfar this behaviour is what is expected, however, changing back to pattern2 still gives us default sorting of timestampfield1, as this field does exists for some of the documents. This causes the documents to appear as mostly unsorted.

I hope this clarifies our case.

Kind regards,

Michael Yeh

[kertal]

thank you @ThunderM will have a closer look here!