Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sorting does not work when querying an alias #7343

Closed
OlegYch opened this issue Aug 19, 2014 · 15 comments
Closed

sorting does not work when querying an alias #7343

OlegYch opened this issue Aug 19, 2014 · 15 comments
Assignees
Labels

Comments

@OlegYch
Copy link

OlegYch commented Aug 19, 2014

using 1.3.1 and 1.3.2 the ordering of results differs when searching on alias vs searching on the index with actual data from that alias (only one index in the alias has any data in the type)

GET myindex_search/_alias
{
   "myindex_7": {
      "aliases": {
         "myindex_search": {},
         "myindex": {}
      }
   },
   "users_idx3": {
      "aliases": {
         "users_idx": {},
         "myindex_search": {},
         "users_idx_search": {},
      }
   }
}
GET myindex_search/mytype/_search
{
  "query": {
    "constant_score": {
      "filter": {
        "and": {
"filters": [
              {"term": {
                "user_id": 582313
              }}

          ]
        }
      }
    }
  },
  "sort": [
    {
      "data.start_date": {
        "order": "asc",
        "ignore_unmapped": "true"
      }
    }
  ]
,
"aggs": {
  "max": {
    "max": {
      "field": "data.order_date"
    }
  }
}, 
  "size": 110
}

returns

{
   "took": 5,
   "timed_out": false,
   "_shards": {
      "total": 20,
      "successful": 20,
      "failed": 0
   },
   "hits": {
      "total": 16,
      "max_score": null,
      "hits": [
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408441377.377091-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 1210.14,
                  "end_date": 1408449667.817116,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408441377.377091,
                  "number_of_steps": 1620,
                  "duration": 8290.440024495125,
                  "order_date": 1408449667.817116,
                  "id": "1408441377.377091-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.958624986e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408452938.713164-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 10.458,
                  "end_date": 1408453879.153991,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408452938.713164,
                  "number_of_steps": 14,
                  "duration": 940.4408271312714,
                  "order_date": 1408453879.153991,
                  "id": "1408452938.713164-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.958682104e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408050000-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 5336.568,
                  "end_date": 1408136400,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408050000,
                  "number_of_steps": 7144,
                  "duration": 86400,
                  "order_date": 1408136400,
                  "id": "1408050000-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408050000
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408222800-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 4076.379,
                  "end_date": 1408309200,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408222800,
                  "number_of_steps": 5457,
                  "duration": 86400,
                  "order_date": 1408309200,
                  "id": "1408222800-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408222800
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408463037.221965-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 177.786,
                  "end_date": 1408470644.548557,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408463037.221965,
                  "number_of_steps": 238,
                  "duration": 7607.326591610909,
                  "order_date": 1408470644.548557,
                  "id": "1408463037.221965-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408463037.221965
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408456930.040181-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 5.229,
                  "end_date": 1408463006.851273,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408456930.040181,
                  "number_of_steps": 7,
                  "duration": 6076.811092078686,
                  "order_date": 1408463006.851273,
                  "id": "1408456930.040181-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.95870183e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1407790800-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 2.241,
                  "end_date": 1407823148.624126,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1407790800,
                  "number_of_steps": 3,
                  "duration": 32348.62412595749,
                  "order_date": 1407823148.624126,
                  "id": "1407790800-0"
               },
               "user_id": 582313
            },
            "sort": [
               1407790800
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1407877200-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 4134.645,
                  "end_date": 1407963600,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1407877200,
                  "number_of_steps": 5535,
                  "duration": 86400,
                  "order_date": 1407963600,
                  "id": "1407877200-0"
               },
               "user_id": 582313
            },
            "sort": [
               1407877200
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1407963600-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 6652.035,
                  "end_date": 1408050000,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1407963600,
                  "number_of_steps": 8905,
                  "duration": 86400,
                  "order_date": 1408050000,
                  "id": "1407963600-0"
               },
               "user_id": 582313
            },
            "sort": [
               1407963600
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408136400-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 10236.141,
                  "end_date": 1408222800,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408136400,
                  "number_of_steps": 13703,
                  "duration": 86400,
                  "order_date": 1408222800,
                  "id": "1408136400-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408136400
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408309200-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 3161.304,
                  "end_date": 1408395600,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408309200,
                  "number_of_steps": 4232,
                  "duration": 86400,
                  "order_date": 1408395600,
                  "id": "1408309200-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408309200
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408395600-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 2337.363,
                  "end_date": 1408474407.530582,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408395600,
                  "number_of_steps": 3129,
                  "duration": 78807.53058201075,
                  "order_date": 1408474407.530582,
                  "id": "1408395600-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408395600
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408449687.813152-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 5.229,
                  "end_date": 1408450160.890107,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408449687.813152,
                  "number_of_steps": 7,
                  "duration": 473.0769553780556,
                  "order_date": 1408450160.890107,
                  "id": "1408449687.813152-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.95866604e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1407823148.624126-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 1706.148,
                  "end_date": 1407877200,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1407823148.624126,
                  "number_of_steps": 2284,
                  "duration": 54051.37587404251,
                  "order_date": 1407877200,
                  "id": "1407823148.624126-0"
               },
               "user_id": 582313
            },
            "sort": [
               1407823148.624126
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408387971.543879-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 871.749,
                  "end_date": 1408395600,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408387971.543879,
                  "number_of_steps": 1167,
                  "duration": 7628.456121563911,
                  "order_date": 1408395600,
                  "id": "1408387971.543879-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.958361125e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408450305.447627-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 15.687,
                  "end_date": 1408452812.031366,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408450305.447627,
                  "number_of_steps": 21,
                  "duration": 2506.583738684654,
                  "order_date": 1408452812.031366,
                  "id": "1408450305.447627-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.958669096e-315
            ]
         }
      ]
   },
   "aggregations": {
      "max": {
         "value": 4743694330017250000
      }
   }
}

as opposed to correct sorting of data when querying single index

GET myindex_7/mytype/_search
{
  "query": {
    "constant_score": {
      "filter": {
        "and": {
"filters": [
              {"term": {
                "user_id": 582313
              }}

          ]
        }
      }
    }
  },
  "sort": [
    {
      "data.start_date": {
        "order": "asc",
        "ignore_unmapped": "true"
      }
    }
  ]
,
"aggs": {
  "max": {
    "max": {
      "field": "data.order_date"
    }
  }
}, 
  "size": 110
}
{
   "took": 4,
   "timed_out": false,
   "_shards": {
      "total": 8,
      "successful": 8,
      "failed": 0
   },
   "hits": {
      "total": 16,
      "max_score": null,
      "hits": [
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408387971.543879-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 871.749,
                  "end_date": 1408395600,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408387971.543879,
                  "number_of_steps": 1167,
                  "duration": 7628.456121563911,
                  "order_date": 1408395600,
                  "id": "1408387971.543879-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.958361125e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408441377.377091-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 1210.14,
                  "end_date": 1408449667.817116,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408441377.377091,
                  "number_of_steps": 1620,
                  "duration": 8290.440024495125,
                  "order_date": 1408449667.817116,
                  "id": "1408441377.377091-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.958624986e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408449687.813152-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 5.229,
                  "end_date": 1408450160.890107,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408449687.813152,
                  "number_of_steps": 7,
                  "duration": 473.0769553780556,
                  "order_date": 1408450160.890107,
                  "id": "1408449687.813152-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.95866604e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408450305.447627-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 15.687,
                  "end_date": 1408452812.031366,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408450305.447627,
                  "number_of_steps": 21,
                  "duration": 2506.583738684654,
                  "order_date": 1408452812.031366,
                  "id": "1408450305.447627-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.958669096e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408452938.713164-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 10.458,
                  "end_date": 1408453879.153991,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408452938.713164,
                  "number_of_steps": 14,
                  "duration": 940.4408271312714,
                  "order_date": 1408453879.153991,
                  "id": "1408452938.713164-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.958682104e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408456930.040181-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 5.229,
                  "end_date": 1408463006.851273,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408456930.040181,
                  "number_of_steps": 7,
                  "duration": 6076.811092078686,
                  "order_date": 1408463006.851273,
                  "id": "1408456930.040181-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.95870183e-315
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1407790800-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 2.241,
                  "end_date": 1407823148.624126,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1407790800,
                  "number_of_steps": 3,
                  "duration": 32348.62412595749,
                  "order_date": 1407823148.624126,
                  "id": "1407790800-0"
               },
               "user_id": 582313
            },
            "sort": [
               1407790800
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1407823148.624126-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 1706.148,
                  "end_date": 1407877200,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1407823148.624126,
                  "number_of_steps": 2284,
                  "duration": 54051.37587404251,
                  "order_date": 1407877200,
                  "id": "1407823148.624126-0"
               },
               "user_id": 582313
            },
            "sort": [
               1407823148.624126
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1407877200-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 4134.645,
                  "end_date": 1407963600,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1407877200,
                  "number_of_steps": 5535,
                  "duration": 86400,
                  "order_date": 1407963600,
                  "id": "1407877200-0"
               },
               "user_id": 582313
            },
            "sort": [
               1407877200
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1407963600-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 6652.035,
                  "end_date": 1408050000,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1407963600,
                  "number_of_steps": 8905,
                  "duration": 86400,
                  "order_date": 1408050000,
                  "id": "1407963600-0"
               },
               "user_id": 582313
            },
            "sort": [
               1407963600
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408050000-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 5336.568,
                  "end_date": 1408136400,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408050000,
                  "number_of_steps": 7144,
                  "duration": 86400,
                  "order_date": 1408136400,
                  "id": "1408050000-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408050000
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408136400-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 10236.141,
                  "end_date": 1408222800,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408136400,
                  "number_of_steps": 13703,
                  "duration": 86400,
                  "order_date": 1408222800,
                  "id": "1408136400-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408136400
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408222800-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 4076.379,
                  "end_date": 1408309200,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408222800,
                  "number_of_steps": 5457,
                  "duration": 86400,
                  "order_date": 1408309200,
                  "id": "1408222800-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408222800
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408309200-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 3161.304,
                  "end_date": 1408395600,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408309200,
                  "number_of_steps": 4232,
                  "duration": 86400,
                  "order_date": 1408395600,
                  "id": "1408309200-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408309200
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408395600-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 2337.363,
                  "end_date": 1408474407.530582,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408395600,
                  "number_of_steps": 3129,
                  "duration": 78807.53058201075,
                  "order_date": 1408474407.530582,
                  "id": "1408395600-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408395600
            ]
         },
         {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408463037.221965-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 177.786,
                  "end_date": 1408470644.548557,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408463037.221965,
                  "number_of_steps": 238,
                  "duration": 7607.326591610909,
                  "order_date": 1408470644.548557,
                  "id": "1408463037.221965-0"
               },
               "user_id": 582313
            },
            "sort": [
               1408463037.221965
            ]
         }
      ]
   },
   "aggregations": {
      "max": {
         "value": 4743694330017250000
      }
   }
}

nevermind the broken aggregations and "sort" field values, this started to happen only after upgrade to 1.3.2

@OlegYch
Copy link
Author

OlegYch commented Aug 19, 2014

the ticket about 1.3.2 is #7341
before an upgrade to 1.3.2 i was also getting random ClassCastExceptions inside org.apache.lucene.search.TopDocs#merge when sorting on that field
not sure if all of this is an indication of one issue
CheckIndex does not report any errors

@OlegYch
Copy link
Author

OlegYch commented Aug 19, 2014

same failure is observed when querying multiple indexes instead of alias, i.e GET myindex_7,users_idx3/mytype/_search

@clintongormley
Copy link
Contributor

OK, looking at this issue and #7341, it looks like you have fields with the same name in different types in the same index. When you try to load these field values into fielddata, it chooses one of the field mappings (whichever one it finds first) and loads in that format.

This is not new - it has been like this since the beginning. Fields with the same name should be mapped in the same way.

We're planning on enforcing this in the future. See #4081

@OlegYch
Copy link
Author

OlegYch commented Aug 20, 2014

why does it work when searching through single index?

@clintongormley
Copy link
Contributor

It just depends on which field it happens to see first.

@OlegYch
Copy link
Author

OlegYch commented Aug 20, 2014

thanks, got it

@OlegYch
Copy link
Author

OlegYch commented Aug 21, 2014

i made sure all types in all indexes from that alias use the same type for that field but the sorting is still wrong when querying through alias, how can that be ?

@OlegYch
Copy link
Author

OlegYch commented Aug 21, 2014

even cleared the cache for all indexes

@OlegYch
Copy link
Author

OlegYch commented Aug 21, 2014

curl -XGET myserver:9200/myalias/_mapping?pretty 2>/bla  |grep -A3 start_date
              "start_date" : {
                "type" : "double"
              },
              "terrain" : {
--
              "start_date" : {
                "type" : "double"
              }
            }

GET myalias/mytype/_search
{
  "query": {
    "constant_score": {
      "filter": {
        "and": {
"filters": [
              {"term": {
                "user_id": 582313
              }}

          ]
        }
      }
    }
  },
  "sort": [
    {
      "data.start_date": {
        "order": "asc",
        "ignore_unmapped": "true"
      }
    }
  ]
,
"aggs": {
  "max": {
    "max": {
      "field": "data.start_date"
    }
  }
},
  "size": 1100
}

{
   "took": 19,
   "timed_out": false,
   "_shards": {
      "total": 20,
      "successful": 20,
      "failed": 0
   },
   "hits": {
      "total": 5,
      "max_score": null,
      "hits": [
         {
            "_index": "myindex",
            "_type": "mytype",
            "_id": "582313_F5114468-DA0E-4E9F-A195-5122AAEBC475",
            "_score": null,
            "_source": {
               "_id": "582313_F5114468-DA0E-4E9F-A195-5122AAEBC475",
               "data": {
                  "id": "F5114468-DA0E-4E9F-A195-5122AAEBC475",
                  "duration": 897.5677289962769,
                  "end_date": 1408444861.339127,
                  "category": "sleep",
                  "deep_duration": 0,
                  "quality": 0.05621061269658856,
                  "light_duration": 50.45283198356628,
                  "phases": [
                     {
                        "duration": 850.4915599822998,
                        "type": "awake",
                        "date": 1408443960.083729
                     },
                     {
                        "duration": 50.45283198356628,
                        "type": "light",
                        "date": 1408444810.575289
                     },
                     {
                        "duration": 0.3110060691833496,
                        "type": "awake",
                        "date": 1408444861.028121
                     }
                  ],
                  "awake_duration": 850.8025660514832,
                  "order_date": 1408444861.339127,
                  "start_date": 1408443963.771398,
                  "editable": true
               },
               "user_id": 582313
            },
            "sort": [
               1408443963.771398
            ]
         },
         {
            "_index": "myindex",
            "_type": "mytype",
            "_id": "582313_61647662-2738-4C6B-A0BD-1536F1EAA313",
            "_score": null,
            "_source": {
               "_id": "582313_61647662-2738-4C6B-A0BD-1536F1EAA313",
               "data": {
                  "id": "61647662-2738-4C6B-A0BD-1536F1EAA313",
                  "duration": 780,
                  "end_date": 1408531044,
                  "distance": 200,
                  "category": "running",
                  "calories": 292.11,
                  "order_date": 1408531044,
                  "start_date": 1408530264,
                  "editable": true
               },
               "user_id": 582313
            },
            "sort": [
               1408530264
            ]
         },
         {
            "_index": "myindex",
            "_type": "mytype",
            "_id": "582313_2FEA2D8A-4B2A-4C71-8A8D-AE0921B3AAAD",
            "_score": null,
            "_source": {
               "_id": "582313_2FEA2D8A-4B2A-4C71-8A8D-AE0921B3AAAD",
               "data": {
                  "id": "2FEA2D8A-4B2A-4C71-8A8D-AE0921B3AAAD",
                  "duration": 7505.24406504631,
                  "end_date": 1408541467.491296,
                  "category": "sleep",
                  "deep_duration": 0,
                  "quality": 0,
                  "light_duration": 0,
                  "phases": [
                     {
                        "duration": 7505.244183063507,
                        "type": "awake",
                        "date": 1408533962.247113
                     }
                  ],
                  "awake_duration": 7505.244183063507,
                  "order_date": 1408541467.491296,
                  "start_date": 1408533962.247231,
                  "editable": false
               },
               "user_id": 582313
            },
            "sort": [
               1408533962.247231
            ]
         },
         {
            "_index": "myindex",
            "_type": "mytype",
            "_id": "582313_71F810C0-2290-46F5-84FE-36AE390B06FA",
            "_score": null,
            "_source": {
               "_id": "582313_71F810C0-2290-46F5-84FE-36AE390B06FA",
               "data": {
                  "id": "71F810C0-2290-46F5-84FE-36AE390B06FA",
                  "duration": 7620,
                  "end_date": 1408557945.435768,
                  "distance": 100,
                  "category": "cycling",
                  "calories": 1189.0375,
                  "order_date": 1408557945.435768,
                  "start_date": 1408550325.435768,
                  "editable": true
               },
               "user_id": 582313
            },
            "sort": [
               1408550325.435768
            ]
         },
         {
            "_index": "myindex",
            "_type": "mytype",
            "_id": "582313_5C2797AA-7BFD-40AD-99FF-35278E719293",
            "_score": null,
            "_source": {
               "_id": "582313_5C2797AA-7BFD-40AD-99FF-35278E719293",
               "data": {
                  "id": "5C2797AA-7BFD-40AD-99FF-35278E719293",
                  "duration": 3600,
                  "end_date": 1408548615,
                  "is_deleted": true,
                  "distance": 100,
                  "category": "cycling",
                  "calories": 561.7500000000001,
                  "order_date": 1408548615,
                  "start_date": 1408545015,
                  "editable": true
               },
               "user_id": 582313
            },
            "sort": [
               1408545015
            ]
         }
      ]
   },
   "aggregations": {
      "max": {
         "value": 1408550325.435768
      }
   }
}

@clintongormley
Copy link
Contributor

@OlegYch Apologies - I was wrong here, didn't read carefully enough, and was confused by the ClassCastException, which is usually related to the problem I described before.

This does indeed look like a bug. Do you have the stack trace emitted when merging? Also, could you provide:

GET /_settings
GET /_mapping
GET /_aliases

thanks

@clintongormley clintongormley self-assigned this Aug 22, 2014
@OlegYch
Copy link
Author

OlegYch commented Aug 25, 2014

@clintongormley there doesn't seem to be any exceptions caused by query
i've sent you the results of those requests on email, please let me know if there is anything else i can do to diagnoze it

@jpountz
Copy link
Contributor

jpountz commented Sep 4, 2014

I'm not sure the bug is related to aliases. The sort values look wrong in both cases (even when querying a single index), see:

        {
            "_index": "myindex_7",
            "_type": "mytype",
            "_id": "582313_1408387971.543879-0",
            "_score": null,
            "_source": {
               "data": {
                  "distance": 871.749,
                  "end_date": 1408395600,
                  "calories": 0,
                  "motion_type": "stationary",
                  "start_date": 1408387971.543879,
                  "number_of_steps": 1167,
                  "duration": 7628.456121563911,
                  "order_date": 1408395600,
                  "id": "1408387971.543879-0"
               },
               "user_id": 582313
            },
            "sort": [
               6.958361125e-315
            ]
         }

The value in sort should be equal to the value in start_date but they are completely different. But it happens that if you interpret the bits of 1408387971 (the integer part of start date) as double bits, you get 6.958361125E-315, which is the sort value.

So in spite of the fact that the field is mapped as a double in the mappings, it looks like it has been indexed as a long. I will try to dig more...

@jpountz
Copy link
Contributor

jpountz commented Sep 4, 2014

The only explanation I can find is that the same field has been dynamically mapped as a long on some shards and as a double on other shards (before mappings propagation replicated the mappings to all nodes).

@OlegYch
Copy link
Author

OlegYch commented Sep 5, 2014

i explicitly put the mapping for that type before putting any data in it, so i doubt that is the case
i believe the sort value was borked because there were different types in different mappings for that field
currently the value in sort is correct (as i made sure there are no conflicts, and use full field path including type name), but the sorting is still wrong

@clintongormley
Copy link
Contributor

This should have been resolved by the changes in #8870 in 2.0. Closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants