Can't sort on field with custom date format mapping

[jensbengtsson]

I'm running elasticsearch 1.4

{

    "status": 200,
    "name": "Bevatron",
    "cluster_name": "poc",
    "version": {
        "number": "1.4.0",
        "build_hash": "bc94bd81298f81c656893ab1ddddd30a99356066",
        "build_timestamp": "2014-11-05T14:26:12Z",
        "build_snapshot": false,
        "lucene_version": "4.10.2"
    },
    "tagline": "You Know, for Search"

}

I have a type with the following mapping:

{
   "bopo": {
      "mappings": {
         "ProductionTypeCapacity": {
            "properties": {
               "AddedDate": {
                  "type": "date",
                  "format": "dateOptionalTime"
               },
               "ChangedDate": {
                  "type": "date",
                  "format": "dateOptionalTime"
               },
               "CorporateBrandCapacities": {
                  "type": "string",
                  "index": "not_analyzed"
               },
               "HadesScriptUsed": {
                  "type": "string"
               },
               "Key": {
                  "type": "string"
               },
               "Month": {
                  "type": "date",
                  "format": "yyyy-MM"
               },
               "ProductionUnitId": {
                  "type": "string",
                  "index": "not_analyzed"
               },
               "SupplierId": {
                  "type": "string",
                  "index": "not_analyzed"
               }
            }
         }
      }
   }
}

A hit looks like this:

  {
            "_index": "bopo",
            "_type": "ProductionTypeCapacity",
            "_id": "ProductionTypeCapacity-120886-120288-2014-12",
            "_score": 1,
            "_source": {
               "Key": "120886-120288-2014-12",
               "AddedDate": "2014-03-04T02:33:04.9430000",
               "ChangedDate": "2014-03-04T10:12:24.3830000",
               "CorporateBrandCapacities": "Home",
               "Month": "2014-12",
               "ProductionUnitId": "120288",
               "SupplierId": "120886",
               "HadesScriptUsed": "CapacityRegistrationScript"
            }
         }

When I execute the following query:

POST bopo/ProductionTypeCapacity/_search
{
  "query": {
    "match_all": {}
  },
  "sort": [
    {
      "Month": {
        "order": "desc"
      }
    }
  ]
}

I get this error:

{
   "error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[81vkOCltQm6PWmlJSmiVog][bopo][0]: QueryPhaseExecutionException[[bopo][0]: query[ConstantScore(cache(_type:ProductionTypeCapacity))],from[0],size[10],sort[<custom:\"Month\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@149bf6c8>!]: Query Failed [Failed to execute main query]]; nested: ClassCastException[org.elasticsearch.index.fielddata.plain.AbstractAtomicOrdinalsFieldData$1 cannot be cast to org.elasticsearch.index.fielddata.AtomicNumericFieldData]; }{[81vkOCltQm6PWmlJSmiVog][bopo][1]: QueryPhaseExecutionException[[bopo][1]: query[ConstantScore(cache(_type:ProductionTypeCapacity))],from[0],size[10],sort[<custom:\"Month\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@603af23>!]: Query Failed [Failed to execute main query]]; nested: ClassCastException[org.elasticsearch.index.fielddata.plain.AbstractAtomicOrdinalsFieldData$1 cannot be cast to org.elasticsearch.index.fielddata.AtomicNumericFieldData]; }{[81vkOCltQm6PWmlJSmiVog][bopo][2]: QueryPhaseExecutionException[[bopo][2]: query[ConstantScore(cache(_type:ProductionTypeCapacity))],from[0],size[10],sort[<custom:\"Month\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@695b5d68>!]: Query Failed [Failed to execute main query]]; nested: ClassCastException[org.elasticsearch.index.fielddata.plain.AbstractAtomicOrdinalsFieldData$1 cannot be cast to org.elasticsearch.index.fielddata.AtomicNumericFieldData]; }{[81vkOCltQm6PWmlJSmiVog][bopo][3]: QueryPhaseExecutionException[[bopo][3]: query[ConstantScore(cache(_type:ProductionTypeCapacity))],from[0],size[10],sort[<custom:\"Month\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@4627c21b>!]: Query Failed [Failed to execute main query]]; nested: ClassCastException[org.elasticsearch.index.fielddata.plain.AbstractAtomicOrdinalsFieldData$1 cannot be cast to org.elasticsearch.index.fielddata.AtomicNumericFieldData]; }{[81vkOCltQm6PWmlJSmiVog][bopo][4]: QueryPhaseExecutionException[[bopo][4]: query[ConstantScore(cache(_type:ProductionTypeCapacity))],from[0],size[10],sort[<custom:\"Month\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@446d2a0b>!]: Query Failed [Failed to execute main query]]; nested: ClassCastException[org.elasticsearch.index.fielddata.plain.AbstractAtomicOrdinalsFieldData$1 cannot be cast to org.elasticsearch.index.fielddata.AtomicNumericFieldData]; }]",
   "status": 500
}

[clintongormley]

Hi @jensbengtsson

Are you creating mappings manually when you create the index, or are you relying on dynamic mapping to add fields for you?

[jensbengtsson]

Manual mapping

[clintongormley]

@jpountz please can you take a look

[jpountz]

This looks to me like a conflict between several types, one having Month mapped as a string which has been used to build fielddata, and another one mapping Month as a date.

@jensbengtsson Does your bopo index have another type that maps Month as a string? Note that this issue would also occur if you first indexed some data with Month mapped as a string, deleted the type (only the type, not the whole index), and then indexed data again with Month mapped as a date.

[jensbengtsson]

@jpountz
OK, I thought deleting the mapping would make me able to change the type definition. But then it makes sense because the entire index has not been recreated, only the specific type mapping.

[jpountz]

@jensbengtsson I am closing this issue now, but please reopen if recreating the whole index doesn't solve the issue.

[bimalkeeth]

Hi
Restricting Deleting type cause huge bottle neck in parent type indexing as is allways required to create new index. we can not drop particuler type and recreate.

[jpountz]

@bimalkeeth We do not deny that deleting types would be useful. However, there is no way to properly support it at the Lucene level, so we had no choice but to remove this feature which was causing trouble in the past since Lucene's field infos could go out of sync with elasticsearch's mappings.

[clintongormley]

@bimalkeeth the new reindex api makes reindexing really easy