Offsets spuriously lost from field after indexing a separate type with matching field name

[robinhughes]

If there are two mappings within the index that have a matching field name - one configured with and one without index_options: offsets then we can add a document of the first type (i.e. the mapping with offsets), and query it successfully using the postings highlighter.

But after adding several documents of the second type (i.e. the mapping without offsets), the same query (returning the same single original document only) fails with the exception:

IllegalArgumentException[field 'title' was indexed without offsets, cannot highlight

This failure takes about 10 seconds to reproduce when using refresh_interval: 1s, and indexing the documents with a 1s pause between documents. This suggests that perhaps it is something in the index refresh processing that is getting confused between the mappings when updating the index.

The failure only happens after adding several extra documents of the second type over a period spanning several refresh_intervals. Attempts to recreate the issue with a longer refresh_interval resulted in many more documents of the second type having to be added before the issue arose.

The following gist is a short shell script that demonstrates this issue with elasticsearch 1.4.0:

https://gist.github.com/robinhughes/4b1f381a94dd905d71a6

[clintongormley]

Hi @robinhughes

You're running into a known issue: fields with the same name in different types must have the same mapping. In the next version we're planning on enforcing this requirement. Closing in favour of #4081