Merging mappings with subobjects: false can result in invalid mappings

[felixbarny]

Elasticsearch Version

8.11

Installed Plugins

No response

Java Version

bundled

OS Version

macOS 14.2

Problem Description

I found a way to corrupt the mappings using subobjects: false. The reason is that ES is only validating that there must be no object mappers within in an object that's configured with subobjects: false while parsing mappings:

elasticsearch/server/src/main/java/org/elasticsearch/index/mapper/ObjectMapper.java

Lines 312 to 320 in fb5c744

	if (objBuilder.subobjects.value() == false && type.equals(NestedObjectMapper.CONTENT_TYPE)) {
	throw new MapperParsingException(
	"Tried to add nested object ["
	+ fieldName
	+ "] to object ["
	+ objBuilder.name()
	+ "] which does not support subobjects"
	);
	}

However, it's possible to create an invalid mapping using two distinct mapping updates that are valid on their own but invalid when the mappings are merged.

This invalid mapping merge is caught in testing with this assertion that re-parses the merged mappings:

elasticsearch/server/src/main/java/org/elasticsearch/index/mapper/MapperService.java

Line 526 in fb5c744

assert assertSerialization(newMapper);

Since assertions are disabled in production, it's possible to create an invalid mapping.

To fix this, we could enable the assertion in production, too. Or, maybe a better way would be to catch the error scenario earlier, during mapping merge.

In these cases, we could check if we're trying to add an ObjectMapper even though subobjects are disabled:

elasticsearch/server/src/main/java/org/elasticsearch/index/mapper/ObjectMapper.java

Lines 546 to 549 in fb5c744

	if (mergeIntoMapper == null) {
	merged = mergeWithMapper;
	} else if (mergeIntoMapper instanceof ObjectMapper objectMapper) {
	merged = objectMapper.merge(mergeWithMapper, reason, objectBuilderContext);

Steps to Reproduce

PUT test
{
  "mappings": {
    "properties": {
      "parent": {
        "type": "object",
        "subobjects": false
      }
    }
  }
}

PUT test/_mapping
{
  "properties": {
    "parent.child.grandchild": {
      "type": "keyword"
    }
  }
}

# POST test/_search 503 Service Unavailable
{
  "error": {
    "root_cause": [
      {
        "type": "no_shard_available_action_exception",
        "reason": null
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "test",
        "node": null,
        "reason": {
          "type": "no_shard_available_action_exception",
          "reason": null
        }
      }
    ]
  },
  "status": 503
}
# GET test/_mapping 200 OK
{
  "test": {
    "mappings": {
      "properties": {
        "parent": {
          "subobjects": false,
          "properties": {
            "child": {
              "properties": {
                "grandchild": {
                  "type": "keyword"
                }
              }
            }
          }
        }
      }
    }
  }
}

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)

[felixbarny]

This will be fixed with #103542