FieldMapper null values, serialization and include_defaults
#58823
Labels
:Search Foundations/Mapping
Index mappings, including merging and defining field types
Team:Search Foundations
Meta label for the Search Foundations team in Elasticsearch
team-discuss
Comments
romseygeek
added
:Search Foundations/Mapping
romseygeek
added a commit
that referenced
this issue
Sep 1, 2020
Several field mappers have a null_value parameter, that allows you to specify a placeholder value to insert into a document if the incoming value for that field is null. The default value for this is always null, meaning "add no placeholder". However, we explicitly bar users from setting this parameter directly to null (done in #7978, in order to fix an NPE). This exclusion means that if a mapper is serialized with include_defaults, then we either need to special-case null_value to ensure that it is not output when it holds the default value, or we find that the resulting serialized form cannot be used to create a mapping. This stops us doing some useful generic testing of mappers. This commit permits null as a parameter value for null_value, and changes the tests to check that it is a) permissible and b) applied without throwing errors. As part of the testing changes, a new base class MapperServiceTestCase is refactored from MapperTestCase, holding the various helper methods related to building mappings but not the single-mapper specific abstract methods. Closes #58823
romseygeek
added a commit
to romseygeek/elasticsearch
that referenced
this issue
Sep 1, 2020
Several field mappers have a null_value parameter, that allows you to specify a placeholder value to insert into a document if the incoming value for that field is null. The default value for this is always null, meaning "add no placeholder". However, we explicitly bar users from setting this parameter directly to null (done in elastic#7978, in order to fix an NPE). This exclusion means that if a mapper is serialized with include_defaults, then we either need to special-case null_value to ensure that it is not output when it holds the default value, or we find that the resulting serialized form cannot be used to create a mapping. This stops us doing some useful generic testing of mappers. This commit permits null as a parameter value for null_value, and changes the tests to check that it is a) permissible and b) applied without throwing errors. As part of the testing changes, a new base class MapperServiceTestCase is refactored from MapperTestCase, holding the various helper methods related to building mappings but not the single-mapper specific abstract methods. Closes elastic#58823
romseygeek
added a commit
that referenced
this issue
Sep 2, 2020
Several field mappers have a null_value parameter, that allows you to specify a placeholder value to insert into a document if the incoming value for that field is null. The default value for this is always null, meaning "add no placeholder". However, we explicitly bar users from setting this parameter directly to null (done in #7978, in order to fix an NPE). This exclusion means that if a mapper is serialized with include_defaults, then we either need to special-case null_value to ensure that it is not output when it holds the default value, or we find that the resulting serialized form cannot be used to create a mapping. This stops us doing some useful generic testing of mappers. This commit permits null as a parameter value for null_value, and changes the tests to check that it is a) permissible and b) applied without throwing errors. As part of the testing changes, a new base class MapperServiceTestCase is refactored from MapperTestCase, holding the various helper methods related to building mappings but not the single-mapper specific abstract methods. Closes #58823
61 tasks
javanna
added
Team:Search Foundations
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We have a number of field mappers that allow users to set a
null_value, for use when a documentdoes not contain a value for that particular field. In general, the default for this value is
null, meaningdon't do anything, but we do not allownullto be set explicitly as this value.When we serialize a field mapper, we allow users to ask for the default values to be emitted even if they
have not been set. For
null_values, this yields a quandary: if we emit the valuenull, then the serializedmapping cannot be used in a subsequent put mapping request, because
nullis not accepted as avalid value; but if we don't emit anything, then it looks as though we're missing a field even though the
user has asked for all configuration values. #57666 added tests to check serialization and deserialization,
and stopped null values being emitted, but it's not clear what the right choice is.
The text was updated successfully, but these errors were encountered: