Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deprecate extracting _routing and _id from document fields #6730

Closed
clintongormley opened this issue Jul 4, 2014 · 11 comments · Fixed by #9623
Closed

Deprecate extracting _routing and _id from document fields #6730

clintongormley opened this issue Jul 4, 2014 · 11 comments · Fixed by #9623
Labels
>breaking :Search Foundations/Mapping Index mappings, including merging and defining field types Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch

Comments

@clintongormley
Copy link
Contributor

Currently routing and ID values can be passed in the query string and url respectively, but they can also be extracted from fields within the document.

This has a significant performance impact because each doc needs to be parsed on the node which receives the index/get/update/delete request in order to know which node should handle the request.

On top of that, there are clashes between (eg) routing values set in fields and parent settings.

We should deprecate the functionality to extract these values from fields, and make it the responsibility of the client code instead.

It should still be possible to set _routing to required. Perhaps we should set this automatically if the user ever passes in a routing or parent value at index time?

Relates to #8870
@clintongormley clintongormley mentioned this issue Jul 4, 2014
Closed
@jpountz
Copy link
Contributor

jpountz commented Jul 8, 2014

+1

@clintongormley clintongormley mentioned this issue Jul 8, 2014
Closed
@javanna
Copy link
Member

javanna commented Jul 9, 2014

+1

This was referenced Jul 17, 2014
Closed
Closed
@jpountz jpountz removed the discuss label Sep 5, 2014
This was referenced Nov 11, 2014
Closed
Closed
@clintongormley clintongormley added the help wanted adoptme label Nov 27, 2014
@clintongormley
Copy link
Contributor Author

This change needs to be made in a backwards compatible way.

From 2.0:

From 3.0:

  • remove support completely

Users should still be able to set _routing.required as it is a useful safety check. Perhaps we should set this value automatically if the user indexes a document with a custom routing value?

@clintongormley clintongormley added the :Search Foundations/Mapping Index mappings, including merging and defining field types label Dec 10, 2014
@clintongormley clintongormley mentioned this issue Dec 10, 2014
Closed
13 tasks
@jpountz
Copy link
Contributor

jpountz commented Dec 24, 2014

We should deprecate extracting the _timestamp field as well.

@clintongormley clintongormley mentioned this issue Dec 24, 2014
Closed
@clintongormley
Copy link
Contributor Author

We should deprecate extracting the _timestamp field as well.

@jpountz I'm not sure about this, but I think it requires a separate discussion, so I've opened #9058

@clintongormley
Copy link
Contributor Author

Also see #5558

This was referenced Dec 30, 2014
Closed
Closed
@rjernst rjernst mentioned this issue Feb 10, 2015
Merged
rjernst added a commit to rjernst/elasticsearch that referenced this issue Feb 10, 2015
@rjernst
Mappings: Remove ability to set path for _id and _routing on 2.0+ ind…
b3474f6 
…exes

_id and _routing now no longer support the 'path' setting on indexes
created with 2.0.  Indexes created before 2.0 still support this
setting for backcompat.

closes elastic#6730
rjernst added a commit that referenced this issue Feb 10, 2015
@rjernst
Mappings: Deprecate extracting _id and _routing from document
67fe3dc 
See #6730
@rjernst rjernst removed the help wanted adoptme label Feb 10, 2015
masteinhauser added a commit to masteinhauser/logstash-output-elasticsearch that referenced this issue Feb 27, 2015
@masteinhauser
Support routing option for documents
b58934f 
The bulk API supports a specified `_routing` value for document
placement within a shard in an index. This should be exposed for custom
routing for aliases, filtering, etc.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-routing

This also respects the move away from the routing parameter within the
event as defined in:
elastic/elasticsearch#6730
@masteinhauser masteinhauser mentioned this issue Feb 27, 2015
Closed
masteinhauser added a commit to masteinhauser/logstash-output-elasticsearch that referenced this issue Feb 27, 2015
@masteinhauser
Support routing option for documents
9472bc4 
The bulk API supports a specified `_routing` value for document
placement within a shard in an index. This should be exposed for custom
routing for aliases, filtering, etc.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-routing

This also respects the move away from the routing parameter within the
event as defined in:
elastic/elasticsearch#6730
@ostefano
Copy link

ostefano commented Apr 7, 2015

Can someone confirm me that this is why copy_to inside _id mappings does not currently work?

@rjernst
Copy link
Member

rjernst commented Apr 7, 2015

This change was made only to master. As far as I can tell, copy_to was never supported for the _id field.

@rjernst rjernst mentioned this issue May 8, 2015
Closed
@celesteking
Copy link

I had to dig through the bloat over internet in order to get to this information. You should really DOCUMENT that there will be no substitution for path and that copy_to can't be used to "extract" _id.

@rjernst rjernst mentioned this issue Jun 22, 2015
Merged
rjernst added a commit to rjernst/elasticsearch that referenced this issue Jun 22, 2015
@rjernst
Mappings: Lockdown _timestamp
12e7cbe 
This is a follow up to elastic#8143 and elastic#6730 for _timestamp. It removes
support for `path`, as well as any field type settings, and
enables docvalues for _timestamp, for 2.0.  Users who need to
adjust these settings can use a date field.
@ankane ankane mentioned this issue Oct 29, 2015
Closed
6 tasks
@siddharthlatest siddharthlatest mentioned this issue Mar 27, 2018
Merged
@itsnavee
Copy link

is there an alternative to using your own ID to avoid duplicate record publishing

@jpountz
Copy link
Contributor

jpountz commented Jan 2, 2019

@itsnavee No, there isn't. In the future please ask questions at discuss.elastic.co as we want to keep this repo about feature requests and bug reports.

@mwilliammyers mwilliammyers mentioned this issue Nov 20, 2019
Open
@javanna javanna added the Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch label Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
>breaking :Search Foundations/Mapping Index mappings, including merging and defining field types Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Mappings: Remove ability to set path for _id and _routing on 2.0+ indexes rjernst/elasticsearch
7 participants
@clintongormley @rjernst @jpountz @celesteking @javanna @ostefano @itsnavee