Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update schema mappings to include all current layers #238

Merged
merged 1 commit into from
Aug 29, 2017

Conversation

missinglink
Copy link
Member

@missinglink missinglink commented Aug 28, 2017

the schema allows creating new _types dynamically using the _default_ API, we have been using this feature when introducing new layers such as street.

there is, however, an advantage to creating the _type explicitly. Elasticsearch will error instead of returning 0 results for any _type that is requested but is not present in the _mapping.

by creating all known layers at schema creating time, we can avoid these sort of errors, this is less relevant for full-planet, all-layer builds and more relevant to smaller builds which only import a subset of layers.

I haven't seen a bug report for it, but I imagine that if someone only imported WOF and requested something from a combination of street or region layers, a 500 level error would be produced rather than simply returning matches from the region layer.

Copy link
Member

@orangejulius orangejulius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and will certainly help someone with the right combination of data imported.

In the long run, it looks like Elasticsearch is going to deprecate the concept of types completely in versions 6 and 7. Since we use the same type for all our different records, any sense of how much work it would be to use only a single type?

@missinglink
Copy link
Member Author

missinglink commented Aug 29, 2017

my understanding is that a type was introduced to give the feeling of a table (ie. a logical subset of a database) but internally all the data is actually all sitting in the same index, it is simply implemented as an indexed filter operation on the index.

if that's the case then we could begin to introduce a "layer" field which is a copy of the _type and then migrate the queries over to use a boolean filter instead of the existing. type API.

doesn't sound technically difficult but sounds time-consuming and error-prone.

@missinglink missinglink merged commit 105fca5 into master Aug 29, 2017
@orangejulius
Copy link
Member

So I knew something about this seemed familiar. It turns out we've been filtering on a layer field for over a year. This means that as we desire, we can start to remove usage of _type all together, and definitely in time for Elasticsearch changes which are still a long way out.

@orangejulius orangejulius deleted the update_layers branch April 16, 2018 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants