Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize indexing by letting view specify a "type" value #590

Closed
snej opened this issue Feb 10, 2015 · 6 comments
Closed

Optimize indexing by letting view specify a "type" value #590

snej opened this issue Feb 10, 2015 · 6 comments

Comments

@snej
Copy link
Contributor

snej commented Feb 10, 2015

It's very common for a view to index only a particular type of document, as indicated by its type property. So the map block will often start with if (![doc[@"type] isEqual: @"sometype"]) return;.

If we allowed a view to state that it only applies to docs with a specific type value, then the indexer could use that information to avoid a lot of wasted time fetching and parsing irrelevant docs:

  • In SQLite we can add a type column to rev and index it, then restrict the indexer query to only the matching type(s).
  • In ForestDB we can put the type into the document metadata and check it before reading the document body.
@snej
Copy link
Contributor Author

snej commented Feb 10, 2015

Note 1: CBLQueryBuilder can make use of this by checking whether the map predicate tests doc.type.

Note 2: If we want developers not using CBLQueryBuilder to use this, we'll need an API addition — something like a CBLView.documentType property.

@snej snej added in progress and removed backlog labels Mar 10, 2015
@snej snej self-assigned this Mar 10, 2015
@snej snej closed this as completed in f2dd833 Mar 10, 2015
snej added a commit that referenced this issue Mar 10, 2015
This will automatically optimize the view indexing for a lot of common queries.
For #590
@snej
Copy link
Contributor Author

snej commented Mar 11, 2015

There's a tech note in the wiki now, describing what this is and how to use it.

snej added a commit that referenced this issue Mar 11, 2015
Added a benchmark too (View_Benchmarks.m)
Fixes #590

WATCH OUT: This optimization won't work correctly in SQLite-based dbs
used with earlier revisions of CBL 1.1. (It does work correctly with
existing 1.0.x dbs that are upgraded by this commit or later.)
The reason is that the existing rows in the 'revs' table don't have the
new 'doc_type' column filled in. I could write code to fill it in when
adding the new column, but I don't think the work is justified since
this will never affect actual users, only a few bleeding-edge adopters.
@jeanregisser
Copy link

Great addition!

Would it be possible to make the "type" key configurable?

When my team migrated to couchbase we had a set of existing documents to migrate and couldn't use the "type" key because it was already used in them for a different purpose.
We ended up using a key name "doctype" instead.

@snej
Copy link
Contributor Author

snej commented Mar 12, 2015

I knew someone was going to ask that… 😬 I'll have to think about it. It's probably not too hard, but I'm not sure where to store the magic property name and how to set it.

@skela
Copy link

skela commented May 5, 2016

I have a somewhat related question in that we don't have "type", but use a document id prefix as a type descriptor. Would it be possible to make use of this ?

@snej
Copy link
Contributor Author

snej commented May 5, 2016

Same answer, I'm afraid! If we make it configurable we can do something like letting you specify a pattern in the docID.

There wasn't any issue covering making this flexible, so I've just filed one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants