Skip to content

Sync Gateway usage of DCP

Jump to bottom
Adam Fraser edited this page Dec 17, 2015 · 1 revision

Sync Gateway uses a DCP feed from the Couchbase Server bucket to maintain it's channel index. In 1.2 our use cases for DCP expand to support the distributed index functionality.

Sync Gateway 1.1.1 and earlier

On startup, each Sync Gateway node starts a DCP feed for all vbuckets on the server, starting at the high seqnos for the buckets. It works this stream, doing channel index processing for the following document types:

  • Update and delete mutations for 'normal' Sync Gateway docs. These are the standard revision-tree based docs created via the Sync Gateway REST API (either directly or via client replication). These include revision tree information in the document body (stored in the _sync property
  • Update and delete mutations for Sync Gateway user and role documents. These are non-versioned documents that also contain channel assignment information.
  • All other document events over the feed are ignored.
  • Rollback, snapshot start/end are also used in the same way.

Sync Gateway 1.2

In 1.2, we've added the ability for Sync Gateway to use a shared channel index. In this mode, the DCP feed is sharded across all Sync Gateway nodes in the cluster. Each Sync Gateway node is assigned a subset of vbuckets, and works a DCP feed for those vbuckets. On startup, each node starts a DCP feed where indexing previously left off.

The set of mutations we're interested in on the sharded feed aligns with the uses cases from 1.1 listed above. A key difference in 1.2 is that we're using the vbucket sequences we see on the feed to build the channel index and drive replication. (in 1.1, we're still using our SG-managed global sequence).

Clone this wiki locally