Under the Hood

This wiki page describes how Zoom works under the hood. In particular, it describes what commands Zoom sends to Redis when you call Save, Find, etc. and how indexes and queries work.

Basic Operations

Save

When you call Save, a few things happen:

First, if you have embedded RandomId in your model struct, ModelId has not previously been called, and the Id property is an empty string, Zoom randomly generates and sets the Id field for the model. The id is generated here and consists of 4 components:

1. The current UTC unix time with second precision
2. An atomic counter which is always 4 characters long and cycles through the range of 0 to 11,316,495
3. A unique hardware identifier based on the MAC address of the current machine
4. A pseudo-randomly generated sequence of 6 characters

Second, the exported fields in the struct are converted to a form Redis can understand, namely they are all converted to a slice of bytes. Primitive fields (string, int, uint, float64, etc) are converted via the Redis driver. Non-nil pointer fields are dereferenced and their underlying values are stored. Nil pointer fields are stored as "NULL". All other fields are converted to binary encoding via the gob package. The conversion occurs here. Once all the fields are converted, they are stored in Redis hash via the HMSET command. The key used for the model hash is exposed via the ModelKey method.

Third, if CollectionOptions.Index is true, the model id is added to a set of all models for a specific type via the SADD command. The key used for all the model ids is exposed via the AllIndexKey method. This set is used for the FindAll method as well as a starting point for unordered queries.

Fourth, any indexed fields identified by the struct tag zoom:"index" will be indexed and stored in a sorted set. The key used for the sorted set for each field index is exposed via the FieldIndexKey method. See below for more information about how indexes work.

Here is a more concrete example. Let's say you have a Person type:

type Person struct {
    Name string
    Age  int     `zoom:"index"`
    zoom.RandomId
}

And you create and save a new person like so:

person := Person{
    Name: "Bob",
    Age: 27,
}
if err := People.Save(&person); err != nil {
    // handle err
}

When you call Save, a random id will be generated for the model. For the sake of simplicity let's say the id that was generated was "foo". Here are the Redis commands that will be run:

# Add the fields to the main model hash
HMSET Person:foo Name Bob Age 27
# Add the model id to the set of all person ids
SADD Person:all foo
# Index the Age field in a sorted set
ZADD Person:Age 27 foo

Find

You might be able to guess how Find works based on the description of Save. Find simply gets all the fields from the hash in Redis and scans them into a model type. As a more concrete example, the following code:

p := Person{}
if err := People.Find("foo", &p); err != nil {
    // handle error
}

Will result in the following Redis command:

HMGET Person:foo Name Age

Then Zoom will simply convert the reply from Redis back into the types of the original fields and set the field values using reflection.

Delete

The following happens when you call Delete:

1. The main hash for the model is deleted with the DEL command.
2. The model id is removed from the set of all model ids for a specific type.
3. The model id is removed from any field indexes.

So if you run the following code:

if _, err := People.Delete("foo"); err != nil {
    // handle error
}

Here are the Redis commands that will be executed:

DEL Person:foo
SREM Person:all foo
ZREM Person:Age foo

Queries and Indexes

For these examples we're going to define a new model type:

type IndexedModel struct {
    Int    int    `zoom:"index"`
    Bool   bool   `zoom:"index"`
    String string `zoom:"index"`
}

Numeric Indexes

You already have seen a little bit about how numeric indexes work. Zoom stores numeric indexes as a sorted set, where each score corresponds to a field value, and each member is the id of some model. So if we saved a model like so:

model := IndexedModel{
    Int: 5,
}
if err := IndexedModels.Save(&model); err != nil {
   // handle error
}

The index would be saved with the following Redis command:

ZADD IndexedModel:Int 5 foo

Note that we are again pretending that the id assigned to the model was "foo" for simplicity. The same basic rule applies for other types of numeric field types, such as float and uint.

Bool Indexes

Indexes on bool fields work similarly to numeric fields. There is just an extra conversion step where true is converted to a score of 1 and false is converted to a score of 0. So if we saved a model with the Bool field set like so:

model := IndexedModel{
    Bool: true,
}
if err := IndexedModels.Save(&model); err != nil {
   // handle error
}

The Bool index would be saved with the following Redis command:

ZADD IndexedModel:Bool 1 foo

String Indexes

String indexes are a little different, because we cannot use strings as scores for sorted sets in Redis. So instead, Zoom relies on a workaround. String indexes are stored as sorted sets where all the scores are 0 and each member has the following format: value\x00id, where \x00 is the NULL character. So if we saved a model with the String field set like so:

model := IndexedModel{
    String: "bar",
}
if err := IndexedModels.Save(&model); err != nil {
   // handle error
}

The String index would be saved with the following Redis command:

ZADD IndexedModel:String 0 "bar\x00foo"