Firefox Sync TokenServer

This service is responsible for allocating Firefox Sync users to one of several Sync Storage nodes. It provides the "glue" between Firefox Accounts and the SyncStorage API, and handles:

• Checking the user's credentials as provided by FxA
• Sharding users across storage nodes in a way that evenly distributes server load
• Re-assigning the user to a new storage node if their FxA encryption key changes
• Cleaning up old data from e.g. deleted accounts

The service was originallly conceived to be a general-purpose mechanism for connecting users to multiple different Mozilla-run services, and you can see some of the historical context for that original design here and here.

In practice today, it is only used for connecting to Firefox Sync.

How to run the server

Like this:

$ make install
$ ./local/bin/pip install gunicorn
$ ./local/bin/gunicorn --paste etc/tokenserver-dev.ini

API

Firfox Sync clients must first obtain user credentials from FxA, which can be either:

• A BrowserID assertion with audience of https://token.services.mozilla.com/
• An OAuth access token bearing the scope https://identity.mozilla.com/apps/oldsync

They then provide this in the Authorization header of a GET request to the Tokenserver, which will respond with the URL of the user's sync storage node, and some short-lived credentials that can be used to access it.

More detailed API documentation is available here.

Using BrowserID

To access the user's sync data using BrowserID, the client must obtain a BrowserID assertion with audience matching the tokenserver's public URL, as well as the user's Sync encryption key. They send the BrowserID assertion in the Authorization header, and the first half of the hex-encoded SHA256 digest of the encryption key in the X-Client-State header, like so:

GET /1.0/sync/1.5
Host: token.services.mozilla.com
Authorization: BrowserID <assertion>
X-Client-State: <hex(sha256(kSync))[:32]>

Using OAuth

To access the user's sync data using OAuth, the client must obtain an FxA OAuth access_token with scope https://identity.mozilla.com/apps/oldsync, and the corresponding encryption key as a JWK. They send the OAuth token in the Authorization header, and the kid field of the encryption key in the X-KeyID header, like so:

GET /1.0/sync/1.5
Host: token.services.mozilla.com
Authorization: Bearer <access_token>
X-KeyID: <JWK['kid']>

Response

The tokenserver will validate the provided credentials, and either look up the user's existing storage node allocation or assign them to a new one. It responds with the location of the storage node and a set of short-lived credentials that can be used to access it:

{
 'id': <token>,
 'key': <request-signing secret>,
 'api_endpoint': 'https://db42.sync.services.mozilla.com/1.5/12345',
 'uid': 12345,
 'duration': 300,
}

Storage Token

The value of <token> is intended to be opaque to the client, but is in fact an encoded JSON blob signed using a secret key shared between the tokenserver and the storage nodes. This allows the tokenserver to securely communicate information about the user to their storage node. The fields contained therein include:

• uid: A numeric userid that uniquely identifies this user, on this storage node, using this encryption key
• node: The intended storage node on which these credentials can be used
• expires: A timestamp for when the credentials expire
• fxa_uid: The user's stable FxA user id, as a hex string
• fxa_kid: The key-id of the JWK representing the user's sync encryption key

Specifying the Data Store

The data store is specified in the [tokenserver] section as sqluri. If you want to use a longer lived data store than sqlite3, you will need to specify the Data Source Name (DSN) as a Universal Resource Locator (URL) such as: sqluri = mysql+pymysql://scott:tiger@localhost/tokenserver. See the SQLAlchemy Engine specification for details.

TokenServer comes with support for pymysql, mysqldb, and sqlite3 by default. Additional databases will require pip install.

Data Model

The core of the TokenServer's data model is a table named users that maps each user to their storage node, and that provides enough information to update that mapping over time. Each row in the table contains the following fields:

• uid: Auto-incrementing numeric userid, created automatically for each row.
• service: The service the user is accessing; in practice this is always sync-1.5.
• email: Stable identifier for the user; in practice this is always <fxa_uid>@api.accounts.firefox.com.
• nodeid: The storage node to which the user has been assigned.
• generation: A monotonically increasing number provided by the FxA server, indicating the last time at which the user's login credentials were changed.
• client_state: The hash of the user's sync encryption key.
• keys_changed_at: A monotonically increasing timestamp provided by the FxA server, indicating the last time at which the user's encryption keys were changed.
• created_at: Timestamp at which this node-assignment record was created.
• replaced_at: Timestamp at which this node-assignment record was replaced by a newer assignment, if any.

As you can see, this table contains some unnecessarily general names; these are a legacy of earlier plans to re-use Tokenserver for multiple Mozilla services and with multiple identity providers.

The generation column is used to detect when the user's FxA credentials have been changed and to lock out clients that have not been updated with the latest credentials. Tokenserver tracks the highest value of generation that it has ever seen for a user, and rejects BrowserID assertions in which the generation number is less than that high-water mark. Note that OAuth clients do not provide a generation number, because OAuth tokens get revoked immediately when the user's credentials are changed.

The client_state column is used to detect when the user's encryption key changes. When it sees a new value for client_state, Tokenserver will replace the user's node assignment with a new one, so that data encrypted with the new key will be written into a different storage "bucket" on the storage nodes.

The keys_changed_at column tracks the timestamp at which the user's encryption keys were last changed. BrowserID clients provide this as a field in the assertion, while OAuth clients provide it as part of the X-KeyID header. Tokenserver will check that changes in the value of keys_changed_at always correspond to a change in client_state, and will use this pair of values to construct the fxa_kid field that is communicated to the storage nodes.

When replacing a user's node assignment, the previous column is not deleted immediately. Instead, it is marked as "replaced" by setting the replaced_at timestamp, and then a background job periodically purges replaced rows (including making a DELETE request to the storage node to clean up any old data stored under that uid).

For this scheme to work as intended, it's expected that storage nodes will index user data by either:

1. The tuple (fxa_uid, fxa_kid), which identifies a consistent set of sync data for a particular user, encrypted using a particular key.
2. The numeric uid, which changes whenever either of the above two values change.