Skip to content
This repository has been archived by the owner on Jul 13, 2023. It is now read-only.

Commit

Permalink
docs: update to reflect message table rotation is optional
Browse files Browse the repository at this point in the history
Closes #1194
  • Loading branch information
jrconlin committed May 4, 2018
1 parent 83595a1 commit 9d5cc64
Showing 1 changed file with 42 additions and 22 deletions.
64 changes: 42 additions & 22 deletions docs/architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -129,44 +129,64 @@ a lookup is done against these ``chids``.

.. _table-rotation:

Message Table Rotation
----------------------
Message Table Rotation (legacy)
-------------------------------

As of version 1.45.0, message table rotation can be disabled. This is because
DynamoDB now provides automatic entry expiration. This is controlled in our
data by the "expiry" field. (***Note***, field expiration is only available in
full DynamoDB, and is not replicated with the mock DynamoDB API provided for
development.) The following feature is disabled with the `no_table_rotation`
flag set in the `autopush_shared.ini` configuration file.

If table rotation is disabled, the last message table used will become
'frozen' and will be used for all future messages. While this may not be
aesthetically pleasing, it's more efficient than copying data to a new,
generic table. If it's preferred, service can be shut down, previous tables
dropped, the current table renamed, and service brought up again.

To avoid costly table scans, autopush uses a rotating message and router table.
Clients that haven't connected in 30-60 days will have their router and message
table entries dropped and need to re-register.
..
Tables are post-fixed with the year/month they are meant for, ie:
*Message Table Rotation information*

messages-2015-02
To avoid costly table scans, autopush uses a rotating message and router
table.
Clients that haven't connected in 30-60 days will have their router and
message table entries dropped and need to re-register.

Tables must be created and have their read/write units properly allocated by a
separate process in advance of the month switch-over as autopush nodes will
assume the tables already exist. Scripts are provided that can be run weekly to
ensure all necessary tables are present, and tables old enough are dropped.
Tables are post-fixed with the year/month they are meant for, i.e. ::

.. seealso::
messages_2015_02

Tables must be created and have their read/write units properly allocated
by a separate process in advance of the month switch-over as autopush
nodes will assume the tables already exist. Scripts are provided that can be
run weekly to ensure all necessary tables are present, and tables old
enough are dropped.

.. seealso::

Table maintenance script: https://github.com/mozilla-services/autopush/blob/master/maintenance.py

Within a few days of the new month, the load on the prior months table will fall
as clients transition to the new table. The read/write units on the prior
month may then be lowered.
Within a few days of the new month, the load on the prior months table will
fall as clients transition to the new table. The read/write units on the
prior month may then be lowered.

Message Table Interaction Rules
-------------------------------
Rotating Message Table Interaction Rules (legacy)
-------------------------------------------------

Due to the complexity of having notifications spread across two tables, several
rules are used to avoid losing messages during the month transition.

The logic for connection nodes is more complex, since only the connection node
knows when the client connects, and how many messages it has read through.

The router table uses the ``curmonth`` field to indicate the last month the
client has read notifications through. This is independent of the last_connect
since it is possible for a client to connect, fail to read its notifications,
then reconnect. This field is updated for a new month when the client connects
**after** it has ack'd all the notifications out of the last month.
When table rotation is allowed, the router table uses the ``curmonth`` field
to indicate the last month the client has read notifications through. This is
independent of the last_connect since it is possible for a client to
connect, fail to read its notifications, then reconnect. This field is
updated for a new month when the client connects **after** it has ack'd all
the notifications out of the last month.

To avoid issues with time synchronization, the node the client is connected to
acts as the source of truth for when the month has flipped over. Clients are
Expand Down

0 comments on commit 9d5cc64

Please sign in to comment.