-
Notifications
You must be signed in to change notification settings - Fork 46
Scalability
Beiwe app data are stored as flat files on Amazon Web Services' Simple Storage Service (AWS S3), rather than in a conventional database.
The reason for this is that conventional databases run into scaling issues at certain sizes, specifically when the database grows too large to be stored on a single machine and needs to be "sharded" into multiple pieces that can be run on multiple machines. Beiwe's flat file storage architecture avoids that problem.
Beiwe generates a massive amount of data; a participant in a study with all data streams turned on could generate about 1 gigabyte of data per month. Most of that is accelerometer data, since every accelerometer datapoint is about 50 bytes of data, and the accelerometer records about 5 to 100 datapoints per second. (Other data streams produce less data simply because they record less frequently.) This means that a study using the accelerometer data stream on 100 participants for 10 months could generate about 1 terabyte of data.
Conventional Databases are also designed for fast retrieval