PufferDB

PufferDB is a ⚡ key-value storage powered by Protocol Buffers (aka Protobuf) and Coroutines.

The purpose of this library is to provide an efficient, reliable and Android independent storage.

Why Android independent? The SharedPreferences and many great third-party libraries (like Paper and MMKV) requires the Android Context to work. But if you are like me and want a kotlin-only data module (following the principles of Clean Architecture), this library is for you!

This project started as a library module in one of my personal projects, but I decided to open source it and add more features for general use. Hope you like!

About Protobuf

Protocol Buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. Compared to JSON, Protobuf files are smaller and faster to read/write because the data is stored in an efficient binary format.

Features

• Fast
• Works on Android and JVM
• Simple API
• Thread-safe
• Wrappers for Coroutines and RxJava

Supported types

So far, PufferDB supports the following types:

• Double and List<Double>
• Float and List<Float>
• Int and List<Int>
• Long and List<Long>
• Boolean and List<Boolean>
• String and List<String>

Getting Started

Import to your project

1. Add the JitPack repository in your root build.gradle at the end of repositories:

allprojects {
    repositories {
        maven { url 'https://jitpack.io' }
    }
}

2. Next, add the desired dependencies to your module:

dependencies {
    // Core library
    implementation "com.github.adrielcafe.pufferdb:core:$currentVersion"

    // Android helper
    implementation "com.github.adrielcafe.pufferdb:android:$currentVersion"

    // Coroutines wrapper
    implementation "com.github.adrielcafe.pufferdb:coroutines:$currentVersion"

    // RxJava wrapper
    implementation "com.github.adrielcafe.pufferdb:rxjava:$currentVersion"
}

Current version:

Platform compatibility

	core	android	coroutines	rxjava
Android	✓	✓	✓	✓
JVM	✓		✓	✓

Core

As the name suggests, Core is a standalone module and all other modules depends on it.

To create a new Puffer instance you must tell which file to use.

val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile)
// or
val puffer = PufferDB.with(pufferFile, myCoroutineScope, myCoroutineDispatcher)

If you are on Android, I recommend to use the Context.filesDir as the parent folder. If you want to save in the external storage remember to ask for write permission first.

Its API is similar to SharedPreferences:

puffer.apply {
    val myValue = get<String>("myKey")
    val myValueWithDefault = get("myKey", "defaultValue")
    
    put("myOtherKey", 123)

    getKeys().forEach { key ->
        // ...
    }

    if(contains("myKey")){
        // ...
    }

    remove("myOtherKey")

    removeAll()
}

But unlike SharedPreferences, there's no apply() or commit(). Changes are saved asynchronously every time a write operation (put(), remove() and removeAll()) happens.

Threading

PufferDB uses a ConcurrentHashMap to manage a thread-safe in-memory cache for fast read and write operations.

Changes are saved asynchronously with the help of a StateFlow (to save the most recent state in a race condition) and Mutex locker (to prevent simultaneous writes).

It is possible to run the API methods on the Android Main Thread, but you should avoid that. You can use one of the wrapper modules or built in extension functions for that (listed below).

Android

The Android module contains an AndroidPufferDB helper class:

class MyApp : Application() {

    override fun onCreate() {
        super.onCreate()
        // Init the PufferDB when your app starts
        AndroidPufferDB.init(this)
    }
}

// Now you can use it anywhere in your app
class MyActivity : AppCompatActivity() {

    // Returns a default Puffer instance, the file is located on Context.filesDir
    val corePuffer = AndroidPufferDB.withDefault()

    // Returns a File that should be used to create a Puffer instance
    val pufferFile = AndroidPufferDB.getInternalFile("my.db")
    val coroutinePuffer = CoroutinePufferDB.with(pufferFile)
}

Coroutines

The Coroutines module contains a CoroutinePufferDB wrapper class and some useful extension functions:

val pufferFile = File("path/to/puffer/file")
val puffer = CoroutinePufferDB.with(pufferFile)

// All methods are suspend functions that runs on Dispatchers.IO context
launch {
    puffer.apply {
        val myValue = get<String>("myKey")
        val myValueWithDefault = get("myKey", "defaultValue")
        
        put("myOtherKey", 123)

        getKeys().forEach { key ->
            // ...
        }

        if(contains("myKey")){
            // ...
        }

        remove("myOtherKey")

        removeAll()
    }
}

If you don't want to use this wrapper class, there's some built in extension functions that can be used with the Core module:

val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile) // <- Note that we're using the Core PufferDB

launch {
    puffer.apply {
        val myValue = getSuspend<String>("myKey")

        val myValue = getAsync<String>("myKey").await()
        
        // You can use your own coroutine scope and dispatcher
        putSuspend("myOtherKey", 123, myCoroutineScope, myCoroutineDispatcher)

        putAsync("myOtherKey", 123, myActivityScope).await()
    }
}

RxJava

The RxJava module contains a RxPufferDB wrapper class and some useful extension functions:

val pufferFile = File("path/to/puffer/file")
val puffer = RxPufferDB.with(pufferFile)

puffer.apply {
    // Some methods returns Single<T>...
    get<String>("myKey") // OR get("myKey", "defaultValue")
        .subscribeOn(Schedulers.io())
        .observeOn(AndroidSchedulers.mainThread())
        .subscribe { myValue ->
            // ...
        }

    // ... And others returns Completable
    put("myOtherKey", 123)
        // ...
        .subscribe {
            // ...
        }

    getKeys()
        // ...
        .subscribe { keys ->
            // ...
        }

    contains("myKey")
        // ...
        .subscribe { contains ->
            // ...
        }

    remove("myOtherKey")
        // ...
        .subscribe {
            // ...
        }

    removeAll()
        // ...
        .subscribe {
            // ...
        }
}

Like the Coroutines module, the RxJava module also provides some useful built in extension functions that can be used with the Core module:

val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile) // <- Note that we're using the Core PufferDB

puffer.apply {
    val myValue = getSingle<String>("myKey").blockingGet()

    putCompletable("myOtherKey", 123).blockingAwait()

    getKeysObservable().blockingSubscribe { keys ->
        // ...
    }
}

Benchmark

Write & Read

	Write 1k strings (ms)	Read 1k strings (ms)
PufferDB	20	5
SharedPreferences	278	7
MMKV	13	8
Paper	818	169
Binary Prefs	121	9
Hawk	15183	207

	Write 100k strings (ms)	Read 100k strings (ms)
PufferDB	259	32
SharedPreferences	💥	💥
MMKV	871	516
Paper	💥	💥
Binary Prefs	1082	101
Hawk	💥	💥

File size

	1k strings (kb)
PufferDB	25
SharedPreferences	20
MMKV	40
Paper	61
Binary Prefs	53
Hawk	27

	100k strings (kb)
PufferDB	2.907
SharedPreferences	💥
MMKV	4.104
Paper	💥
Binary Prefs	5.175
Hawk	💥

Tested on Moto Z2 Plus

You can run the Benchmark through the sample app.