Nontrivial Addons

Building a nontrivial addon (node-sqlite3)

Building native nodejs addons for Android is a little tricky. I will walk through building a complicated (and very useful) node addon for Android, sqlite3. Although this specific guide is used for sqlite3, many of the learnings can be applied to other addons as well.

Also, follow the steps taken at skabbes/anode-addon-example

Obtaining the code view diff

Grab a copy of node-sqlite3 to start with

git clone git://github.com/developmentseed/node-sqlite3.git

Pull in the alternative sqlite3 sources view diff

Download the amalgamation sources from the sqlite website, and extract it into node-sqlite3/deps.

Copy in Makefiles from anode sample addon code view diff

Sample files of both Android.mk and Application.mk are available in anode/sdk/addon/sample/ start. These are good starting points for writing the Android makefiles

Modifying Android.mk and Application.mk for SQLite view diff

The Application.mk is very simple and should be self explanatory, except one part APP_STL.

Using APP_STL in your makefile

Since the sqlite3 bindings make use of the STL, but the android version of it is rather crippled, you will need to link in your own version. Thankfully, Android gives you several choices.

1. system -> Use the default minimal system C++ runtime library.
2. gabi++_static -> Use the GAbi++ runtime as a static library.
3. gabi++_shared -> Use the GAbi++ runtime as a shared library.
4. stlport_static -> Use the STLport runtime as a static library.
5. stlport_shared -> Use the STLport runtime as a shared library.
6. gnustl_static -> Use the GNU STL as a static library.
7. gnustl_shared -> Use the GNU STL as a shared library.

The Android.mk file is a bit more involved, this particular version builds 2 separate modules, sqlite3 as a shared library, and sqlite3 bindings for node. These 2 portions are pretty obviously separated in the repo.
You must also supply the cppflags and cflags to the compiler (if any are necessary). You will notice custom cpp flags for building the module, these are taken right out of the addon's wscript file. You will notice in node-sqlite3 a function called build which defines the compiler flags to use. This will be very similar for other addons.

def build(bld):
   obj = bld.new_task_gen("cxx", "shlib", "node_addon")
   build_internal_sqlite3(bld)
   obj.cxxflags = ["-g", "-D_FILE_OFFSET_BITS=64", "-D_LARGEFILE_SOURCE",
                   "-DSQLITE_ENABLE_RTREE=1", "-pthread", "-Wall"]
   # snip ...

Build it for the first time!

At this point, you are ready to build. Since we didn't follow the "normal" Android conventions for layout out our project directory, we need to specify a couple of variables to ndk-build

ndk-build NDK_PROJECT_PATH=. NDK_APPLICATION_MK=Application.mk

Ensure that ndk-build (from the Android NDK) is on your build path. Running the above command will produce three files in libs/armeabi/. Hold on to these, you will need them later.

1. libgnustl_shared.so
2. libsqlite3.so
3. sqlite3_bindings.node

Loading the correct version of sqlite3 _ bindings from Javascript view diff

Next, we need to ensure that the Javascript portions of the node module actually use the new built version of the addon. We will change this

// Original version
var sqlite3 = module.exports = exports = require('./sqlite3_bindings.node');

to this

// Android Version
var sqlite3 = module.exports = exports = require('sqlite3_bindings.node');

Fix uv_default_loop() bugs view diff

At this time, you would expect everything to work, but unfortunately there are bugs in practically every single node addon which uses uv_queue_work (include sqlite3). Instead of uv_default_loop() calls, addons should use the node v0.7.x api node::Isolate::Loop(). Since anode doesn't support v0.7.x (yet) we must replace every call to uv_default_loop() with our patched version of node::Isolate::Loop(). Without making this change, your addon will appear to load, but won't actually do any work.

/*
 * Add a compatible function call to the 'correct' way to determine the
 * current loop in node 0.7.  This is essential if you are using Isolates.
 *
 * This should be removed once the addon is updated for at least node 0.7.x
 */
uv_loop_t * Loop(){
#if defined(HAVE_ISOLATES) && HAVE_ISOLATES
    return node::Isolate::GetCurrent()->GetLoop();
#else
    return uv_default_loop();
#endif
}

Build it for the second time!

All the changes in the code should still compile just fine, plus we need new libraries since we changed code.

ndk-build NDK_PROJECT_PATH=. NDK_APPLICATION_MK=Application.mk

Load the libraries and use the addon

In order to use the libraries, you will need to place them into /data/data/com.yourdomain.android/node_modules and also System.load them. Anode already does this process for libjninode.so and bridge.node in RuntimeNative.java.

// IN THIS ORDER
System.load("/path/on/device/to/libgnustl_shared.so");
System.load("/path/on/device/to/libsqlite3.so");
System.load("/data/data/com.yourdomain.android/node_modules/sqlite3_bindings.node");

Congrats, you can now use node-sqlite3 bindings on Android. I have tested this, and it is compatible with Android's sqlite implementation as well.