The state of Blob Support?

[keyz182]

Hi,

I'm currently maintaining an offline mapping application. For the offline mapping, I use the MBTiles format, which is basically an SQLite db with map tiles (usually PNG/JPG) stored as BLOBs.

I also need to use SQLite for other data storage purposes in the app.

Until now, I have been using your plugin for data storage, and cordova-plugin-mbtiles for mapping which has worked, but is duplicating the work of utilising and operating SQLite files.

I have come up with this gist to allow me to use your plugin to read the MBTiles packages. As can be seen, it's not very complicated, but does rely on reading BLOBs.

In #255, you mention BLOB support is removed. The patch mentioned in the gist seems to be enough to get reading BLOBs working on Android. Is there more to it than I'm seeing, and other problems I've not encountered? I'm just trying to asses if there's any reason I shouldn't make the switch.

The example in the gist seems to work fine for me, and provides performace comparable to to the cordova-plugin-mbtiles plugin.

Thanks,
Keyz

[brodycj]

Sorry I didn't respond yesterday. I am in the process of some major restructuring and was also planning some changes to the blob interface. I found that maintaining the existing behavior of blob handling was getting in my way. Also, the existing code seems to support the reading of blob fields from the database but not writing blob fields to the database, making this a very limited case.

For the future, not sure if I want to keep using Base64 encoding/decoding, due to its inefficiency.

[aarononeal]

MBTiles support was the scenario I had in mind when I implemented the original blob support. In this case, external tools are used to populate the tile database and apps just need to read the cached image back.

Although not needed for this scenario, native Blob writing support for iOS and Android was part of the same commit, so unless that didn't get integrated, the statement above is not entirely accurate.

Also, base64 is used only for marshaling the data across, not when it's at rest. Does Cordova offer a better alternative now?

[brodycj]

@aarononeal, I only integrated the changes to remove the embedded LIBB64 from the iOS version. I was planning to look at your other blob enhancements after I am finished with some project restructuring.

I liked your SQLBlob but was thinking it could be its own project that could also help people using normal Web SQL (in the browser).

[brodycj]

Also, base64 is used only for marshaling the data across, not when it's at
rest. Does Cordova offer a better alternative now?

I don't completely understand what you are trying to ask.

AFAIK: in general Cordova does not internally care about Base64. Cordova is simply using JSON encoding and decoding to facilitate communication between the Javascript part and the native part, as transparently as possible.

[aarononeal]

The JS SQLBlob could certainly be a standalone component. I had been considering including it with a general DataURL project that I've been planning to publish. But, all of the native side still needs to be integrated into the plugin to enable the possibility of binary blobs, and you could do this today as an option for folks even without the JS SQLBlob piece.

About base64--you mentioned you weren't sure if you wanted to keep it due to inefficiency, so I was wondering what alternatives you had in mind. Cordova doesn't have any way to marshal binary data directly that I am aware of.

[keyz182]

Thanks for the responses. As @aarononeal mentioned, MBTiles need only read blobs. Base64 is useful for MBTiles as the base64 image is inserted into the tag directly, leaving the browser to decode.

I'm also wondering what alternatives you have in mind to Base64 to pass binary data from Plugin to JS?

[brodycj]

Base64 is useful for MBTiles as the base64 image is inserted into the tag directly, leaving the browser to decode.

Agreed. We know that in general, Base64 was used to make it easier for Javascript to handle what is normally binary-encoded data.

One thing I really had wanted to avoid was encoding and decoding Base64 in Javascript itself.

Also, PouchDB took a different solution to handling binary data, as described in the WebSQL case of https://github.com/nolanlawson/state-of-binary-data-in-the-browser/blob/master/README.md#chrome (and .../README.md#safariios) and in section 3 ("http://pouchdb.com/2014/10/26/10-things-i-learned-from-reading-and-writing-the-pouchdb-source.html") of http://pouchdb.com/2014/10/26/10-things-i-learned-from-reading-and-writing-the-pouchdb-source.html.

AFAIK it was possible for PouchDB to store and retrieve binary data strings using this plugin, without converting to Base64. @nolanlawson can you confirm this or not?

I also want to avoid is breaking PouchDB, in the case where it stores a binary string in a Blob field and then retrieves it later.

I am really busy with some other tasks right now, can look into a solution within the next week or so.

[nolanlawson]

@brodybits Answered here: #255 (comment)

[brodycj]

As stated in README.md, the BLOB type is currently not supported. I am still extremely busy dealing with some other issues, and am also planning some changes to the API to improve the handling of large SQL batches.

Looking through the code, here is the state of the blob support, which is subject to change:

• storing of blobs has never been supported. File Support storage of BLOB type #263 was filed as a possible enhancement for this.
• for iOS version: upon reading of a field that is actually a BLOB, the data will be automatically returned as a Base64 string
• for Android version:
  • by default, using sqlite4java implementation, which will treat a BLOB field from the database just like any other TEXT field (not sure if it will work at all)
  • if you include the androidDatabaseImplementation: 2 option when opening the database~~, the implementation~~ and then read a BLOB field, the implementation will automatically return the data as a Base64 string