Packaged lmdb release fails with "Device Busy"

[jcea]

Steps to reproduce:

1. pkgin install lmdb pip2.7
2. pip install lmdb
3. Run the following Python code TWICE:

#!/usr/bin/env python

import lmdb
lmdb.open('/tmp/qq', map_size=512*1024*1024, subdir=True,
          mode=0o700, max_dbs=64, max_readers = 64, max_spare_txns=50)

The first execution creates the database.

If you execute the code again, it will fails with a "Device Busy". This affect too to stock utilities, it is not related to Python. For instance, execute "mdb_dump -a /tmp/qq >/dev/null" after creating the database:

[jcea@Babylon5 /tmp]$ mdb_dump -a /tmp/qq >/dev/null
mdb_env_open failed, error 16 Device busy

It seems that current LMDB development version (not released yet) contains a bug fix for this: Check the two "memset()" lines in https://github.com/dw/py-lmdb/blob/master/lib/mdb.master/mdb.c#L4994 . After paching 0.9.18 release with those two lines, library works as expected.

[mamash]

Do you do not have to install the packaged lmdb at all, the Python lmdb extension uses its own embedded lmdb source instead. For your particular case, you should request a new release for dw/py-lmdb that contains the fix.

I'll backport the fix to the standalone lmdb package (the same fix was already brought to our openldap packages, where embedded source is used too).

As far as I can see, all relevant upstreams fixed the problem, it's just a matter of releases getting pushed out.

[mamash]

For the record, this is the upstream fix commit:

LMDB/lmdb@c367c1f

[jcea]

I have configured python-lmdb to use the system version because I am using too the command line tools and I want to be sure that I am using the same version everywhere. But you are right that I should bug the python maintainer to release a new version: jnwatson/py-lmdb#142

Since this is already solved upstream, WHEN can I expect a "pkgin full-upgrade" to get the fix?

[mamash]

Our pkgsrc production cluster is currently undergoing a full revamp and I'm pretty sure it will take a few more days. I will provide a better ETA as soon as I can.

[jcea]

I have configured python-lmdb to use the system version because I am using too the command line tools and I want to be sure that I am using the same version everywhere. But you are right that I should bug the python maintainer to release a new version: jnwatson/py-lmdb#142

Since this is already solved upstream, WHEN can I expect a "pkgin full-upgrade" to get the fix?

[mamash]

And I have no update yet to give you... :)

[jcea]

Ping... :-?

[jperkin]

I've backported this in 0e07fe6 and e2b45f0 and the updated packages should be available later today.

[jcea]

Should I do anything special?

[root@P2Ppriv ~]# date
December 19, 2016 03:26:16 AM 
[root@P2Ppriv ~]# pkgin update
processing remote summary (https://pkgsrc.joyent.com/packages/SmartOS/2016Q3/x86_64/All)...
database for https://pkgsrc.joyent.com/packages/SmartOS/2016Q3/x86_64/All is up-to-date
[root@P2Ppriv ~]# pkgin full-upgrade
nothing to do.

If this is something only available in future 2016Q4, it is a month away, the fix is trivial and current 2016Q3 version doesn't work at all... :-?

[jperkin]

Ah, it was backported to 2015Q4, not 2016Q3. I'm on vacation now but may get a few minutes to work on this.

[jperkin]

This should be available in 2016Q3 now.

[jcea]

Confirmed. It works correctly now. Enjoy your holidays.

Thanks a lot!!.