mk_dirs_if_missing fragilities

[thorwhalen]

dol.filesys.mk_dirs_if_missing needs deeper testing, mixing with other store transformers etc.

In general, need more combinatorial wrapper testing.

Dir making only happens on write (consider other ops too!)

By doing so, we'll notice at least one thing, which will make us revise the design of mk_dirs_if_missing, or at least it's name:
It only makes the directories if the directories are missing when we try to write something in it. If, for example, we try to list keys before there was any write, we won't get an empty list (which we should get if the directory doesn't exist). Another example, if we try to get a value for a key, we won't get a KeyError (we should), but a "directory doesn't exist" error.

Maybe we should do the directory check and making at the id_of_key level? The init? What do we do in lazy mode? What do we do in the init? Do we separate root dir and keys like A/B/C (which cannot be made in advance because we don't know what keys a users will chose to write in)?

from know.scrap.mall_making import *
from tempfile import mkdtemp
p = mkdtemp()
pp = os.path.join(p, 'A')
ppp = os.path.join(pp, 'B')
print(f"{os.path.exists(p)=}")
print(f"{os.path.exists(pp)=}")
print(f"{os.path.exists(ppp)=}")

from dol import Files
from dol import mk_dirs_if_missing
F = mk_dirs_if_missing(Files)
s = F(ppp)
list(s)
# FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/j5/brqb67791slb1f04g53vcb780000gp/T/tmp9s1zhmn6/A/B/'

mk_dirs_if_missing after wrap_kvs, breaks

from dol import Files

class G(Files):
    pass
    
from dol import mk_dirs_if_missing
F = mk_dirs_if_missing(G)
s = F(ppp)
# s['foo'] = b'bar'
# list(s)
s['foo'] = b'bar'

# Wrapping mk_dirs_if_missing(G) works
from dol import wrap_kvs

F = wrap_kvs(mk_dirs_if_missing(G))
s = F(ppp)
s['foo'] = b'bar'

# But wrapping G breaks mk_dirs_if_missing
F = mk_dirs_if_missing(wrap_kvs(G))
s = F(ppp)
s['foo'] = b'bar'
# FileNotFoundError: [Errno 2] No such file or directory: ''

┆Issue is synchronized with this Asana task by Unito