lstat succeeds that should fail (part of pattern with multiple other operations)

[agroce]

import shutil
import pyfakefs.fake_filesystem
import os

ROOT_PATH = "/Volumes/ramdisk/test"

fs = pyfakefs.fake_filesystem.FakeFilesystem()
fs.CreateDirectory('/Volumes')
fs.CreateDirectory('/Volumes/ramdisk')
fs.CreateDirectory('/Volumes/ramdisk/test')
os0 = pyfakefs.fake_filesystem.FakeOsModule(fs)
opener0 = pyfakefs.fake_filesystem.FakeFileOpen(fs)

if False:
    os0 = os
    shutil.rmtree(ROOT_PATH)
    os.mkdir(ROOT_PATH)
    opener0 = open

component0 = "alpha" 
path0 = ROOT_PATH 
path1 = ROOT_PATH 
path0 += os.path.sep + component0 
component0 = "beta" 
path1 += os.path.sep + component0 
component0 = "" 
os0.symlink(path1,path0) 
path0 += os.path.sep + component0 
stat0 = os0.lstat(path0) 

OS raises OSError: [Errno 2] No such file or directory: '/Volumes/ramdisk/test/alpha/'

[agroce]

Substitute remove for lstat and you get basically the same issue; looking for others that produce the same problem now.

[agroce]

Ok, full list of essentially equivalent issues:

================================================================================
FAILED WITH os0.mkdir(path0) 
(<type 'exceptions.AssertionError'>, AssertionError(" (<type 'exceptions.OSError'>) == (<type 'NoneType'>) ",), <traceback object at 0x104a62098>)
================================================================================
FAILED WITH os0.makedirs(path0) 
(<type 'exceptions.AssertionError'>, AssertionError(" (<type 'exceptions.OSError'>) == (<type 'NoneType'>) ",), <traceback object at 0x105c3fef0>)
================================================================================
FAILED WITH os0.remove(path0) 
(<type 'exceptions.AssertionError'>, AssertionError(" (<type 'NoneType'>) == (<type 'exceptions.OSError'>) ",), <traceback object at 0x105a6f200>)
================================================================================
FAILED WITH stat0 = os0.lstat(path0) 
(<type 'exceptions.AssertionError'>, AssertionError(" (<type 'NoneType'>) == (<type 'exceptions.OSError'>) ",), <traceback object at 0x105b08998>)
================================================================================
FAILED WITH result = os0.readlink(path0) 
(<type 'exceptions.AssertionError'>, AssertionError(" (<type 'NoneType'>) == (<type 'exceptions.OSError'>) ",), <traceback object at 0x104e9cd40>)
================================================================================
FAILED WITH os0.rename(path0,path1) 
(<type 'exceptions.AssertionError'>, AssertionError(" (<type 'NoneType'>) == (<type 'exceptions.OSError'>) ",), <traceback object at 0x105a96758>)
================================================================================
FAILED WITH result = os0.path.lexists(path0) 
(<type 'exceptions.AssertionError'>, AssertionError(' (True) == (False) ',), <traceback object at 0x105a32518>)
================================================================================
FAILED WITH result = os0.path.islink(path0) 
(<type 'exceptions.AssertionError'>, AssertionError(' (True) == (False) ',), <traceback object at 0x1059b02d8>)

[agroce]

(produced using this script:

import sut
sut = sut.sut()
t = sut.loadTest("lstatnorm.test")[:-1]

cdone = {}

for a in sut.actions():
    if sut.actionClass(a) in cdone:
        continue
    sut.replay(t)
    if a[1]():
        ok = sut.safely(a)
        if not ok:
            print "=" * 80
            print "FAILED WITH", sut.prettyName(a[0])
            print sut.failure()
            cdone[sut.actionClass(a)] = True

I think fixing this would actually remove a large portion of the failing tests I have, and it's a clear repeated pattern, so this might be a good one to tackle.

[agroce]

(you can get a more verbose and full of extra stuff similar result doing:

> tstl_generalize lstatnorm.test
...
tat0 = os0.lstat(path0)                                                             # STEP 10
#  or os0.mkdir(path0) 
#  or os0.makedirs(path0) 
#  or os0.remove(path0) 
#  or stat1 = os0.lstat(path0) 
#  or stat2 = os0.lstat(path0) 
#  or stat3 = os0.lstat(path0) 
#  or stat4 = os0.lstat(path0) 
#  or result = os0.readlink(path0) 
#  or os0.rename(path0,path1) 
#  or result = os0.path.lexists(path0) 
#  or result = os0.path.islink(path0) 
GENERALIZED IN 52.8304390907 SECONDS

at the command line; FYI if you ever start running TSTL tests much yourself. Generalize will often show if a bug is part of a general pattern.

[mrbean-bremen]

I think fixing this would actually remove a large portion of the failing tests I have

Ok, I may have a look tomorrow but can't promise anything.

[agroce]

No rush, I'll just run the regression directory when you fix it, but it seems to cover a lot of the current Mac OS issues (how important it is, involving both symlinks and extra separators, I leave to you to judge; probably unimportant except for shutting up the tests).

[mrbean-bremen]

Ok, here is the behavior for these cases:

command	Real OS	Fake FS
lstat Linux	OSError 2	OSError 2 ✅
lstat MacOS	OSError 2	no error ❌
lstat Windows	OSError 22	OSError 2 ❌
mkdir+makedirs Linux	OSError 17	OSError 17 ✅
mkdir+makedirs MacOS	no error	OSError 17 ❌
mkdir+makedirs Windows	OSError 17	OSError 17 ✅
remove+rename Linux	OSError 20	no error ❌
remove+rename MacOS	OSError 2	no error ❌
remove+rename Windows	OSError 22	no error ❌
readlink Linux	OSError 2	OSError 40 ❌
readlink MacOS	OSError 2	no error ❌
readlink Windows	OSError 22	OSError 22 ✅
lexists+islink Linux	False	False ✅
lexists+islink MacOS	False	True ❌
lexists+islink Windows	False	False ✅

Looks like a bit of work...

[mrbean-bremen]

This is a lot of special handling again for the different functionality, so I doubt that fixing this will help with many others,

[agroce]

We can just drop testing extra path separators, of course, and these will all go away.

[agroce]

I have no problem with that, though it means any interesting weirdness with path separators is going to be missed. I can't figure a good way to not always produce tons of these, but catch more "interesting" stuff (and I'm not sure what that would even be)

[mrbean-bremen]

While this took a while to fix because of the different cases, it is still faster than fixing each of these separately, so the generalization really helped here.