Python path.py usage ?

[ClementPinard]

Hello, thanks for this awesome work !

Have you considered using path.py to wrapp os.path function ?
https://pypi.python.org/pypi/path.py

As code compacity and readability seems to be a major concern, it could be nice to use it. It's avalaible for python 2.7 and 3.5, os independent, and can be downloaded via pip.

Here is an example for ImageFolder dataset :

def make_dataset(dir, class_to_idx):
    images = []
    for target in os.listdir(dir):
        d = os.path.join(dir, target)
        if not os.path.isdir(d):
            continue

        for filename in os.listdir(d):
            if is_image_file(filename):
                path = '{0}/{1}'.format(target, filename)
                item = (path, class_to_idx[target])
                images.append(item)

    return images

and now with path.py wrapper :

from path import Path
def make_dataset(dir, class_to_idx):
    dir = Path(dir)
    images = []
    for d in dir.dirs():
        target = str(d.basename())

        for path in d.files():
            if path.ext in IMG_EXTENSIONS:
                item = (path, class_to_idx[target])
                images.append(item)

    return images

Maybe you considered it but still decided not to use it ? In that case why ? (like not the accusing 'why', that is a genuine question as i am not very experienced for deploying big python frameworks)

[apaszke]

No, we haven't considered it, but to be honest it doesn't seem to improve the example code substantially, and because of this I don't think it's worth adding to dependencies. As far as I know os.path is OS independent too.

The less dependencies, the better.

[ClementPinard]

path.py is actually a wrapper for os.path, so both are OS independent.

But as you can see e.g. here or (ironically in the same snippet I quoted earlier) here , everyone is really inclined to use the / character to describe a folder instead of os.path.join which will cause problems for windows users (who are waiting for pytorch ! :p ).
So overloading the / operator into the os.path.joinmethod is a really elegant way of dealing things for me.

Also path.py is already contained in anaconda so a lot of people already has it

[apaszke]

First thing, that's an issue with tnt, not vision and can easily be solved by replacing it with os.path.join. It's as simple as that, and adding path.py as dependency won't make anyone stop using / and string operations. If they don't care to use os.path they won't care to use path.py either. It doesn't hurt to use os.path, and I don't find it less readable than an OOP interface.

I really don't like adding dependencies that are not needed. It's fine to do it for personal projects, but not at all for libraries that are installed on so many computers. Dependencies are additional problems you have to worry about. If they break, your code will be broken too. If they get deleted (see famous leftPad issue from npm) you have a problem and you have to rewrite your code and remove them asap. I'd rather stick to a simple and safe way with os.