OMNIGLOT Dataset #46
OMNIGLOT Dataset #46
Conversation
torchvision/datasets/omniglot.py
Outdated
|
|
||
| def __getitem__(self, index): | ||
| filename=self.all_items[index][0] | ||
| path=self.all_items[index][2]+"/"+filename |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
|
Hi, The omniglot dataset has been simplified and now returns pairs of (filename,target). Pairs of (image,target) can be obtained by using the transforms.FilenameToPILImage transformer. A test script has been also proposed |
|
Any chance this will be merged into the repository? |
|
Sorry for the delay on this, I will try to review in the coming days. As some things have changed in the last months, this will likely need some minor modifications @ludc are you still interested in working on this? |
|
@ludc according to https://github.com/brendenlake/omniglot there are two splits: |
| @@ -27,6 +27,13 @@ def __call__(self, img): | |||
| img = t(img) | |||
| return img | |||
|
|
|||
| class FilenameToPILImage(object): | |||
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| print("Download finished.") | ||
|
|
||
| def find_classes(root_dir): | ||
| retour=[] |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| return | ||
|
|
||
| # download files | ||
| try: |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
|
|
||
| for url in self.urls: | ||
| print('== Downloading ' + url) | ||
| data = urllib.request.urlopen(url) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| r=root.split('/') | ||
| lr=len(r) | ||
| retour.append((f,r[lr-2]+"/"+r[lr-1],root)) | ||
| print("== Found %d items "%len(retour)) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| if (f.endswith("png")): | ||
| r=root.split('/') | ||
| lr=len(r) | ||
| retour.append((f,r[lr-2]+"/"+r[lr-1],root)) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| def index_classes(items): | ||
| idx={} | ||
| for i in items: | ||
| if (not i[1] in idx): |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| - target_transform: how to transform the target | ||
| - download: need to download the dataset | ||
| ''' | ||
| def __init__(self, root, transform=None, target_transform=None, download=False): |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| def __len__(self): | ||
| return len(self.all_items) | ||
|
|
||
| def _check_exists(self): |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
| raise RuntimeError('Dataset not found.' | ||
| + ' You can use download=True to download it') | ||
|
|
||
| self.all_items=find_classes(os.path.join(self.root, self.processed_folder)) |
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
|
Closed in favour of #323 |
The OMNIGLOT class can be used to load OMNIGLOT. Note that there is no clear train/test in OMNIGLOT so the class returns the whole dataset. Automatic downloading is included