Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent image loading in torchvision before and after 0.8.0 version #3482

Closed
k-sobolev opened this issue Mar 1, 2021 · 3 comments
Closed

Comments

@k-sobolev
Copy link

k-sobolev commented Mar 1, 2021

🐛 Bug

The standard ILSVRC2012 validation set dataloader returns different data depending on torchvision version. The problem starts from version 0.8.0. Images loaded by torchvision=<0.7.0 and torchvision>=0.8.0 are shifted by one pixel horizontally relative to each other. This makes model evaluations inconsistent. For example, this is the source of issue when validation scores from code and validation scores are reported in the documentation are different.

To Reproduce

Steps to reproduce the behavior:

  1. Run the code below under two versions of torchvision (<=0.7.0 and >=0.8.0).
import torch
from torchvision import datasets, transforms

# put your dataset path here
dataset_path ='path_to_ILSVRC2012'

val_loader = torch.utils.data.DataLoader(
            datasets.ImageFolder(dataset_path + '/val',
                                 transform=transforms.Compose([
                                     transforms.Resize(256),
                                     transforms.CenterCrop(224),
                                     transforms.ToTensor(),
                                     transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                                          std=[0.229, 0.224, 0.225]),
                                 ])),
            batch_size=1, shuffle=False, num_workers=1, pin_memory=True)

data, target = next(iter(val_loader))
print(data[0])
  1. Check printed values.

Expected behavior

Printed tensors should be the same, but they are shifted by one pixel horizontally relative to each other.

Environment

Here are presented 2 environment options: for torchvision==0.7.0 and for torchvision==0.8.0

PyTorch version: 1.6.0 / PyTorch version: 1.7.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: CentOS Linux 7 (Core) (x86_64)
GCC version: (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Clang version: Could not collect
CMake version: Could not collect

Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: GeForce GTX 1080 Ti
Nvidia driver version: 460.32.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] efficientnet-pytorch==0.6.3
[pip3] numpy==1.16.5
[pip3] pytorch-transformers==1.2.0
[pip3] torch==1.6.0/1.7.0
[pip3] torchfile==0.1.0
[pip3] torchvision==0.7.0/0.8.0

  • PyTorch / torchvision Version: 1.6.0/07.0 // 1.7.0/0.8.0
  • OS (e.g., Linux):
  • How you installed PyTorch / torchvision (conda, pip, source): pip
  • Build command you used (if compiling from source):
  • Python version: 3.7 (64-bit runtime)
  • CUDA/cuDNN version: 10.2
  • GPU models and configuration: GeForce GTX 1080 Ti
  • Any other relevant information:

Additional context

cc @vfdev-5

@fmassa
Copy link
Member

fmassa commented Mar 2, 2021

Thanks for opening this issue! There was one change made to center_crop I believe to allow for torchscript compatibility (there was a bug in torchscript), but since then we have fixed it in #3118

Can you check with a torchvision nightly to see if this problem persists, as I believe this might already have been fixed?

@k-sobolev
Copy link
Author

Thanks for the prompt reply! I have checked loaded images with a torchvision nightly and confirm that output pixels are the same as when I use torchvision<=0.7.0. Thus, the problem is solved.

I suppose that #3152 is also fixed since now ILSVRC2012 evaluation metrics are consistent with documentation.

@fmassa
Copy link
Member

fmassa commented Mar 3, 2021

Thanks for confirming that this has now been fixed!

I believe we have fixed the behavior of center crop before @datumbox re-run the evaluations, so I believe the numbers should be compatible with what we should get now.

I'm closing this issue as I believe this has been fixed already, let us know if you find anything else

@fmassa fmassa closed this as completed Mar 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants