-
Notifications
You must be signed in to change notification settings - Fork 6.8k
Removing the re-size for validation data, which breaking the validation accuracy of CIFAR training #12362
Conversation
…on accuracy of CIFAR training
@szha may I have your comments, I use this example for the regression tracking of mxnet with mkldnn backend, I'll be helpful if this issue can be addressed. |
Does it affect the imagenet validation perf? |
@zhreshold Thanks for the feedback, I use the pre-trained ResNet-50 model and validation-set of ImageNet-1k, and the accuracy with/w.o my changes are same: see test result: Without my changes: With my changes: The imagenet validation perf and accuracy are same. But without my changes, the validation accuracy trends of CIFAR10+ResNet50/VGG16 are as below, obviously it is not expected. |
Seems this regression also goes into 1.3.0rc0 realese. |
@zhreshold sorry for pushing, may I have you comments or approval? Thanks. |
@juliusshufan Thanks for the result, it's fair enough to go through. Again, thanks for the contribution! |
* Revert "Removing the re-size for validation data, which breaking the validation accuracy of CIFAR training (#12362)" This reverts commit ceabcaa. * Revert "[MXNET-580] Add SN-GAN example (#12419)" This reverts commit 46a5cee. * Revert "Remove regression checks for website links (#12507)" This reverts commit 619bc3e. * Revert "Revert "Fix flaky test: test_mkldnn.test_activation #12377 (#12418)" (#12516)" This reverts commit 7ea0533. * Revert "further bump up tolerance for sparse dot (#12527)" This reverts commit 90599e1. * Revert "Fix broken URLs (#12508)" This reverts commit 3d83c89. * Revert "Temporarily disable flaky tests (#12520)" This reverts commit 35ca13c. * Revert "Add support for more req patterns for bilinear sampler backward (#12386)" This reverts commit 4ee866f. * Revert "Change the way NDArrayIter handle the last batch (#12285)" This reverts commit 597a637.
…on accuracy of CIFAR training (apache#12362)
* Revert "Removing the re-size for validation data, which breaking the validation accuracy of CIFAR training (apache#12362)" This reverts commit ceabcaa. * Revert "[MXNET-580] Add SN-GAN example (apache#12419)" This reverts commit 46a5cee. * Revert "Remove regression checks for website links (apache#12507)" This reverts commit 619bc3e. * Revert "Revert "Fix flaky test: test_mkldnn.test_activation apache#12377 (apache#12418)" (apache#12516)" This reverts commit 7ea0533. * Revert "further bump up tolerance for sparse dot (apache#12527)" This reverts commit 90599e1. * Revert "Fix broken URLs (apache#12508)" This reverts commit 3d83c89. * Revert "Temporarily disable flaky tests (apache#12520)" This reverts commit 35ca13c. * Revert "Add support for more req patterns for bilinear sampler backward (apache#12386)" This reverts commit 4ee866f. * Revert "Change the way NDArrayIter handle the last batch (apache#12285)" This reverts commit 597a637.
Description
The dataloader for example/image-classification resizes the imagesize of validation dataset to 256 by default, and which breaks the validation accuracy of CIFAR10, this PR trying to fix it.
Below firgure is the validation accuracy trends without this fix (model is ResNet-50).
Checklist
Changes
example/image-classification/common/data.py
Comments