Replicating the results

[Ellenisawake]

Does anybody succeded in getting the results as the paper claimed?
So far by running the VisDA experiment with ResNet101 using default settings I get this:

Mean class accuracy 62.54% (v.s 71.9 reported in paper)

[Ellenisawake]

Updated results on VisDA17 classification (70.11%):

Difference is that previous result was obtained running the code under PyTorch 0.4, this one is using PyTorch 0.3. And running evaluation in either 0.3 or 0.4 gives the same accuracy. Suspect that behavior of some function(s) changed dramastically from 0.3 to 0.4.
Thanks to the author for pointing out that:)

[guantinglin]

Hi, @Ellenisawake
do you have any idea what makes this performance difference betewwn pytorch 0.3 and 0.4?

[jiaojiao-Chen]

Hi, @Ellenisawake
do you use the code the author applied with VisDa classification directly? I couldn't understand the variable of 'val' in the test function, can you tell me your understand?

[Ellenisawake]

Hi, @Ellenisawake
do you have any idea what makes this performance difference betewwn pytorch 0.3 and 0.4?

I didn't check in detail. Suggest that you just use 0.3 for running it or adding anything on top. As PyTorch has already evolved to v1.0, personally don't think there's much significance in figuring out version conflicts between 0.3 & 0.4.

[Ellenisawake]

Hi, @Ellenisawake
do you use the code the author applied with VisDa classification directly? I couldn't understand the variable of 'val' in the test function, can you tell me your understand?

I think 'val' is just a flag used to distinguish a test run and a validation run within training.
'''
if not val and value > 60:
torch.save(F1.state_dict(), save_path+''+args.resnet+''+str(value)+'_'+'F1.pth')
'''
Here the author only wants the model saved when in validation and accuracy >60.