Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unable to reproduce the result on CIFAR-100 with the same settings in the paper #1

Open
Berumotto1 opened this issue Sep 15, 2023 · 2 comments

Comments

@Berumotto1
Copy link

In the experiment of VGG16 on CIFAR100, I set the l=4, init_threshold=4, seed=42, the acc of SNN at T=2 is 0.0100 and the acc of SNN at T=64 is 0.7534(0.7705 in the paper).
In the experiment of ResNet20 on CIFAR100, I set the l=8, init_threshold=4, seed=42, the acc of SNN at T=64 is 0.6890(0.7055 in the paper).
Thank you for providing the code, expecting for your reply.

@putshua
Copy link
Owner

putshua commented Oct 1, 2023

Please refer to the latest version.

@bjourne
Copy link

bjourne commented Oct 1, 2024

Hello!

Fwiw, I'm am able to reproduce your results on CIFAR100 with VGG16 and L=8:

SRC ANN T=1 T=2 T=4 T=8 T=16 T=32 T=64 T=128
Mine 77.15 43.08 52.73 63.49 71.77 75.71 76.87 77.05 77.10
Google Drive 77.41 35.38 52.71 66.04 70.75 73.54 74.43 74.34 74.41
Paper - 44.98 52.46 62.09 70.71 74.83 76.41 76.73 76.73

Mine refers to the model I trained myself, Google Drive to your pretrained model, and Paper to the figures in the paper. I even get slightly better performance for some values of T. Only difference is that I train with LR=0.1 instead of 0.05.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants