Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implementation details about the testing process #5

Open
jiauxan opened this issue Jun 13, 2023 · 7 comments
Open

implementation details about the testing process #5

jiauxan opened this issue Jun 13, 2023 · 7 comments

Comments

@jiauxan
Copy link

jiauxan commented Jun 13, 2023

I ran the code of LCR you provided, pretraining the model on the CUB dataset for 100 epochs and then freezing the model parameters and only optimizing the linear classifier . However, I could only achieve a maximum acc1 around 64.71. I would like to know if there is any step I might have missed or done incorrectly. thanks a lot!

@eafn
Copy link

eafn commented Jun 23, 2023

me too

@GANPerf
Copy link
Owner

GANPerf commented Jun 26, 2023

I ran the code of LCR you provided, pretraining the model on the CUB dataset for 100 epochs and then freezing the model parameters and only optimizing the linear classifier . However, I could only achieve a maximum acc1 around 64.71. I would like to know if there is any step I might have missed or done incorrectly. thanks a lot!

I appreciate your interest. Could you please let me know which checkpoint you have been using? How about the acc1 performance on StanfordCars & Aircraft? To attain the highest acc1 accuracy, it is advisable to consider selecting the checkpoint with the best retrieval performance, rather than relying solely on the last epoch during the pretraining process. For your reference, I have provided the checkpoint at the following link: Checkpoint Link

Repository owner deleted a comment from andayangyang Jun 26, 2023
@jiauxan
Copy link
Author

jiauxan commented Jun 26, 2023

Thank you for your reply! I only test on the CUB dataset and select the checkpoint with the best retrieval performance of the result of main_moco.py(the best retrieval performance gained around 35 epochs and reached acc around 42). Using the checkpoint on the last epoch to gain a linear classifier always attains the result around 62. The parameters I set are the same as you have given in the paper.

@jiauxan
Copy link
Author

jiauxan commented Jun 26, 2023

Another question I wonder regarding training is about loading the pre-trained model parameters you have provided. I modified the pre-trained model instead of the one you have given. However, the size of the fc is mismatched with the LCR. The num-classes for resnet-50 is 1000 instead of 256 you have given. I would like to know whether the parameters of the checkpoint you have provided should be loaded by resnet-50(encoder_q) instead of the LCR, and the error happened during I used moco_linear.py.
image

@GANPerf
Copy link
Owner

GANPerf commented Jun 27, 2023

Another question I wonder regarding training is about loading the pre-trained model parameters you have provided. I modified the pre-trained model instead of the one you have given. However, the size of the fc is mismatched with the LCR. The num-classes for resnet-50 is 1000 instead of 256 you have given. I would like to know whether the parameters of the checkpoint you have provided should be loaded by resnet-50(encoder_q) instead of the LCR, and the error happened during I used moco_linear.py. image

Hi Jiaxuan, I wanted to mention that I believe the LCR (2048->256) modification would be more suitable because the ResNet-50 model has a different structure (2048->1000).

@jiauxan
Copy link
Author

jiauxan commented Jun 27, 2023

Another question I wonder regarding training is about loading the pre-trained model parameters you have provided. I modified the pre-trained model instead of the one you have given. However, the size of the fc is mismatched with the LCR. The num-classes for resnet-50 is 1000 instead of 256 you have given. I would like to know whether the parameters of the checkpoint you have provided should be loaded by resnet-50(encoder_q) instead of the LCR, and the error happened during I used moco_linear.py. image

Hi Jiaxuan, I wanted to mention that I believe the LCR (2048->256) modification would be more suitable because the ResNet-50 model has a different structure (2048->1000).

Hi, I also agree with that. But it seems that the size of fc in the model in the checkpoint you have provided mismatched the model in your code(I mean the LCR model), as if in the pic above. I wonder whether the checkpoint you have provided is gained using main_moco.py.

@shisofia
Copy link

Hello, I have encountered the same problem. May I ask if you have solved this problem and could you tell me your solution? @jiauxan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants