PSNR decreased during training

[yingxuanhi]

Excuse me, I modify your DRSformer_arch.py code and try to get better performance(add a convolution branch in TKSA).

I find that the psnr decrease during the training(ex. psnr :24,1236 -> 24,3902 ->26.1262->13.0792 ->13.0792).

If I want to conquer this issue, may I reduce the learning rate to train? If yes, which part of codes I can modify?

If no, what can I do?

I'm sorry to disturb your precious time.

[cschenxiang]

Excuse me, I modify your DRSformer_arch.py code and try to get better performance(add a convolution branch in TKSA).

I find that the psnr decrease during the training(ex. psnr :24,1236 -> 24,3902 ->26.1262->13.0792 ->13.0792).

If I want to conquer this issue, may I reduce the learning rate to train? If yes, which part of codes I can modify?

If no, what can I do?

I'm sorry to disturb your precious time.

You can modify the learning rate here:
https://github.com/cschenxiang/DRSformer/blob/main/Options/Deraining.yml ---line84

But I'm not sure if the problem is caused by learning rate. I suggest checking the code and other details.

[yingxuanhi]

Excuse me, I modify your DRSformer_arch.py code and try to get better performance(add a convolution branch in TKSA).
I find that the psnr decrease during the training(ex. psnr :24,1236 -> 24,3902 ->26.1262->13.0792 ->13.0792).
If I want to conquer this issue, may I reduce the learning rate to train? If yes, which part of codes I can modify?
If no, what can I do?
I'm sorry to disturb your precious time.

You can modify the learning rate here: https://github.com/cschenxiang/DRSformer/blob/main/Options/Deraining.yml ---line84

But I'm not sure if the problem is caused by learning rate. I suggest checking the code and other details.

Hello! I have reduced the learning rate from the original 0.0003 to 0.0002 and successfully completed the training.

However, I noticed something:

I had previously trained the original DRSformer you provided (learning rate=0.0003, Dataset: Rain200H).

When comparing it with the DRSformer model I have now trained (added a convolution branch in TKSA, learning rate=0.0002, Dataset: Rain200H), I found that the modified model has a lower PSNR (approximately decreased by 0.08).

Could this be due to the reduction in learning rate?

Additionally, I have attached the code for the modifications I made to TKSA; no other parts have been changed.
Top-K Sparse Attention (TKSA).txt

Thanks for your help!!!

[JiangYun77]

When I use the TKSA ,I meet same issue that PSNR decrease during the training(29.2 to 16.2),have you solve this issue by reducing learning rate? Looking forward your reply.

[yingxuanhi]

When I use the TKSA ,I meet same issue that PSNR decrease during the training(29.2 to 16.2),have you solve this issue by reducing learning rate? Looking forward your reply.

yeah, I have set the learning rate from 310^-4 to 2.510-4 and it finished the training . I hope it can help you!

[zhaofengnain]

@yingxuanhi How long does it take to complete the reproduction code?