support for fp16 training

[baibaidj]

Describe the feature
FP16 training

Motivation
FP16 training facilitates faster training. Apex is recommended.
Using default FP32, the possible batch size is small leading to slower training with possible suboptimal performance.

Related resources
https://github.com/NVIDIA/apex

Additional context
No

[hellock]

FP16 has been supported.

[baibaidj]

I cannot find Fp16OptimizerHook class in the mmseg. Is it included in the newest mmcv?
When I add this in the opitmizer_config, the system prompts that this class was not registered.
Thank you.

[xvjiarui]

Hi @baibaidj
You may use the latest MMCV (1.0.3).

[baibaidj]

Hi @baibaidj
You may use the latest MMCV (1.0.3).

Thanks. It worked.
But, I came across degradation in performance when using FP16 training.
The training log is as follows. Could you please take a look and see what's going wrong?
Thank you again.

With some effort, I finally locate the bug. I did not add self.fp16_enabled = False to the BaseDecodeHead class.
After checking, the auto_fp16 class in mmcv seems to only use the attribute (fp16_enabled), rather than the value to determine if to enable fp16 training.
To be honest, it is on the first sight confusing. It would be clearer by self.fp16_enabled = True.
This fp16_enabled attribute is also added in BaseSegmentor class. I was wondering why it has appeared twice? Thank you.