Add API for quantization-aware training in dygraph mode

[wanghaoshuang]

PR types

Others

PR changes

APIs

Describe

Add API for quantization-aware training in dygraph mode

1. Expose an API named paddle.quantization.QAT
2. Define a wrapper class to insert quanters into model for QAT
3. Add QuantedConv2D and QuantedLinear
4. Add some functions in QuantConfig for QAT
5. Add unittest for QAT

如下图所示，本PR主要新增三个模块：Wrapper、QuantedLayers（包括QuantedConv2D、QquantedLinear）、QAT

Wrapper

装饰器，将任意Layer和Quanter/Observer包装成一个Layer。

QuantedLayers

与Wrapper类似，装饰器类，用于在Layer前插入Observer/Qaunter. Wrapper针对无权重Layer，QuantedLayers针对带权重的特定Layer，比如QuantedConv2D.

QAT

类图

时序图

示例

以下示例来自paddle/tests/quantization/test_qat.py
原始模型

Model(
  (features): Sequential(
    (0): Conv2D(3, 6, kernel_size=[3, 3], padding=1, data_format=NCHW)
    (1): ReLU()
    (2): MaxPool2D(kernel_size=2, stride=2, padding=0)
    (3): Conv2D(6, 16, kernel_size=[5, 5], data_format=NCHW)
    (4): ReLU()
    (5): MaxPool2D(kernel_size=2, stride=2, padding=0)
  )
  (fc): Sequential(
    (0): Linear(in_features=576, out_features=120, dtype=float32)
    (1): Linear(in_features=120, out_features=84, dtype=float32)
    (2): Linear(in_features=84, out_features=10, dtype=float32)
  )
)

调用qat.quantize(mdoel)之后的模型：

Model(
  (features): Sequential(
    (0): QuantedConv2D(
      (weight_quanter): FakeQuanterWithAbsMaxObserverLayer()
      (activation_quanter): FakeQuanterWithAbsMaxObserverLayer()
    )
    (1): ObserveWrapper(
      (_observer): FakeQuanterWithAbsMaxObserverLayer()
      (_observed): ReLU()
    )
    (2): ObserveWrapper(
      (_observer): FakeQuanterWithAbsMaxObserverLayer()
      (_observed): MaxPool2D(kernel_size=2, stride=2, padding=0)
    )
    (3): QuantedConv2D(
      (weight_quanter): FakeQuanterWithAbsMaxObserverLayer()
      (activation_quanter): FakeQuanterWithAbsMaxObserverLayer()
    )
    (4): ObserveWrapper(
      (_observer): FakeQuanterWithAbsMaxObserverLayer()
      (_observed): ReLU()
    )
    (5): ObserveWrapper(
      (_observer): FakeQuanterWithAbsMaxObserverLayer()
      (_observed): MaxPool2D(kernel_size=2, stride=2, padding=0)
    )
  )
  (fc): Sequential(
    (0): QuantedLinear(
      (weight_quanter): FakeQuanterWithAbsMaxObserverLayer()
      (activation_quanter): FakeQuanterWithAbsMaxObserverLayer()
    )
    (1): QuantedLinear(
      (weight_quanter): FakeQuanterWithAbsMaxObserverLayer()
      (activation_quanter): FakeQuanterWithAbsMaxObserverLayer()
    )
    (2): QuantedLinear(
      (weight_quanter): FakeQuanterWithAbsMaxObserverLayer()
      (activation_quanter): FakeQuanterWithAbsMaxObserverLayer()
    )
  )
)

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[yghstill]

LGTM

[risemeup1]

LGTM

[zhangbo9674]

LGTM

[chenwhql]

LGTM for setup.py.in