Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature (quantizer): Adding weight normalization-based integer quantization #559

Merged
merged 16 commits into from
Mar 24, 2023

Conversation

i-colbert
Copy link
Collaborator

@i-colbert i-colbert commented Mar 20, 2023

Adding experimental narrow per-channel weight normalization-based signed integer quantizer based on Quantized Neural Networks for Low-Precision Accumulation with Guaranteed Overflow Avoidance, by I. Colbert, A. Pappalardo, and J. Petri-Koenig, with support for both L1 and L2 weight normalization.

Using the decoupled rescaling integer quantization arithmetic where the weight normalization statistics calculation (d_w) and norm vector parameterization (g) are combined with the scaling factor to become the pre-clipping scaling factor (i.e., pre_scale) and the conventional scaling factor (s) is the post-clipping scaling factor (i.e., post_scale). For further details on the arithmetic, see ParameterPreScalingWeightNorm. For further details on the weight normalization-based quantization technique, see the referenced paper.

@i-colbert i-colbert marked this pull request as ready for review March 21, 2023 22:07
@i-colbert i-colbert changed the title Feature (quantizer): Adding WeightNormIntQuant Feature (quantizer): Adding weight normalization-based integer quantization Mar 22, 2023
@volcacius volcacius self-requested a review March 23, 2023 16:03
@volcacius volcacius merged commit 735b183 into Xilinx:dev Mar 24, 2023
@i-colbert i-colbert deleted the icolbert/wniq branch March 24, 2023 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants