Chisel4ml and a subtle rounding issue.

[jurevreca12]

Hey,
I am the main author of Chisel4ml (https://github.com/cs-jsi/chisel4ml), a library similar to hls4ml and FINN, but based on the Chisel HCL. Currently, we are taking QKeras models and converting them "manually" down to our own custom representation. We would like to switch to QONNX however, since this would save us a lot of work, and enable support for Brevitas.

I am currently testing qonnx, and one subtle rounding mode issue caught my attention. This issue concerns the quantized_bits operator in QKeras. When testing different configurations I stumbled upon a peculiar difference in rounding mode behavior. This is best shown with example:

def test_quantized_bits_rounding_mode():
    alpha1 = qkeras.quantized_bits(bits=3, integer=2, keep_negative=True, alpha=1)
    alpha111 = qkeras.quantized_bits(bits=3, integer=2, keep_negative=True, alpha=[1, 1, 1])
    alpha_po2 = qkeras.quantized_bits(bits=3, integer=2, keep_negative=True, alpha='auto_po2')
    try:
        assert np.array_equal(alpha1(np.array([2.5, 2.5, 3.5])), alpha111(np.array([2.5, 2.5, 3.5])))
        assert np.array_equal(alpha1(np.array([2.5, 2.5, 3.5])), alpha_po2(np.array([2.5, 2.5, 3.5])))
    finally:
        print(alpha1.scale)
        print(alpha111.scale)
        print(alpha_po2.scale)

The function above will fail on the second assert (for QKeras 0.9.0). However, the scaling factors printed in the finally block will be 1, [1,1,1] and [1,1,1]. The reason is that when using "auto_po2" the rounding mode is actually "round half up". This can be seen on:
https://github.com/google/qkeras/blob/67e7c6b8cbd6befd594f142187ac4b73b35512ac/qkeras/quantizers.py#L570C45-L570C46

v = tf.floor(tf.abs(x) / scale + 0.5)

I've just started to look at the converter for QKeras, and it seems that quantized_bits is always converted to rounding_mode="ROUND", which is a round half to even operation. Thus, this seems to be a discrepancy. Am I missing something here? Anyway, If you are open to it, I'd be glad to open an issue on this, and/or a pull request.

[maltanar]

I imagine @jmduarte or perhaps @jmitrevs would be best placed to help answer this on the QKeras side of things.
For the Quant specification itself, I don't think we have a corresponding rounding mode but feel free to propose one.

[jurevreca12]

Does CEIL rounding mode round up, regardless of fractions? i.e. is CEIL(2.2)==3? If so then yes, this rounding mode is not specified. In any case, I think adding some extra information on the exact behavior of the rounding mode, would improve the specification.
How should I propose this rounding mode? with an issue or a pull request?

[maltanar]

While it may be best to double-check this against what Brevitas does, our implementation for Quant resolves the rounding mode as:

https://github.com/fastmachinelearning/qonnx/blob/main/src/qonnx/custom_op/general/quant.py#L134-L145

def resolve_rounding_mode(mode_string):
    """Resolve the rounding mode string of Quant and Trunc ops
    to the corresponding numpy functions."""
    normalized_mode_string = mode_string.upper()
    if normalized_mode_string == "ROUND":
        return np.round
    elif normalized_mode_string == "CEIL":
        return np.ceil
    elif normalized_mode_string == "FLOOR":
        return np.floor
    else:
        raise ValueError(f"Could not resolve rounding mode called: {normalized_mode_string}")

and in turn, np.ceil operates like this:

>>> a = np.array([-1.7, -1.5, -0.2, 0.2, 1.5, 1.7, 2.0])
>>> np.ceil(a)
array([-1., -1., -0.,  1.,  2.,  2.,  2.])

[maltanar]

Feel free to go straight for a PR, if this will be a new rounding mode.

[maltanar]

On a separate note, I'm a big Chisel fan and did a bunch of hardware generator work with it back in the day. I see you have started your own FPGA component libraries as part of it, here's mine which was originally written for Chisel 2 and ported to Chisel 3 by @erlingj - feel free to take a look and pick things up from there if you find them useful:

https://github.com/maltanar/fpga-tidbits

There's also a DSD'23 paper "FPGA-tidbits: Rapid Prototyping of FPGA Accelerators in Chisel" that goes with it, but I seem to have trouble finding a usable link..

[jurevreca12]

I've actually been to this conference, and read this paper. Funnily, they finally uploaded the papers to IEEEXplore today. Here is the link: https://ieeexplore.ieee.org/document/10456791. I will look at the possibility of integrating, specifically the platform-wrapper functionality seems interesting to me. It would be nice to make it easier to actually deploy to FPGAs.

[jurevreca12]

Discussion continued in #110