Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GarNet and GarNetStack in config.py #344

Merged
merged 11 commits into from
Dec 27, 2021

Conversation

yiiyama
Copy link
Contributor

@yiiyama yiiyama commented May 20, 2021

I was asked by a Ph.D. student at Imperial College London why hls4ml config doesn't work with GarNet. Found out that graph_layers support isn't there in config.py - which is of course the case because I didn't do it (hls4ml config wasn't really a thing when I was working on GarNet).
This PR adds support to GarNet and GarNetStack in config.py and gives some default precision values for their parameters. These default values are however most likely not optimal, because in config.py we don't deal with tensor shapes and therefore we don't know the number of vertices that goes into the layer, which is a critical input when computing optimal precisions.
So the lines that gets generated with hls4ml config are there mostly for documentation purpose to say "hey these are the configurable parameters", and users are actually better off commenting them out when actually doing a conversion (proper default precisions will be calculated using tensor shapes), unless they really have specific precision values they want to use. Is this something to be documented somewhere? (add a comment line in the generated config?)

@jmduarte jmduarte self-requested a review May 27, 2021 17:23
Copy link
Member

@jmduarte jmduarte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yiiyama Do you have an example of a realistic HLSConfig you would use?

Here's my naive attempt at a conversion script, which fails due to #365.

Also, as noted in #365, I'm wondering if we can solve this so HLSModel.compile() and predict() work. I think @vloncar said he had an idea for a workaround in case we can't include hls_math.h due to copyright issues.

I also wonder if output_activation='linear' can also be allowed here in addition to None (Naively, that's what I tried at first when I saw the error message).

@yiiyama
Copy link
Contributor Author

yiiyama commented Aug 13, 2021

@jmduarte

Do you have an example of a realistic HLSConfig you would use?

This is the configuration file used for the HLS GarNet paper. Basically you are better off not setting too many options. In particular, as I write in the PR intro above, the config dict returned by config_from_keras_model for GarNet is actually suboptimal - it just creates entries for all available tuning parameters and sets default values for them, while the GarNet class in hls_layers.py can actually compute the optimal precision for the weight parameters given an actual model.

Also, as noted in #365, I'm wondering if we can solve this so HLSModel.compile() and predict() work. I think @vloncar said he had an idea for a workaround in case we can't include hls_math.h due to copyright issues.

This and this are my attempts at solving the issue. @vloncar what do you think?

I also wonder if output_activation='linear' can also be allowed here in addition to None (Naively, that's what I tried at first when I saw the error message).

That's a good point; 'linear' is allowed now.

I tweaked your gist a bit here. The only differences are

  • GarNet parameters (8, 8, 16 instead of 32, 32, 32 which you had - I think that's too large for synthesis)
  • x passed to hls_model.predict() needed to be all doubles because of how predict is written
  • Variable precisions set super-high so that we can actually see that keras and hls4ml predictions are synched

With the current HEAD of this PR branch, the script does execute and ends up with synched-ish result.

@jmduarte
Copy link
Member

Re: #365 (comment) I wanted to check how our python API result differs from csim/cosim.

I ran this updated gist: https://gist.github.com/jmduarte/c1eedff75fc941e755ae82f7d626093f

Partial good news is python API and csim seem to be similar up to some truncation effects? Bad news is cosim currently fails.

  • hls_model.predict() output:
x hls4ml predict:
 [-0.05145645 -0.00219154 -0.00580788 -0.09418297 -0.01958847 -0.01342964 -0.01376438 -0.06519222]
  • csim output:
Quantized predictions
-0.0514565 -0.00219154 -0.00580788 -0.094183 -0.0195885 -0.0134296 -0.0137644 -0.0651922 
  • cosim error:
ERROR: [COSIM 212-346] C test bench instrumentation failed: Cannot determine C object for RTL port input_1_0_V.
ERROR: [COSIM 212-5] *** C/RTL co-simulation file generation failed. ***
ERROR: [COSIM 212-4] *** C/RTL co-simulation finished: FAIL ***

@jmduarte
Copy link
Member

jmduarte commented Aug 15, 2021

Also, now that testing is merged, I wonder if we can add a test similar to the gist?

Now done in 3adc72c

Copy link
Member

@jmduarte jmduarte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good now! Batched inputs should also work with #414

test/pytest/garnet.py Outdated Show resolved Hide resolved
@jmduarte jmduarte merged commit 4c5d89d into fastmachinelearning:master Dec 27, 2021
calad0i pushed a commit to calad0i/hls4ml that referenced this pull request Jul 1, 2023
…garnet

GarNet and GarNetStack in config.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants