Vitis HLS backend #629
Vitis HLS backend #629
Conversation
|
Thanks for this PR. It will be very useful for me. When I tried synthesizing the code that has many 1D CNNs with Vitis, I get the following error. I will try commenting out the pipeline pragma, but this is shared with Vivado. |
|
Some warnings I get: Sadly, I still seem to have the line buffer 1d CNN implementation seem to take very long to synthesize (which is why I was using encoded with Vivado_HLS). |
|
I fixed the warnings. The current CNN implementations for io_parallel are going to be replaced with #600 so I did't test them thoroughly. The warning in dense_stream was due to me forgetting to commit the change 🤦♂️ |
|
I would say this is ready for review/merge. Far from complete as it is missing the resource strategy and some warnings about the interfaces still need tweaks, but it mostly works. I tested this on a range of individual layers and toy models. I'm attaching the script used to build the models and run synthesis ( (note that you'll get your results faster if you manually run the build steps in parallel 😉) I'm also including the actual results. Overall, I see slight differences, like the slightly lower usage of LUTs, a cycle spared here & there. Always in favor of Vitis. The failing |
|
Are there still issues with the resource implementation? I am testing this on a 1D CNN, and the compiler seems to fail when I use latency. This particular has a with of 11 with 32 channels, and 64 filters of width 9. |
|
Using the resource implementation, I have successfully used this backend for a 1D CNN Vitis workflow. |
| super().__init__() | ||
|
|
||
| def write_nnet_utils_overrides(self, model): | ||
| ################### |
There was a problem hiding this comment.
The comment should be made clearer (and changed to a docstring)
There was a problem hiding this comment.
Really? Pretty sure it is obvious to you and everyone who looked what this does. People who can't figure this out shouldn't be looking at the codebase in the first place 😄. Jokes aside, like mentioned in the other comment, docstrings will be addressed in a separate PR.
|
The FINN people mentioned the default pipeline time changing. Are there any issues related to that? |
|
|
||
|
|
||
| class ValidateConvImplementation(OptimizerPass): | ||
|
|
There was a problem hiding this comment.
Would prefer docstrings in new classes that are added describing their purpose.
There was a problem hiding this comment.
I'm planning a new PR that adds docstrings to most of the remaining codebase, not just Vitis parts.
| from hls4ml.report import parse_vivado_report | ||
|
|
||
|
|
||
| class VitisBackend(VivadoBackend): |
There was a problem hiding this comment.
I have some concerns inheriting from the VivadoBackend for two reasons. One is that it seems to violate the "is a" requirement for inheritance. Maybe the common features can be factored out? The other is more long-term. In the future when Vitis HLS becomes our primary backend, do we still want to inherit from VivadoBackend?
There was a problem hiding this comment.
I would agree that having a parent XilinxBackend from which VivadoBackend and VitisBackend are derived would be more elegant, but if we plan on deprecating and ultimately phasing out one and switching to the other I think this is fine. The clean solution would require another level in the hierarchy of backend inheritance, which may confuse the developers of flows. Once we are happy with the state of Vitis (i.e., when we gather more feedback from users, which seems very positive so far), we can move stuff from VivadoBackend to VitisBackend and make the Vivado the "override" of Vitis.
jmitrevs
dismissed
their stale review
We discussed at the hls4ml meeting that we would prefer discussing this PR in an hls4ml meeting first.
|
Since 94dbe80 there is support for resource strategy in Dense/Conv layers. The one corner case (when |
|
What about the interplay with |
|
I get the following synthesis error with the Vitis backend. |
|
When working with CNNs I get lots of warnings of the type: inside a dataflow section. The warning seems to be because the weights and biases are global variables, not local variables. As a test, I put the weights inside the function body, right after the |
There was a problem hiding this comment.
I think this line will need to be updated for Vitis (at least 2022.2):
hls4ml/hls4ml/templates/vivado/build_prj.tcl
Line 241 in 8b27ba6
It appears we're supposed to use the xsct command line tool now instead of invoking vitis directly, but the syntax seems to have changed and just doing xsct vivado_synth.tcl doesn't work
|
Hello All, is this pull request still under review? I would be interested in using Vitis_HLS in combination with hls4ml and also willing to do some testing if required. Please let me know! Best, |
|
Yes, it is still under consideration for merging, hopefully we'll decide on Friday whether to include it in this release cycle (as an experimental feature) or postpone it to the next release. In fact, I'm expanding the test suite to include Vitis right now. Expect some commits later today that will rebase this to current main branch. Feel free to test it in any way you need it. We would be interested to get feedback on QoR differences between Vitis and Vivado, especially anything that works worse on Vitis. So far we're aware of issues with resource strategy and @jmitrevs had more success with relaxed clock uncertainty. The reduce operation (used by e.g., pooling layers) also uses more resources in some cases. |
|
Right, I would be quite keen on these changes once they become available. currently im setting up a test setup with this Vitis_port branch in this pull request. I can definitely provide some testing on the Vitis end. However our I currently don't have Vivado_HLS setup so comparisons between the two I cannot provide I think. You mentioned later today some commits to be rebased in the current main branch. Which main branch is this(of which fork/repo)? Could you point me to it? Best, |
|
This is the latest Vitis branch, I meant I will resolve current merge conflicts so it can be merged into the main branch of hls4ml and these commits will appear here. |
|
Ahh great news, thanks very much. I'll keep an eye on it then. If you need any specific testing feel free to let me know. |
|
Hi, I was trying to build my model but its taking quite a long time. Do you perhaps have any small testing models that i can try to build, which you have verifier work with Vivado 2022.1? (vitis_hls) |
* Add quantized sigmoid, fix quantized tanh for QKeras (fastmachinelearning#569) * snapshot of beginnings * make version that works for Vivado, error for Quartus * Change order of precision from quantizer * add hard sigmoid and tanh * fix setting of slope and shift type * revert config parsing--seems a little strange but works * fix hard_sigmoid and hard_tanh for streaming * update pytest for quantized tanh and sigmoid * remove inadvertently included matoplotlib * add special case when W == min_width. * fix merge of main * Go back to having AP_TRN and AP_WRP as defaults * handle case when use_real_tanh is not defined * make the activations use AP_RND_CONV (and AP_SAT) by default * remove use of use_real_tanh in test since not always supported * fix incorrect default types for Keras (not QKeras) hard_sigmoid * Mostly fix up things for Quartus * get rid of intermediate cast * fix an i++ compilation issue * Quartus seems to not like ac_fixed<1,0,false>, so make 2 bits. * fix activation quantizer * make sat, round defeult activation parameters, don't need to set again * Make the slope and shift not be configurable for HardActivation * some pre-commit fixes * pre-commint //hls to // hls fixes * update CI version * fixes for parsing errors from pre-commits * remove qactivation from list of activation_layers * print_vivado_report function for nicer reports (fastmachinelearning#730) * print_vivado_report function for fancier reports * Fancy reports (#51) * fix uram divide by 0 * add test * fix parsing of vsynth in 2020.1; add test * Update test_report.py * exclude pregenerated reports --------- Co-authored-by: Javier Duarte <[email protected]> --------- Co-authored-by: Jovan Mitrevski <[email protected]> Co-authored-by: Vladimir <[email protected]>
jmduarte
added
enhancement
please test
Vitis HLS backend
|
I'm trying to run a model on vitis_hls 2023.2 getting the following error after extracting successfully the model from hls4ml. make: *** [Makefile.rules:318: csim.exe] Error 1 I would be glad if i get any help on this matter. Thank you! |
Description
An initial attempt at supporting Vitis HLS. Introduces a new backend (
Vitis) that takes most of the stuff over fromVivadobackend and changes things a little. I tried to bring the two backends closer together, so the changes that Vitis requires but are begnin in Vivado backend are ported to Vidado backend. The more extensive ones are implemented as overrides of the existing HLS routines (currently only the streaming implementations needed this). The biggest change in Vitis HLS is the more strict use of pragmas, so some workarounds we did before are no longer possible, and require changes. For example, setting stream depth in a loop is no longer possible (loop variable is not constant, so pragma refuses to use it), which meansencodedCNN implementation doesn't work and is removed from Vitis backend. There are also big issues withResourcestrategy in general. Regardless of which of the three implementations I try to use, the compiler will reintroduce the modulo operator and synthesize anuremcore which adds to the latency significantly. I'll try to resolve this later unless someone volunteers (yeah, unlikely).Some plots comparing the latency and resource usage between the two will follow.
Type of change
Tests
TODO.
We will also need to extend the synthesis tests to include the Vitis HLS (probably meaning we either fully switch to CERN machines, or we split over multiple machines at Fermilab.
Test Configuration:
Checklist