Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optional fusion of convolution and BatchNorm layers during inference #185

Open
theabhirath opened this issue Jul 26, 2022 · 0 comments
Open

Optional fusion of convolution and BatchNorm layers during inference #185

theabhirath opened this issue Jul 26, 2022 · 0 comments
Labels
enhancement New feature or request layers Related to the Layers module - generic layers for reuse by models

Comments

@theabhirath
Copy link
Member

theabhirath commented Jul 26, 2022 •
edited
Loading

Convolution and BatchNorm layers have been fused during inference in many models, most notably LeViT. It would be a good idea to have this as an option in the conv_norm function in Layers with a dispatch specifically for BatchNorm.
@theabhirath theabhirath added the enhancement New feature or request label Jul 26, 2022
@theabhirath theabhirath added the layers Related to the Layers module - generic layers for reuse by models label Jul 26, 2022
@theabhirath theabhirath moved this from Bugs to Enhancements in Metalhead.jl API Redesign (GSoC 2022) Jul 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request layers Related to the Layers module - generic layers for reuse by models
Projects
No open projects
Metalhead.jl API Redesign (GSoC 2022)
Archived in project
Milestone
No milestone
Development

No branches or pull requests
1 participant
@theabhirath