Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update ConvMixer to support reactant #1063

Merged
merged 12 commits into from
Jan 1, 2025
Merged

Conversation

avik-pal
Copy link
Member

@avik-pal avik-pal commented Nov 9, 2024

Copy link

codecov bot commented Nov 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 75.81%. Comparing base (3c3a432) to head (29a0cb9).

❗ There is a different number of reports uploaded between BASE (3c3a432) and HEAD (29a0cb9). Click for more details.

HEAD has 72 uploads less than BASE
Flag BASE (3c3a432) HEAD (29a0cb9)
92 20
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1063       +/-   ##
===========================================
- Coverage   85.82%   75.81%   -10.01%     
===========================================
  Files         149      147        -2     
  Lines        6173     6145       -28     
===========================================
- Hits         5298     4659      -639     
- Misses        875     1486      +611     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch from 090b87e to 23ed44b Compare November 10, 2024 02:01
@avik-pal avik-pal changed the title docs: update ConvMixer to support reactant feat: update ConvMixer to support reactant Nov 10, 2024
@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch from db3a12d to ef770a9 Compare November 11, 2024 02:55
Copy link
Contributor

github-actions bot commented Nov 11, 2024

Benchmark Results (ASV)

main ec3c44e... main/ec3c44e6c11390...
basics/overhead 0.123 ± 0.0011 μs 0.131 ± 0.0054 μs 0.939
time_to_load 0.9 ± 0.0041 s 0.902 ± 0.0029 s 0.998

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch 2 times, most recently from 25f13de to ee2fb4f Compare November 15, 2024 02:55
src/layers/pooling.jl Outdated Show resolved Hide resolved
@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch 2 times, most recently from b0709d4 to 442bb41 Compare November 15, 2024 23:20
@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch 2 times, most recently from a60c1de to 017076f Compare November 27, 2024 03:55
@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch 6 times, most recently from dc5e56e to fee949f Compare December 6, 2024 13:07
@avik-pal
Copy link
Member Author

avik-pal commented Dec 6, 2024

Not completely functional yet "error: expects input feature dimension (256) / feature_group_count = kernel input feature dimension (256). Got feature_group_count = 256."

xref: EnzymeAD/Reactant.jl#331

@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch 3 times, most recently from 438fede to e3ab45f Compare December 20, 2024 06:36
@avik-pal
Copy link
Member Author

Unfortunately there is a strong discrepancy in the gradients between Zygote and Reactant. Need to debug

@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch from d646b75 to a7fb9c9 Compare December 20, 2024 09:53
@avik-pal avik-pal changed the base branch from main to ap/reactant_updates December 20, 2024 09:53
@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch 3 times, most recently from 8e1ba6a to 644b60c Compare December 22, 2024 05:04
@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch from 0260eb6 to 4851036 Compare December 28, 2024 19:44
@avik-pal avik-pal force-pushed the ap/reactant_updates branch from 3919d03 to 33b1fc4 Compare December 30, 2024 15:05
@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch 3 times, most recently from c816f6a to ee5306c Compare December 30, 2024 16:56
@avik-pal avik-pal force-pushed the ap/reactant_updates branch 5 times, most recently from 960a49a to 10e11e1 Compare December 30, 2024 22:24
Base automatically changed from ap/reactant_updates to main December 30, 2024 22:28
@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch 3 times, most recently from 00c3206 to cf407e7 Compare December 31, 2024 14:30
@avik-pal avik-pal force-pushed the ap/conv_mixer_reactant branch from 29a0cb9 to fbfc851 Compare January 1, 2025 17:38
@avik-pal avik-pal marked this pull request as ready for review January 1, 2025 21:55
@avik-pal avik-pal merged commit 2bf2ca8 into main Jan 1, 2025
29 of 34 checks passed
@avik-pal avik-pal deleted the ap/conv_mixer_reactant branch January 1, 2025 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant