Add ConvNeXt family of models to keras.applications

[sayakpaul]

Describe the feature and the current behavior/state.

ConvNeXt [1] family of models doesn't use attention or any new components for that but still achieves really good performance on ImageNet-1k while being efficient. They also demonstrate good performance on a variety of downstream tasks.

ConvNeXt family of models was trained using recipes used for training Vision Transformer-based models. Another point to note is that these models were evolved to simulate the design choices of Swin Transformers [2].

Will this change the current api? How?

Yes:

keras.applications.ConvNeXtBase, keras.applications.ConvNeXtTiny, ...

Who will benefit from this feature?

Keras users using keras.applications in their projects.

Contributing

• Do you want to contribute a PR? (yes/no): Yes
• If yes, please read this page for instructions
• Briefly describe your candidate solution(if contributing): I have implemented the non-isotropic ConvNeXt variants here (with all the original pre-trained params ported). I also co-contributed the TF variant of this to Hugging Face transformers (Add TFConvNextModel huggingface/transformers#15750).

References

[1] https://arxiv.org/abs/2201.03545
[2] https://arxiv.org/abs/2103.14030

@fchollet @LukeWood

[hertschuh]

Hi Sayak, this looks like a great addition. Would you be willing to put up a PR for this?

[sayakpaul]

Thanks. I think the following answers your question :)

Do you want to contribute a PR? (yes/no): Yes

[LukeWood]

Sweet - yes we are happy to take this, and we can take it in keras.applications. Cheers @sayakpaul and thanks!

[LukeWood]

Feel free to send PRs my way

[sayakpaul]

Alright! I will get started as soon as I can then.

[sayakpaul]

Join the conversation here: #16421.