🗺️ KerasHub Roadmap 🗺️

[mattdangerw]

Here's an overview of the features we intend to work on in the near future. Feel free to comment on this issue below to suggest new directions for us to improve the library!

Models

We are always adding new models to the library. Here's what is currently on our radar.

• Vision models
  • We will be continually porting models over from KerasCV.
    • 📢 KerasNLP is now KerasHub 📢 #1831.
  • Yolov11.
  • ViT.
  • CoCa.
  • FLUX
  • LayoutLmV3.
• NLP models
  • Continue Gemma* releases.
    • As more models are released in the Gemma family of models, we will bring these to KerasHub on an ongoing basis.
  • Llama 3.1.
  • Qwen 2, Add qwen2 support #1679.
  • Mixtral.
  • all-MiniLM.
• Audio models
  • Add a high-level Whisper text to speech task with generate() support.
  • Audio Spectrogram Transformer.
  • wav2vec.

Feature improvements

• Feature extractor task support.
• Improved generation.
  • Move generation functionality to base classes #1861.
  • Directly use the backbone functional graph for CausalLM.generate() #1862.
  • Add support for JetStream generative inference for all KerasHub LLMs #1863.
• Improved quantization support (here and core Keras).
  • int4 support, QAT, more quantization options.
• Improved multi-host training support on Jax.
  • Add auto variable sharding for all backbones/tasks #1689.
  • Guide for multi-host distributed training with KerasHub #1850.
• An option for pythonic preprocessing decoupled from tf.data.
• Support RLHF and other instruction fine-tuning options beyond supervised fine-tuning.

Integrations

• Continue to add conversion support for Huggingface Transformers and Timm checkpoints.
  • See this blog post.
• Support JetStream, Add support for JetStream generative inference for all KerasHub LLMs #1863.
  • Allow native, high-throughput Jax LLM inference on TPUs.