ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy

Hong Li^✶, Zhiquan Tan^✶, Xingyu Li^✶, Weiran Huang

 

 

In this paper, we propose an adapter-based two-stage learning paradigm, a multi-modal continual learning scheme that consists of experience-based learning and novel knowledge expansion, which helps the model fully use experience knowledge and compensate for novel knowledge. Extensive experiments demonstrate that our method is proficient for continual learning. It expands the distribution of representation upstream while also minimizing the negative impact of forgetting previous tasks. Additionally, it enhances the generalization capability for downstream tasks. Furthermore, we incorporate both multi-modal and uni-modal tasks into upstream continual learning. We observe that learning from upstream tasks can help with downstream tasks.

Installation

Usage

Reference

@article{li2024atlas,
  title={ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy},
  author={Li, Hong and Tan, Zhiquan and Li, Xingyu and Huang, Weiran},
  journal={arXiv preprint arXiv:2410.10923},
  year={2024}
}