Language-based colorization produces plausible and visually pleasing colors under the guidance of user-friendly natural language descriptions. In this paper, we propose L-CAD to perform language-based colorization with any level descriptions.
Additionally, we showcase its capability for efficient automatic colorization.
- Python 3.9
- PyTorch 1.12
- NVIDIA GPU + CUDA cuDNN
Clone this repo:
https://github.com/changzheng123/L-CAD.git
Install PyTorch and dependencies
http://pytorch.org
Install other python requirements
pip install -r requirement.txt
Download pre-trained models here (baidu) (google) and put them into "./models", and run:
python colorization_main.py
Run demo with instance-aware sampling:
python inference.py
Extended COCO-Stuff provides additional correspondence annotations between color words and object words. Download coco2017 images here.
Multi-instance contains samples with distinctive visual characteristics and detailed language descriptions for multiple instances in images.
Download init_model from (baidu) (google) and put it into "./models".
python colorization_main.py -t
Licensed under a Creative Commons Attribution-NonCommercial 4.0 International.
Except where otherwise noted, this content is published under a CC BY-NC license, which means that you can copy, remix, transform and build upon the content as long as you do not use the material for commercial purposes and give appropriate credit and provide a link to the license.
If you use this code for your research, please cite our papers L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors
@InProceedings{lcad,
author = {Chang, Zheng and Weng, Shuchen and Zhang, Peixuan and Li, Yu and Li, Si and Shi, Boxin},
title = {L-CAD: Language-based Colorization with Any-level Descriptions using Diffusion Priors},
booktitle = {{NeurIPS}},
year = {2023}
}
L-CoDe: Language-based colorization using color-object decoupled conditions (AAAI22)
L-CoDer: Language-based colorization with color-object decoupling transformer (ECCV22)
L-CoIns: Language-based colorization with instance awareness (CVPR23)