By Chi Wang, Min Zhou, Tiezheng Ge, Yuning Jiang, Hujun Bao, Weiwei Xu*
This repo is the official implementation of CF-Font: Content Fusion for Few-shot Font Generation (CVF:Paper link; arXiv:2303.14017) accepted by CVPR 2023.
- A poem demo
cffont_style_interpolation_part2_small.mp4
- Comparison with DG-Font
cffont_style_interpolation_part1_small.mp4
pytorch (>=1.0)
tqdm
numpy
opencv-python
scipy
sklearn
matplotlib
pillow
tensorboardX
scikit-image
scikit-learn
pytorch-fid
lpips
pandas
kornia
please refer to https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0 to install the dependencies of deformable convolution.
方正字库 provides free font download for non-commercial users.
- prepare dataset
- Put your font files to a folder and character file to charset
. ├── data │ └── fonts │ ├── Font_Seen240 │ │ ├── 000_xxxx.ttf │ │ ├── 001_xxxx.ttf │ │ ├── ... │ │ └── 239_xxxx.ttf │ └── Font_Unseen60 ├── charset │ ├── check_overlap.py │ ├── GB2312_CN6763.txt # all characters used in TRAIN and TEST │ ├── FS16.txt # Few-Shot characters, should not be included in TESTxxx.txt for fairness (can be included in TRAINxxx.txt) │ ├── TEST5646.txt # all characters used in TEST │ └── TRAIN800.txt # all characters used in TRAIN └── ...
- Generate dataset with the full standard Chinese character set (6763 in total) of GB/T 2312 :
After that, your file tree should be:
sh scripts/01a_gen_date.sh
. ├── data │ ├── fonts │ └── imgs │ ├── Seen240_S80F50_FULL │ │ ├── id_0 │ │ │ ├── 0000.png │ │ │ ├── 0001.png │ │ │ ├── ... │ │ │ └── 6762.png │ │ ├── id_1 │ │ ├── ... │ │ └── id_239 │ └── Unseen60_S80F50_FULL ├── charset └── ... - Get subsets with train, test and fewshot character txts.
After that, your file tree should be:
sh scripts/01b_copy_subset.sh
. ├── data │ ├── fonts │ └── imgs │ ├── Seen240_S80F50_FS16 │ │ ├── id_0 │ │ │ ├── 0000.png │ │ │ ├── 0001.png │ │ │ ├── ... │ │ │ └── 0015.png │ │ ├── id_1 │ │ ├── ... │ │ └── id_239 │ ├── Seen240_S80F50_FULL │ ├── Seen240_S80F50_TEST5646 │ │ ├── id_0 │ │ │ ├── 0000.png │ │ │ ├── 0001.png │ │ │ ├── ... │ │ │ └── 5645.png │ │ ├── id_1 │ │ ├── ... │ │ └── id_239 │ ├── Seen240_S80F50_TRAIN800 │ │ ├── id_0 │ │ │ ├── 0000.png │ │ │ ├── 0001.png │ │ │ ├── ... │ │ │ └── 0799.png │ │ ├── id_1 │ │ ├── ... │ │ └── id_239 │ ├── Unseen60_S80F50_FS16 │ ├── Unseen60_S80F50_FULL │ ├── Unseen60_S80F50_TEST5646 │ └── Unseen60_S80F50_TRAIN800 ├── charset └── ...
- Put your font files to a folder and character file to charset
- Train base network
Option: In order to evaluate the training of the network, we can use the script
# enable PC-WDL with the flag `--wdl` and PC-PKL with the flag `-pkl` sh scripts/02a_run_ddp.shscripts/option_run_inf_dgfont.shto inference. - Train CF-Font
- Select basis. Basis fonts better contain a standard font, like
song.- Manually. If you want select manually, please put basis ids (one line, seperated with a space) to a txt file, like:
0 1 2 3 4 5 6 7 8 9 - By clustering.
# Content embeddings collection sh scripts/03a_get_content_embeddings.sh # obtain basis ids through clustering sh scripts/03b_cluster_get_cf_basis.sh
- Manually. If you want select manually, please put basis ids (one line, seperated with a space) to a txt file, like:
- Get subsets with basis font ids.
sh scripts/03c_copy_basis_subset.sh
- Get basis weight for content fusion:
sh scripts/03d_cal_cf_weights.sh
- Train CF-Font
# make a folder for CF-Font training, and copy the pretrain model here. sh scripts/03e_init_cf_env.sh # train CF-Font sh scripts/03f_run_ddp_cf.sh
- Select basis. Basis fonts better contain a standard font, like
- Inference and evaluation:
# Inference (SII with the flag `--ft`) sh scripts/04a_run_inf_cf.sh # Evaluation ## get scores for each font sh scripts/04b_get_scores.sh ## get mean scores (use `-j` to skip the unwanted fonts, like basis fonts) sh scripts/04c_cal_mean_scores.sh
@InProceedings{Wang_2023_CVPR,
author = {Wang, Chi and Zhou, Min and Ge, Tiezheng and Jiang, Yuning and Bao, Hujun and Xu, Weiwei},
title = {CF-Font: Content Fusion for Few-Shot Font Generation},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2023},
pages = {1858-1867}
}We would like to thank Alimama (Alibaba Group) and State Key Lab of CAD&CG (Zhejiang University) for their support and advices in our project. Our code is based on DG-Font.
If you have any questions, please feel free to contact [email protected].
