Skip to content

Latest commit

 

History

History
executable file
·
161 lines (115 loc) · 6.65 KB

README_en.md

File metadata and controls

executable file
·
161 lines (115 loc) · 6.65 KB
Raw

BeautyMaster

中文 | English

DEMO | VIDEO

👋 join us on Static Badge

📘 Introduction

The "BeautyMaster" uses the InternLM2 large model, XTuner fine-tuning tool, and LMDeploy deployment tool to provide users with personalized fashion matching suggestions. By uploading full-body photos and wardrobe clothing, combined with seasons, temperature, weather, travel purposes and personal needs, the most perfect dressing plan can be customized.

The highlights of the project are as follows:

  • Fine-tuning of large models 🔧: By fine-tuning the InternLM2 model with the help of XTuner, the accuracy of personalized recommendations is improved;

  • Support personalized dressing 👗: Customize personalized fashion matching solutions according to the user's individual needs and occasions;

  • Intelligent recommendation 🌦️📅: Combine seasons, temperature, weather and other factors to provide users with practical dressing suggestions;

  • Easy to use 📸: Upload photos and clothes to easily obtain fashion matching solutions and say goodbye to choice difficulties;

  • Professional guidance 💡🔍: With the help of big data analysis and artificial intelligence technology, create the best fashion look for users;

  • Good scalability ✨: It can be expanded to facial makeup, hairstyles, bags and accessories in the later stage;

  • Immersive experience 🎨🌟: In the later stage, the matching and dressing will be applied to the user to provide 3D visual effects.

Welcome Everyone is welcome to pull request and ✨star✨ our project, your ✨star✨ will provide us with the motivation to move forward!!!

💪 Goal

  • Primary
  1. Train a VLM to have basic aesthetic ability.
  2. Align human aesthetic abilities.
  3. Choose the right match according to aesthetic ability.
  4. The previous results are presented through pictures.
  • Flow

alt text

✊ Todo

Train a VLM to have basic aesthetic ability

  • Data Processing(WIP)
    • Collect fashion wear data.
    • Look for a VLM to label ready-made images internVL.
    • Make a data annotation tool.
    • Correct data form VLM for test and train.

Align human aesthetic abilities

Choose the right match according to aesthetic ability.

  • The trained VLM gives suggestions on what to wear

The previous results are presented through pictures

  • Segment Tool

  • Virtual Try-on(WIP)

    • IDM-VTON: Features: it could keep background of the VToN, Techs: IP-Adapter, TryonNet, GarmentNet.
    • COTTON-size-does-matter: Features: it can adjust the size of VToN with a parameter.
    • OOTDiffusion: Techs: LDM-based, outfitting UNet, outfitting dropout, Classifier-free guidance.
    • OutfitAnyone: Ultra-high quality virtual try-on for Any Clothing and Any Person (Including Anime character pictures in demo).
    • StableVITON: Features: it could keep background of the VToN, Techs: LDM-based, zero cross-attention blocks, attention total variation loss and augmentation.
    • AnyFit

  • Visualize the results of the suggestions to the user

    • ControlNet
    • 4d-dress: A 4D Dataset of Real-world Human Clothing with Semantic Annotations
    • champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance
    • MOSS

📂 Repo structure (WIP)

├── README.md
├── README_cn.md
├── docs
├── scripts
├── beautymaster
│   ├── datasets
│   ├── third_party
│   │   ├── internvl
│   │   ├── IDM-VTON                     
│   │   ├── champ
│   │   ├── archpp
│   │   ├── ControlNet
│   │   └── OOTDiffusion
│   ├── demo
│   ├── src
│   ├── train     
│   └── utils
├── requirements.txt

🛠️ Requirements and Installation

  1. Clone this repository and open BeautyMaster folder
git clone --recursive https://github.com/RayTang88/BeautyMaster.git
cd BeautyMaster
  1. Install required packages
conda create -n beautyMaster python=3.10 -y
conda activate beautyMaster
  1. Install additional packages for training cases
pip install -r requirements.txt
MAX_JOBS=4 pip install -U flash-attn==2.5.8 --no-build-isolation # faster instalation for flash-attn

Datasets

Refer to Data.md

Infer

set env

export MODEL_ROOT="..."
export CODE_ROOT="..."
export DATA_ROOT="..."

Note: Set the root path for downloading models, the root path for cloning code, and the root path for data respectively.

gradio

python $CODE_ROOT/BeautyMaster/beautymaster/demo/app.py

python

python $CODE_ROOT/BeautyMaster/beautymaster/demo/infer.py

🔒 License

Usage and License Notices: The data, code, and checkpoint are intended and licensed for research use only,non-commercial use. They are also restricted to uses that follow the license agreement of InternVL, RLHF,DPO,ORPO,RLHF-V,segment-anything,SCHP,IDM-VTON,ControlNet,4d-dress,champ. The dataset is CC BY NC 4.0 (allowing only non-commercial use) and models trained using the dataset should not be used outside of research purposes.