Intro:

Random stuff accumulated here. Also has python env setup.

Important links:

Anaconda python link:

https://repo.anaconda.com/archive/Anaconda3-2023.07-1-Linux-x86_64.sh

Prompt reweighting and compel library:

https://huggingface.co/docs/diffusers/using-diffusers/weighted_prompts

Interesting repo for semantic segmentation:

https://github.com/facebookresearch/segment-anything.git

Colab notebook for automatic1111 SD web UI:

https://colab.research.google.com/github/TheLastBen/fast-stable-diffusion/blob/main/fast_stable_diffusion_AUTOMATIC1111.ipynb

Cuda installation for ubuntu:

Main link:

https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu

Verify You Have a CUDA-Capable GPU:

lspci | grep -i nvidia

Verify You Have a Supported Version of Linux:

uname -m && cat /etc/*release

Verify the System Has gcc Installed:

gcc --version

Verify the System has the Correct Kernel Headers and Development Packages Installed:

uname -r

Remove Outdated Signing Key:

sudo apt-key del 7fa2af80

Enroll the new signing key manually:

DISTRO='ubuntu2204'
ARCH='x86_64'
wget "https://developer.download.nvidia.com/compute/cuda/repos/${DISTRO}/${ARCH}/cuda-keyring_1.1-1_all.deb"
sudo dpkg -i 'cuda-keyring_1.1-1_all.deb'

Write the repo:

DISTRO='ubuntu2204'
ARCH='x86_64'
echo "deb [signed-by=/usr/share/keyrings/cuda-archive-keyring.gpg] https://developer.download.nvidia.com/compute/cuda/repos/${DISTRO}/${ARCH}/ /" > "/etc/apt/sources.list.d/cuda-${DISTRO}-${ARCH}.list"

Pin the repo:

DISTRO='ubuntu2204'
ARCH='x86_64'
wget "https://developer.download.nvidia.com/compute/cuda/repos/${DISTRO}/${ARCH}/cuda-${DISTRO}.pin"
sudo mv "cuda-${DISTRO}.pin" '/etc/apt/preferences.d/cuda-repository-pin-600'

Update and install:

sudo apt-get update
sudo apt-get install cuda
sudo apt-get install cuda-toolkit

Install basic OS stuff:

sudo apt-get 'install' \
    'aria2' \
    'build-essential' \
    'cpio' \
    'cuda-toolkit-11-7' \
    'fish' \
    'fizsh' \
    'git-lfs' \
    'squashfs-tools' \
    'zstd' \
;

Download anaconda python:

aria2c -c -j16 -x16 'https://repo.anaconda.com/archive/Anaconda3-2023.07-1-Linux-x86_64.sh'

Activate ENV:

Activate the env:

. /opt/anaconda/bin/activate
conda activate myenv

Important env variables:

export PATH="/usr/local/cuda-11.7/bin/:${PATH}"
export LD_LIBRARY_PATH="/usr/local/cuda-11.7/lib64:${LD_LIBRARY_PATH}"

Execute fizsh:

exec fizsh

ENV Setup with conda:

Current working directory:

SRC_DIR="$(dirname -- "$(realpath -- "${0}")")"
cd "${SRC_DIR}"
pwd

Activate base anaconda:

. /opt/anaconda/bin/activate

Create env:

conda create -n myenv

Activate env:

conda activate myenv

Install basic stuff:

conda install \
    'cython' \
    'ipython' \
    'jinja2' \
    'jupyter' \
    'jupyterlab' \
    'matplotlib' \
    'nbconvert' \
    'numpy' \
    'pandas' \
    'pillow' \
    'pyqt' \
    'python' \
    'requests' \
    'scikit-image' \
    'scikit-learn' \
    'scikit-learn-intelex' \
    'scipy' \
    'tqdm' \
;

Install conda-forge:

conda install -c conda-forge \
    'gradio' \
    'ninja' \
    'opencv' \
    'protobuf' \
    'pudb' \
    'setuptools' \
    'streamlit' \
    'termcolor' \
;

Download and install packages from GIT:

get_repo(){
    URL="${1}"
    D1="$(basename -- "$(dirname -- "${URL}")")"
    D2="$(basename -- "${URL}" | sed 's@\.git$@@g')"
    DIR_PREFIX="${HOME}/GITHUB/${D1}/"
    DIR_FINAL="${DIR_PREFIX}/${D2}"
    mkdir -pv -- "${DIR_PREFIX}"
    cd "${DIR_PREFIX}"
    git clone "${URL}"
    cd "${DIR_FINAL}"
    git pull
}

get_repo 'https://github.com/huggingface/diffusers.git'
pip install -e .

get_repo 'https://github.com/pharmapsychotic/clip-interrogator.git'
pip install -e .

get_repo 'https://github.com/patrickvonplaten/controlnet_aux.git'
pip install -e .

get_repo 'https://github.com/facebookresearch/segment-anything.git'
pip install -e .

get_repo 'https://github.com/SysCV/sam-hq.git'
pip install -e .

get_repo 'https://github.com/IDEA-Research/GroundingDINO.git'
pip install -e .

get_repo 'https://github.com/GoGoDuck912/Self-Correction-Human-Parsing.git'

get_repo 'https://github.com/IDEA-Research/Grounded-Segment-Anything.git'
git submodule update --init --recursive
cd ./segment_anything
pip install -e .

get_repo 'https://github.com/Hzzone/pytorch-openpose.git'

Updated pip stuff - 3:

pip install \
    'accelerate>=0.16.0' \
    'addict' \
    'datasets' \
    'fairscale' \
    'ftfy' \
    'gdown' \
    'huggingface_hub' \
    'mediapipe' \
    'nltk' \
    'onnxruntime' \
    'pycocotools' \
    'PyYAML' \
    'supervision' \
    'tensorboard' \
    'timm' \
    'torch' \
    'torchvision' \
    'transformers>=4.25.1' \
    'xformers' \
    'yapf' \
;