--
this repo was created as part of an answer for
https://askubuntu.com/questions/1228501/how-can-i-choose-the-zoom-virtual-background-feature-using-ubuntu/1255945#1255945
Listing below due to CC By 4.0 licensing agreement.
+ original solution is published by Ben Elder: https://elder.dev/posts/open-source-virtual-background
+ modifications made are listed below.
+ tweaked installation steps to allow softwares to work in my environment.
+ added docker-compose file.
+ updated code to obtain environment variables defined in docker-compose file.
+ updated code to monitor folder to live switch background.
Follow the steps in section instructions
to enable virtual background in Linux during your zoom/teams/slack meetings.
- branch
master
is tested with Ubuntu 20.04, Nvidia GeForce 940MX, and a crappy web cam. - branch
cpu-friendly
is tested with Ubuntu 20.04 and no gpu.
Check out the original author @BenTheElder's post for a detailed explaination on what the code is doing. https://elder.dev/posts/open-source-virtual-background
Also check out a cpu-friendly derivation by fangfufu. https://github.com/fangfufu/Linux-Fake-Background-Webcam
Finally, for those interested in the underlying bodypix. Here is a detailed blog post on this amazing open source library, there is even a live demo page! https://blog.tensorflow.org/2019/11/updated-bodypix-2.html
If you have not yet gone through the above links and just want to know a bit of info on what the code/scripts are doing... essentially, we first grab image frames from the physical camera with OpenCV. For each image, the face/body is cropped with a pretrained neural net with TensorFlow.js Bodypix library, and then merged with a specified background image. The merged images are then continuously generated and used to create a video feed via pyfakewebcam and v4l2loopback. The implementation is dockerized to 2 containers, with the bodypix
container handling the body detection, while the fakecam
container handles the rest of the image processing, and also interfacing with the physical and virtual camera devices.
--
-
install docker (https://docs.docker.com/engine/install/ubuntu)
-
(skip if you have no GPU) install nvidia-docker (https://github.com/NVIDIA/nvidia-docker), and
sudo apt install nvidia-container-runtime
-
(skip if you have no GPU) test nvidia-docker is install properly
docker run --gpus all nvidia/cuda:10.0-base nvidia-smi
- install v4l2loopback
sudo apt-get upgrade -y ;\
sudo apt-get install -y v4l2loopback-dkms v4l2loopback-utils
- setup virtual video device as
/dev/video20
, and assuming the actual video device is/dev/video0
. (for me,/dev/video20
disappears after reboot, so these commands need to be run on boot if you want this device to always appear).
sudo modprobe -r v4l2loopback ;\
sudo modprobe v4l2loopback devices=1 video_nr=20 card_label="v4l2loopback" exclusive_caps=1
- confirm virtual camera is created, for my laptop i see
/dev/video0
,/dev/video1
and/dev/video20
.
find /dev -name 'video*'
- add root to group video (likely unecessary...)
sudo usermod -aG video root ;\
cat /etc/group | grep video
- clone repo
git clone [email protected]:pangyuteng/virtual-background.git vbkgd
cd vbkgd
- build via docker-compose
docker-compose build
- start the virtual camera via docker-compose (assuming gpu is present at
/dev/nvidia0
, physical video device at/dev/video0
and virtual video device at/dev/video20
)
docker-compose up
If this doesn't work and you get something like
ERROR: for vbkgd_bodypix_1 Cannot create container for service bodypix: Unknown runtime specified nvidia
do the steps described here and restart using docker-compose up
-
launch zoom/teams/slack..., select
v4l2loopback
as webcam -
live swap background by replacing file
data/background.jpg
- refresh rate hard coded at 3 seconds.
- build docker
docker build -t bodypix ./bodypix
docker build -t fakecam ./fakecam
- dev mode, remove entrypoint from Dockerfile, rebuild and run below to edit code
# in terminal
docker run --name=bodypix --network=fakecam -p 9000:9000 --gpus=all --shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864 -v ${PWD}/bodypix:/src -it bodypix /bin/bash
# in another terminal
docker run --name=fakecam --network=fakecam $(find /dev -name 'video*' -printf "--device %p ") -v ${PWD}/fakecam:/src -it fakecam /bin/bash