add LICENSE and fix some bugs

LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2021 yoyo-nb
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,75 +1,109 @@
 # Blender addon for driving character
 
-The addon drives the cartoon character by passing SMPL's poses and global translation into 3D model's armature in Blender. Poses and global translation can be obtained from ROMP or any other 3D pose estimation model. If the model outputs poses and global translation at a high FPS, you can drive cartoon characters in Blender in real-time.
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+![stars](https://img.shields.io/github/stars/yanch2116/CDBA.svg?style=flat)
+![GitHub repo size](https://img.shields.io/github/repo-size/yanch2116/CDBA.svg)
 
-## Demo
-
-![image](demo/demo1.gif)
-![image](demo/demo2.gif)
-
-The first demo uses ROMP outputs from the video, which is stored in a file.
-
-The second demo uses ROMP outputs from the webcam in real-time.
-
-The 3D character is downloaded from [Mixamo](https://www.mixamo.com/#/).
-
-## 
-
-## How to Use the [add-on](src/characterDriven.py)
-
-### Data Requester
-
-This addon is a data requester that sends a data request over TCP to `127.0.0.1:9999`.It gets one data at a time from the data server.
-
-After running the addon by pressing ctrl+E in Blender, it keeps asking for data until the server closes the TCP connection. You can also press A to close the TCP connection.
-
-### Data Server
-
-The data server is bound to `127.0.0.1:9999`. After receiving a request from the data requester, one data is sent to the requester.
-
-I've written [server.py](src/server.py) as a example data server (you only need to know a little about Python TCP to understand it).
-
-Real-time data server can be found in [ROMP](https://github.com/Arthur151/ROMP).You just need to run [webcam_blender.sh](https://github.com/Arthur151/ROMP/blob/master/scripts/webcam_blender.sh).
+This addon has two parts. The first part helps you get a 3D character that can be drived by ROMP. The second part is to drive a 3D character in Blender using ROMP based on image, video or webcam.
 
-### Data Format
-
-The data is a Python list of four elements in the form of `[mode,poses,global translation,current keyframe id]`.
-
-1. Mode is an integer, 1 for insert keyframe, 0 for no insert keyframe. Insert keyframes and animation rendering can be carried out later. Keyframes are generally not inserted in real-time mode to make real-time driving cartoon characters smoother.
-2. Poses is a list of length 72.
-3. Global translation is a list of length 3. If you don't need global translation, just go into [0,0,0].
-4. Current keyframe id is an integer.If you insert keyframes, you should set it to the correct keyframe id. If you don't insert keyframes, just set it to 0.
-
-### Steps
-
-1. Install the addon in Blender
-2. Run data server
-3. Press Ctrl+E in Blender to run addon
-4. Press A in Blender to stop addon or wait until the data transfer is complete
-
-> In step 3, you'd better make Armature active, otherwise bugs may occur (key "Pelvis" not found'). Also, the mouse must be placed in the 3D viewport area(where the 3d model is), otherwise the addon will not run.
-
-![picture 1](images/f385c458c9c3531c74c411689ce74a0cf4ffca92588888b9764775f8c7087f75.png)  
-
-
-## Something about Blender
-
-If you're not familiar with Blender, I've upload a [blender project](/blender/Beta.blend) to help you.All you need to do is open it and follow [Steps](#steps) to achieve the effect shown in the Demo.(It's better to know something about [animation](https://www.bilibili.com/video/BV1zh411Y7LX?from=search&seid=4554151926894860198&spm_id_from=333.337.0.0) in Blender.)
-
-If you need a video background in the demo, select Compositing in the top menu bar, click Open Clip in the Movie Clip, and select your video.
+## Demo
 
-![图 2](/images/7cabdcf52c78b5a42642a9acb0d1b835f54376f10fdd11f416e455e5e3b24fc5.png)
+- Use a video to drive a 3D character.
+![image](demo/offline_animation.gif)
+
+- Use webcam to drive a 3D character on a linux computer with GPU 1070.
+![image](demo/webcam_linux_local.gif)
+
+- Use webcam to drive a 3D character on a Macbook Pro connected to a linux server with GPU 1070.
+![image](demo/webcam_mac_server.gif)
+
+- Use webcam to drive a 3D character on a Macbook Pro with CPU.
+![image](demo/webcam_mac_local.gif)
+
+## Installation
+
+### For Blender Addon
+- Download [CharacterDriven-BlenderAddon]()
+- Install the addon in Blender
+- Check your 3d view and there should be two new menu item called CDBA and CATS
+- Install opencv in Blender python
+
+```Shell
+# If your Blender python doesn't have pip, install pip first
+/path/to/blender/python -m pip install opencv-python
+```
+
+### For Simple ROMP
+
+```Shell
+# If you want to use GPU, you need to install pytorch-gpu and cuda first.
+pip install --upgrade setuptools numpy cython
+pip install --upgrade simple-romp
+
+# Test if romp run successfully.
+romp --mode=video --show_largest -i=/path/to/video.mp4 -o=/path/to/results # For GPU
+romp --mode=video --show_largest --onnx -i=/path/to/video.mp4 -o=/path/to/results # For CPU
+```
+
+## How To Use
+
+A video tutorial is available [here]().
+
+### Panel
+
+![image](demo/ui.png)
+
+Common
+- Character: The character's object name.
+- Armature: The character's armature name.
+- IP: The IP address of the romp_server.
+- Port: The port of the romp_server.
+- Use GPU: Whether to use GPU.
+- Use Translation: Whether to use estimated global translation.
+- Insert Keyframes: Whether to insert keyframes.
+- Insert Interval: The interval of inserted keyframes.
+- Compression: The transferred image's compression quality.
+- Fix Bones: Make mixamo rigged character compatible with ROMP.
+
+Image or Video
+- Offline Animation: Select your image or video file to drive your character.
+
+Webcam
+- FPS: The frame rate of the webcam.
+- Width: The width of the webcam.
+- Height: The height of the webcam.
+- Webcam Animation: Open your webcam to drive your character.
+
+### Make Your 3D Character Compatible with ROMP
+
+- Use [mixamo auto-rigger](https://www.mixamo.com/#/) to rig your 3D character. Remember to select no fingers (25).
+- Import the rigged 3D character to Blender.
+- If your character is T-pose, skip this step. If your character is A-pose, 
+   - install the [cats-blender-plugin](https://github.com/absolute-quantum/cats-blender-plugin) in Blender;
+   - select your character and go to `Pose Mode`;
+   - click the `CATS` tab, then click the `Apply as Rest Pose` button;
+> All mixamo rigged characters are T-pose in Object Mode. You need to check whether your character's rest pose is A-pose or T-pose in Edit Mode.
+- Click the `CDBA` tab, then click `Fix Bones` to make your character compatible with ROMP.
+### Drive Your 3D Character
+
+- Start Simple ROMP.
+```Shell
+python romp_server.py [port]
+```
+- Make sure the settings are correct in Blender CDBA Panel
+   - Make sure your character object's and armature's name is same with the one in Common Panel.
+   - Make sure you can connect to the romp_server using the IP and Port in Common Panel.
+   - Other settings are optional, the default is fine.
+- Click Offline Animation to use image or video to drive your character.
+- Click Webcam Animation to use webcam to drive your character.
+
+> The Blender and romp_server is not necessary to be in the same computer. You can install romp_server on a linux server and use port forwarding to make the romp_server can be accessed from your local computer. A simple method is to use VS Code to set up port forwarding as I do in the video tutorial.
 
 If you are familiar with Blender and want to use your own model, you should make sure it's armature is SMPL's skeleton. The armature should name Armature and each bone has the same name as the bones in [demo model](blender/Alpha.fbx)(Only the 24 bones of SMPL skeleton are needed, and the fingers don't need to change their names).
 
-![图 3](/images/6b7e75964fd193b36ae58c94ddd99e6d234de6e085fb65d6f6691b476329b16c.png)
-
-## Useful Resources
-- [Mixamo](https://www.mixamo.com/#/)
-- [Blender 2.8](https://www.bilibili.com/video/BV1T4411N7GE?spm_id_from=333.999.0.0)
-- [Blender 2.9-3.0](https://www.bilibili.com/video/BV1zh411Y7LX?from=search&seid=12526205672689328022&spm_id_from=333.337.0.0)
-- [Blender Manual](https://docs.blender.org/manual/en/latest/)
-- [Blender Python API](https://docs.blender.org/api/current/index.html)
+- [ROMP](https://github.com/Arthur151/ROMP)
 - [neuron_mocap_live-blender](https://github.com/pnmocap/neuron_mocap_live-blender)
 - [QuickMocap-BlenderAddon](https://github.com/vltmedia/QuickMocap-BlenderAddon)
-- [remote-opencv-streaming-live-video](https://github.com/rena2damas/remote-opencv-streaming-live-video)
+- [Mixamo](https://www.mixamo.com/#/)
+- [模之屋](https://www.aplaybox.com/u/680828836)
+- [cats-blender-plugin](https://github.com/absolute-quantum/cats-blender-plugin)

addon/__init__.py

@@ -1,19 +1,22 @@
-import bpy
-from . import ops
-from .panels import *
-
 bl_info = {
-    "name" : "CDBA",
-    "author" : "Chuanhang Yan",
-    "description" : "CharacterDriven-BlenderAddon",
-    "blender" : (2, 93, 0),
+    'category': '3D View',
     "version" : (1, 0, 0),
+    "blender" : (2, 93, 0),
+    "author" : "Chuanhang Yan",
+    "name" : "CharacterDriven-BlenderAddon",
+    "location": "View 3D > Tool Shelf > CDBA",
+    "description" : "An Addon for Driving 3D Character",
+    'wiki_url': 'https://github.com/yanch2116/CDBA',
 }
 
+import bpy
+from . import ops
+from .panels import *
+
 CLASSES = [
-    CommonPanel,
-    OfflinePanel,
-    WebcamPanel,
+    COMMON_PT_myPanel,
+    OFFLINE_PT_myPanel,
+    WEBCAM_PT_myPanel,
     ops.FixBones,
     ops.OfflineAnimation,
     ops.WebcamAnimation,
@@ -26,11 +29,12 @@
     ('port',bpy.props.StringProperty(name='port', default='10005')),
     ('fps',bpy.props.IntProperty(name='fps', default=24)),
     ('insert_keyframe',bpy.props.BoolProperty(name='insert_keyframe', default=True)),
+    ('insert_interval',bpy.props.IntProperty(name='insert_interval', default=1)),
     ('gpu',bpy.props.BoolProperty(name='gpu', default=True)),
     ('translation',bpy.props.BoolProperty(name='translation', default=True)),
     ('quality',bpy.props.IntProperty(name='quality', default=60)),
-    ('width',bpy.props.IntProperty(name='width', default=512)),
-    ('height',bpy.props.IntProperty(name='height', default=512)),
+    ('width',bpy.props.IntProperty(name='width', default=640)),
+    ('height',bpy.props.IntProperty(name='height', default=480)),
 ]
 
 def register():