Skip to content

Commit

Permalink
multi-frame readme, depth units
Browse files Browse the repository at this point in the history
- bump NHVD (multi-frame waiting for merge)
- use depth units compatible with older Realsense cameras
- update readme for new textured depth streaming

Related to #15
  • Loading branch information
bmegli committed Apr 26, 2020
1 parent 6d86d13 commit 587d729
Show file tree
Hide file tree
Showing 3 changed files with 20 additions and 14 deletions.
12 changes: 8 additions & 4 deletions Assets/PointCloudRenderer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,16 @@ public class PointCloudRenderer : MonoBehaviour
void Awake()
{
NHVD.nhvd_net_config net_config = new NHVD.nhvd_net_config{ip=this.ip, port=this.port, timeout_ms=500 };
NHVD.nhvd_hw_config[] hw_config = new NHVD.nhvd_hw_config[]{
NHVD.nhvd_hw_config[] hw_config = new NHVD.nhvd_hw_config[]
{
new NHVD.nhvd_hw_config{hardware="vaapi", codec="hevc", device=this.device, pixel_format="p010le", width=848, height=480, profile=2},
new NHVD.nhvd_hw_config{hardware="vaapi", codec="hevc", device=this.device, pixel_format="rgb0", width=848, height=480, profile=1}
};
//NHVD.nhvd_depth_config depth_config = new NHVD.nhvd_depth_config{ppx = 421.353f, ppy=240.93f, fx=426.768f, fy=426.768f, depth_unit = 0.0001f};
NHVD.nhvd_depth_config depth_config = new NHVD.nhvd_depth_config{ppx = 421.353f, ppy=240.93f, fx=426.768f, fy=426.768f, depth_unit = 0.00003125f};
};

//For depth units explanation see:
//https://github.com/bmegli/realsense-depth-to-vaapi-hevc10/wiki/How-it-works#depth-units
NHVD.nhvd_depth_config depth_config = new NHVD.nhvd_depth_config{ppx = 421.353f, ppy=240.93f, fx=426.768f, fy=426.768f, depth_unit = 0.0001f};
//NHVD.nhvd_depth_config depth_config = new NHVD.nhvd_depth_config{ppx = 421.353f, ppy=240.93f, fx=426.768f, fy=426.768f, depth_unit = 0.00003125f};

nhvd=NHVD.nhvd_init (ref net_config, hw_config, hw_config.Length, ref depth_config);

Expand Down
20 changes: 11 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Example of video and point cloud streaming with hardware decoding and custom [ML

- streaming video to UI element (RawImage)
- streaming video to scene element (anything with texture)
- streaming point clouds (Mesh)
- streaming textured point clouds (Mesh)

This project contains native library plugin.

Expand Down Expand Up @@ -96,9 +96,10 @@ Assuming you are using VAAPI device.

### Receiving (Unity) side

- Open the project in Unity
- Choose `Canvas` -> `CameraView` -> `RawImage`
- For `RawImageVideoRenderer` component define
- open the project in Unity
- choose `Canvas` -> `CameraView` -> `RawImage`
- make sure it is enabled
- for `RawImageVideoRenderer` component define
- `Device`
- note the `Port`

Expand Down Expand Up @@ -128,32 +129,33 @@ If you have Realsense camera you may use [realsense-network-hardware-video-encod
```bash
# assuming you build RNHVE, port is 9766, VAAPI device is /dev/dri/renderD128
# in RNHVE build directory
./realsense-nhve-h264 127.0.0.1 9766 color 640 360 30 10 /dev/dri/renderD128
./realsense-nhve-h264 127.0.0.1 9766 color 640 360 30 20 /dev/dri/renderD128
```

If everything went well you will see 10 seconds video streamed from Realsense camera.

If everything went well you will see 20 seconds video streamed from Realsense camera.

### Streaming to scene (not UI)

Configure as above:
- `VideoQuad` `VideoRenderer` componenent
- make sure it is enabled

### Point cloud streaming

Assuming Realsense D435 camera with 848x480.

Configure as above:
- `PointCloud` `PointCloudRenderer` component
- make sure it is enabled

If you have Realsense camera you may use [realsense-network-hardware-video-encoder](https://github.com/bmegli/realsense-network-hardware-video-encoder).

```bash
# assuming you build RNHVE, port is 9768, VAAPI device is /dev/dri/renderD128
# in RNHVE build directory
./realsense-nhve-hevc 127.0.0.1 9768 depth 848 480 30 50 /dev/dri/renderD128
./realsense-nhve-hevc 127.0.0.1 9768 depth 848 480 30 500 /dev/dri/renderD128
# or for textured point cloud (only D435)
./realsense-nhve-hevc 127.0.0.1 9768 depth+ir 848 480 30 50 /dev/dri/renderD128
./realsense-nhve-depth-ir 127.0.0.1 9768 848 480 30 500 /dev/dri/renderD128 8000000 1000000 0.0001
```

For troubleshooting you may use:
Expand Down

0 comments on commit 587d729

Please sign in to comment.