benchark L515 encoding time with alignment

[bmegli]

L515 pipelines perform additional post-processing to simulate depth units and depth clamping.

From #18 on SBCs depth+color pipelines we already barely keep up with the data rate when performing alignment.

This may need some optimization.

[bmegli]

Scenario:

• testing 7e9d5bd RNHVE bench-l515-postproc branch
• with i7-7820HK laptop as sender
• on Ubuntu 18.04

640x480 depth + 1280x720 color at 30 fps, 8Mb + 1 Mb bitrate, alignment to depth and color

#alignment to depth
./realsense-nhve-depth-color 192.168.0.100 9768 depth 640 480 1280 720 30 500 /dev/dri/renderD128 8000000 1000000 0.0000390625 short_range.json
#alignment to color
./realsense-nhve-depth-color 192.168.0.100 9768 color 640 480 1280 720 30 500 /dev/dri/renderD128 8000000 1000000 0.0000390625 short_range.json

With short_range.json exported from realsense-viewer ShortRange preset and removed lines that we set in RNHVE

{
    "Ambient Light": 2,
    "Apd Temperature": -9999,
    "Avalanche Photo Diode": 18,
    "Confidence Threshold": 1,
    "Depth Offset": 4.5,
    "Depth Units": 0.000250000011874363,
    "Error Polling Enabled": 1,
    "Frames Queue Size": 16,
    "Freefall Detection Enabled": 1,
    "Global Time Enabled": 0.0,
    "Inter Cam Sync Mode": 0.0,
    "Invalidation Bypass": 0.0,
    "Laser Power": 87,
    "Lld Temperature": 45.3513870239258,
    "Ma Temperature": 43.7060546875,
    "Mc Temperature": 43.8653526306152,
    "Min Distance": 190,
    "Noise Filtering": 3,
    "Post Processing Sharpening": 1,
    "Pre Processing Sharpening": 0.0,
    "Sensor Mode": 0.0,
    "Visual Preset": 5,
    "Zero Order Enabled": 0.0
}

[bmegli]

Results

Alignment to depth

./realsense-nhve-depth-color 192.168.0.100 9768 depth 640 480 1280 720 30 500 /dev/dri/renderD128 8000000 1000000 0.0000390625 short_range.json

• post processing between 2-5 ms, typically 2-3 ms
• whole time between 22-33 ms, typically below 28 ms
• CPU usage around 60% of 800% (as reported by htop)

Alignment to color:

./realsense-nhve-depth-color 192.168.0.100 9768 color 640 480 1280 720 30 500 /dev/dri/renderD128 8000000 1000000 0.0000390625 short_range.json

• post processing between 2-8 ms, typically 4-6 ms
• whole time between 32-50 ms, typically exceeds 33 ms
• CPU usage around 60% of 800% (as reported by htop)

[bmegli]

Discussion

Alignment to color is more time consuming. The reason is that that we end up with target resolution (1280x720 for color, 640x480 for depth). Also the resulting depth map when aligning to color will be 1280x720. This is somewhat representative for what we would get with L515 XGA depth (1024x768 ).

On laptop we keep up with the data rate when aligning to smaller depth (typically < 28 ms) and don't keep up when aligning to larger color (typically exceeds 33 ms a lot).

It is clear that we would also have problems on weaker SBC when aligning to color so this test is skipped.

Current naive depth post processing takes a few ms (2-3 or 4-6 for larger data).

[bmegli]

Conclusion

Looking at results naive post-processing takes < 5 ms. Comparing to #18 it is clear that the main villain is alignment for the larger resolution (1280x720 now vs 848x480 for D435).

L515 colored aligned depth pipeline is not usable in realtime at 30 FPS.

Depth aligned would vary between hardware.

L515 depth+color pipelines need optimization.