D455 manual exposure setting giving bogus frames under some scenarios

[fbrygidyn]

---

Required Info
Camera Model	D455
Firmware Version	05.13.00.50
Operating System & Version	Ubuntu 18.04
Kernel Version (Linux Only)	4.15
Platform	PC
SDK Version	2.50
Language	C++ / N/A
Segment	others

When using both realsense-viewer 2.50 and manually using C++ code (SDK 2.50) I am getting bogus darker-than-expected output frames from D455 RGB camera.
The issue occurs seemingly only when inputting 2 specific values while separately there is no issue.

to reproduce: enable RGB camera, set exposure to 166 and then to 333.
The 333 image is noticeably darker than 166.
setting 166 and then 334 doesn't produce errors for example

Additionally after setting 333 and getting those darker bogus frames if I disable and enable the rgb camera then all exposure settings after that give garbage. enabling and disabling auto-exposure fixes this but only until I again try setting 166 > 333 exposure

---

This video should clearly show what is happening

out.mp4

---

png snapshots saved from the realsense-viewer exhibit the same behavior:

166 exposure set manually:

333 set right after that:

after disabling and enabling RGB camera (still supposedly 333 exposure):

---

• I tested 2 separate D455 units - both showed this issue.
• I also have several D435 units - No issues there
• Auto-exposure does not seem to be a problem - afaik it is disabled when a value is entered manually and even when I explicitly disable it the issue occurs the same way.
• I also tested older 2.48 SDK paired with 5.12.14.50 firmware - same issue
• tested in both sunlight and indoors with normal fluorescent ceiling lights

[fbrygidyn]

A thought: maybe this is connected to broken gain adjustments on D455?
Not sure if this is a known issue but gain seems to behave as is it was a postprocessing instead of actual gain control.

For example when pointed at sunlight with 1s exposure I would expect the output image to be pure white regardless of gain but that only happens when gain is set to 64 or higher. numbers from 0 to 63 clip the output to a lesser than white value.

Realsense viewer snapshot of a camera pointed at sunlight with 1s exposure but gain set to 0:

---

edit:
Or maybe this is just some intentional implementation of negative gain without marking it with negative values? (in which case this would definetely be unrelated to original exposure issues)

[MartyG-RealSense]

Hi @fbrygidyn I tested your described problem on two different computers and was unable to replicate it. If an RGB exposure value in a certain range is selected then the Viewer automatically changes it to a value of its own choosing. So if '333' was set then the Viewer auto-changed it to '312', and the image lightness was of an appropriate brightness and not darker than '156'. This occurred both when setting the exposure with the slider and manually typing it in.

The purple color shade of areas of your RGB image even before you change the settings makes me think that you may be experiencing some effects on the RGB image related to bright sunlight that are specific to the D455 camera model. These are described in the Intel PDF document at the link below on page 8 (the final page).

https://www.intelrealsense.com/download/13629/

[fbrygidyn]

@MartyG-RealSense Hi, Thank you for taking a look, and yes the problem with 333 auto-changing to 312 is the reason you can't reproduce it.
This is caused by the usual infinite "wisdom" of Microsoft.
Realsense on windows is using builtin Microsoft UVC stack which instead of implementing UVC specification properly imposes exposure setting limitations:
https://docs.microsoft.com/en-us/windows-hardware/drivers/stream/ksproperty-cameracontrol-exposure

Essentially you can only set 1s/(2^n) exposure values on Windows.
Which boils down to 1s, 500ms, 250ms, 125ms, 62.5ms, 31.2ms, 15.6ms, ... etc
For Realsense 312 exposure means 31.2ms = 1s/32

So you won't be able to reproduce this issue on windows, please check on a Linux based machine if possible.

---

As for the purple color: I am aware of this issue with bright sunlit/low-exposure. there is a lot of info about it both here and on realsense website. AFAIK this is unrelated to what I am experiencing with exposure setting.

[MartyG-RealSense]

There was a past case at #1957 where a RealSense user set RGB exposure to 333 and experienced a drop in FPS. A RealSense team member explained that if the exposure value exceeds the FPS period when using manual exposure then the FPS rate can drop.

[fbrygidyn]

Drop in FPS is perfectly normal when rising exposure beyond FPS period and at first this seemed like completely unrelated but it seems like D455 is behaving in this weird way only when I set the exposure to the period exactly. But again - only from some initial values.

I am able to reproduce the problem when setting exposure:

• from 166 to 1000 (in 1280x720 @ 10FPS mode)
• from 166 to 666 (in 1280x720 @ 15FPS mode)
• from 166 to 333 (in 1280x720 @ 30FPS mode)
• from 100 to 166 (in 424x240 @ 60FPS mode)

---

Honestly: this smells like a firmware bug.

[MartyG-RealSense]

If it was a bug then it is likely that it would not be a high development priority to address, as it is very rare for RealSense users to use large values like these for RGB exposure, and the exactness of the values that trigger it would make it even more unlikely that other users would experience it.

[fbrygidyn]

@MartyG-RealSense I see that you are also the person replying on https://support.intelrealsense.com community forum. And I don't see any other realsense support contact point.
Do you know where firmware issues should be reported?
I imagine you are here to support realsense SDK, not the firmware side of things.

I ask because it seems like there is more that will need to be reported (at least for D455 firmware) - Aside from the problem I described I also noticed I can't even trust metadata from the image frames I get.

---

To elaborate a bit (but that's a tangent):
While writing workarounds for the original issue I added a check for an actual exposure in a given frame.
(described for example here: https://dev.intelrealsense.com/docs/high-dynamic-range-with-stereoscopic-depth-cameras#section-2-4-manual-vs-auto-exposure)

I sometimes get invalid RS2_FRAME_METADATA_ACTUAL_EXPOSURE metadata tied to a frame. I understand somewhat that we need to wait to exposure to change but the metadata in a frame should at the very least be correct so that we know when the setting actually changed...

I tested this like so:

1. set exposure to 166
2. repeat pipeline.wait_for_frames().get_color_frame() until the returned frame metadata matches 166
3. do the same for 333
4. do the same for 500
5. Save all the returned images with apparently "correct"
   exposure.

Most of the time I get correct frames (every one a bit brighter) but sometimes I get for example 2 output frames of the same brightness.

I imagine that SDK is just wrapping the data gotten from firmware side so it isn't really something to fix in librealsense...

(For now as a workaround I just skip one additional frame after the metadata matches what I expected but hell - this shouldn't work like this)

---

Bottom line: There seem to be some firmware bugs that I would like to report somewhere where they would have a chance of getting fixed in the future. Please point me in a better direction if possible.

[MartyG-RealSense]

As your GitHub profile states that you are a member of Samsung R&D Institute Poland, the best way forward may be to request the granting of a Non Disclosure Agreement (NDA) account with Intel so that you can have a one to one technical discussion with them about your needs.

If you email me your contact information (your name, work address, email / telephone) then I can pass the details to Intel to consider an NDA application.

[MartyG-RealSense]

Hi @fbrygidyn Did you have any response to your filing of a request form on Zendesk, please?

[fbrygidyn]

@MartyG-RealSense
For now I only got a confirmation that they were able to reproduce the problem and it was forwarded to the dev team.
So I am waiting for updates ATM. WIll post here if I get any more info.

[MartyG-RealSense]

Thanks very much!

[MartyG-RealSense]

Hi @fbrygidyn Do you have an update about this case that you can provide, please? Thanks!

[fbrygidyn]

@MartyG-RealSense Not yet, Although It's been a month since I got info about the issue being forwarded to the dev team. Will try to ask for an update.

[MartyG-RealSense]

Thanks very much @fbrygidyn

[fbrygidyn]

I got an update that the issue may not be fixed in the short term.but "got on the investigation list'.

So it seems like if it's going to be fixed it's gonna take some time.
Feel free to close this issue if you prefer it. I can always add a comment to a closed issue after I get another update

[MartyG-RealSense]

Thanks very much @fbrygidyn for the update. As you suggest, I will close the case. Please feel free to add new comments at a later date. Thanks again!

[fbrygidyn]

It's been a while but I got an update from Intel zendesk - they started working on this and managed to reproduce the issue.

[MartyG-RealSense]

Thanks very much @fbrygidyn for the update!