Changing the exposure in HDR

[torhans]

I need to continuously capture multiexposure images (tracking a laser point against other features) and do so using "MultiExposureUnmerged" on a RPi4 with IMX708.
I can't figure out how to control the exposure in the two channels. Fixed exposure would be fine, but otherwise by controlling the "metering_modes"/"exposure_modes".
Changing the exposure directly in the code, without the HdrMode, is too slow.

[dgalland]

The documentation on the new HDR possibilities libcamera/Picamera2 RPI4 or RPI5 is very difficult to understand.
If I understood correctly from reading the two pdf guides, it is not an HDR at the sensor level but rather a variant of the AGC algorithm
Firstly, three different modes or “Channels” are defined for the AE/AGC algorithm.
Then there would be three different modes for HDR :
SingleExposure: Accumulation of the same channel? No merge or tonemap
MultiExposure: Merge + tonemap of two channels (on RPI5 only)
MultiExposureUnmerged on RPI4 and RPI5 which would return frames without merge

But in any case it does something:
Auto image and HDR SingleMode image

So lots of questions!

• Firstly I think AeEnable is to be True, you can't set manually the two exposures, they are determined by AE/AGC?
• We don't really understand what is done in SingleExposure
• The example with switch_mode is not very informative when you want to stay in Still configuration

I tried the following scenario:
The camera start with AeEnable False Exposure 5000
AeEnable=True
picam2.set_controls({'HdrMode': libcamera.controls.HdrModeEnum.MultiExposure})
Request loop result:
Exposure: 5000 HdrMode: None HdrChannel: None
Exposure: 5000 HdrMode: None HdrChannel: None
Exposure: 5000 HdrMode: None HdrChannel: None
Exposure: 5000 HdrMode: None HdrChannel: None
Exposure: 5000 HdrMode: None HdrChannel: None
Exposure: 5000 HdrMode: None HdrChannel: None
Exposure: 7742 HdrMode: 2 HdrChannel: 1
Exposure: 8885 HdrMode: 2 HdrChannel: 3
Exposure: 7742 HdrMode: 2 HdrChannel: 1
Exposure: 8885 HdrMode: 2 HdrChannel: 3

From the 7th frame we have the merged frames
The two exposures are different but it's the same frame
I have not yet suceed to have the two different frames in unmerged mode

[torhans]

Thanks, and apologies for not returning earlier.

I actually sort of figured it out on my own. The following code mostly do what I'm after

from picamera2 import Picamera2
from libcamera import controls

tuning = Picamera2.load_tuning_file("imx708.json")
algo = Picamera2.find_tuning_algo(tuning, "rpi.agc")
algo["channels"][1]['base_ev']=1
algo["channels"][2]['base_ev']=100

picam2 = Picamera2(tuning=tuning)
conf=picam2.create_video_configuration()
picam2.configure(conf)
picam2.set_controls({"HdrMode": controls.HdrModeEnum.MultiExposureUnmerged})

picam2.start()
for i in range(10):
    array1, metadata1 = picam2.capture_arrays()
    array2, metadata2 = picam2.capture_arrays()
print(metadata1["ExposureTime"],metadata1["AnalogueGain"] * metadata1["DigitalGain"])
print(metadata2["ExposureTime"],metadata2["AnalogueGain"] * metadata2["DigitalGain"])
picam2.stop_recording()
picam2.close()

By changing the 'base_ev' for the two channels I can change the exposure of the two images. Further understanding the rest of the settings is a work in progress.

N.B. If you try to do this in an interactive console, jupyter or something like that, you need to delete 'picam2' every time you change the settings, and even that does not always work, and you'll need to kill the console.

[dgalland]

I come back to this discussion on the libcamera HDR on a PI5.
We understood that it is not an HDR at the sensor level but a special operating mode of the AGC/AE algorithm in which we define the parameters of two channels "Short" and "Long". Then the HDR algorithm returns
In unmerged mode:
Alternatively the two frames short and long on the main and raw streams
HdrMode: 1 HdrChannel: 1 AeLocked: True Exposure: 2628 Analog: 1.0 Digital: 1.0092315673828125
HdrMode: 1 HdrChannel: 3 AeLocked: True Exposure: 9914 Analog: 1.0 Digital: 1.0024373531341553
Only question here why channel 3 when HDR is defined on channels 1 and 2

It's more complicated in merged mode:
The merge or stitch performed in the Pisp under the control of libcamera with parameters defined in the configuration file, it is followed by an optional tonemap
In the Pisp doc we read:

and

So we understand that:
If Long is < Low keep Long
If Long > High take Short
Otherwise interpolate between Short and Long
This seems logical but I do not understand the default value of the two parameters:

	stitchEnable = params["stitch_enable"].get<int>(0);
	thresholdLo = params["threshold_lo"].get<uint16_t>(50000);
	diffPower = params["diff_power"].get<uint8_t>(13);

That would mean Low = 50000 and High = 50000+8192 ???
Also note the merged frame is returned on the "main" stream only.