This repository contains the code used in our paper Exposing the Unseen: Exposure Time Emulation for Offline Benchmarking of Vision Algorithms accepted at IROS2024.
We created a Dockerfile to easily run our code using a docker-compose.yaml.
If you want to use the docker container, you have to install Docker using this website: https://docs.docker.com/engine/install/
We will first start by a running a version of the code using a small sample of the dataset and running with the default parameters.
Start by cloning this repository on your computer.
git clone -b IROS2024 [email protected]:norlab-ulaval/BorealHDR.gitThe following steps can be achieved using our docker image (recommended way) or with a standard python virtual environment.
You can open the devcontainer in vscode if you are familiar, or build the image with docker compose up --build.
cd BorealHDR/.devcontainer/
docker compose up --buildAfter finishing building the docker image, you can connect to it from a new terminal
docker exec -it borealhdr_container /bin/bashWhen you are inside the docker container, you can direclty emulate images from the dataset by running
cd /home/user/code/scripts/
python3 emulator_threads.pyInstead of using the docker container, you can also direclty run the code locally using a virtual environment. First you have to create the python virtual environment and install the dependencies from the requirements.txt file:
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txtTo emulate images, you need to run the script emulator_threads.py that you can find in BorealHDR/scripts/.
python emulator_threads.pyThe emulator_threads.py loads parameters.yaml file. You can adapt some parameters, for example, you can select which automatic-exposure technique to use and also some debugs parameters. Note that emulator_threads.py uses multiple threads to accelerate the processes. You can then emulate multiple automatic-exposure algorithms by un-commenting methods in automatic_exposure_techniques from parameters.yaml. The following table describes the available parameters related to the emulation.
| Parameter | Description | Values (default first) |
|---|---|---|
exposure_time_init |
Exposure time of the first emulated image | 4.0 |
dataset_folder |
Parent path to the dataset (link to the dataset) | "../data_sample/" |
experiment |
Sequence name | "backpack_2023-09-25-15-05-03" |
depth_emulated_imgs |
Emulate images in 8bits or 12bits | 812 |
emulated_in_color |
Boolean: Emulate images in color (only for 8bits) | TrueFalse |
automatic_exposure_techniques |
Select one or more automatic-exposure methods to emulate | "classical-50""classical-30""classical-70""manual-0""gradient-0""ewg-0""softperc-0" |
save_or_show_emulated_imgs |
Show or save the emulated images | "save""show" |
save_path |
Path to which the results of the emulation will be saved | "/home/user/code/output/emulated_images/" |
The BorealHDR Dataset was acquired at the Montmorency Forest in Québec City, Canada. In winter, this environment creates several HDR scenes coming from snow and trees. It was developed mainly to be used with our emulation technique.
The images were collected using the bracketing technique with six exposure times: 1, 2, 4, 8, 16, and 32 ms. A ground truth is provided using the 3D lidar data and a lidar-inertial-SLAM algorithm based on Libpointmatcher.
BorealHDR contains:
- 55 trajectories
- 10 km
- 5 hours of data
- 393 238 images
- Ground truth 3D maps
- 3D lidar point clouds
- IMU measurements
- GPS data
We provide a compressed version of the dataset in the following tables. In this version, the images still have a depth of 12-bits, but they have been compressed to reduce the overall size of the dataset. If you are interested by the full resolution version, contact us and we will find a way to share it with you!
Once you downloaded a trajectory, you will have to decompress it before using it with the emulator. You also have to put all the trajectories downloaded in the same folder, for exmaple: dataset/backpack_.../, and then only mount the dataset/ folder.
If you are using docker, read the next section. Otherwise, you can skip to here.
If the container is still running on your computer, you have to stop it first with ctrl + c or with docker compose down. Then, to mount the dataset into the docker container, you have to add it to the BorealHDR/.devcontainer/docker-compose.yaml file. You want to change the last line to point to your dataset folder path
- <Add directory dataset>:/home/user/code/dataset_mount_point/Save the file and start the container again
docker compose upConnect to the docker container from a new terminal
docker exec -it borealhdr_container /bin/bashWe need to modify the parameters.yaml file to point to our dataset. Open the file using your favorite editor
cd BorealHDR/
micro parameters.yamlModify the dataset_folder parameter to ../dataset_mount_point/ if you are using the docker, or to <Add directory dataset> if you are running it locally. Depending on the trajectory you downloaded, you also have to change the experiment parameter to the trajectory name (for example: backpack_2023-09-27-12-51-03)
To start the emulation, go to the scripts folder and run
python3 emulator_threads.pyYou may have to copy-paste the link if directly clicking on Download did not work.
| Mont-Bélair | Size (GB) | Download Link |
|---|---|---|
| backpack_2023-09-27-12-46-32 | 6.7 | Download |
| backpack_2023-09-27-12-51-03 | 4.6 | Download |
| backpack_2023-09-27-13-20-03 | 6.3 | Download |
| backpack_2023-09-27-13-25-44 | 6.1 | Download |
| backpack_2023-09-27-13-29-22 | 8.4 | Download |
| backpack_2023-09-27-13-34-17 | 7.9 | Download |
| Campus | Size (GB) | Download Link |
|---|---|---|
| backpack_2023-09-25-15-00-05 | 6.5 | Download |
| backpack_2023-09-25-15-05-03 | 7.6 | Download |
| backpack_2023-09-25-15-22-43 | 14.5 | Download |
| Forest-20 | Size (GB) | Download Link |
|---|---|---|
| backpack_2023-04-20-09-29-14 | 4.6 | Download |
| backpack_2023-04-20-09-51-13 | 18.3 | Download |
| backpack_2023-04-20-10-04-23 | 11.2 | Download |
| backpack_2023-04-20-10-12-17 | 6.8 | Download |
| backpack_2023-04-20-10-17-24 | 10.5 | Download |
| backpack_2023-04-20-10-26-28 | 6.8 | Download |
| backpack_2023-04-20-10-51-41 | 10.1 | Download |
| backpack_2023-04-20-10-59-06 | 9.1 | Download |
| backpack_2023-04-20-11-05-33 | 3.7 | Download |
| backpack_2023-04-20-11-08-58 | 13.8 | Download |
| backpack_2023-04-20-11-17-07 | 8.3 | Download |
| backpack_2023-04-20-11-23-00 | 12.3 | Download |
| backpack_2023-04-20-11-33-10 | 8.8 | Download |
| backpack_2023-04-20-11-43-53 | 12.5 | Download |
| backpack_2023-04-20-14-02-06 | 7.8 | Download |
| backpack_2023-04-20-14-07-00 | 3.0 | Download |
| backpack_2023-04-20-14-09-25 | 2.9 | Download |
| backpack_2023-04-20-14-14-15 | 8.4 | Download |
| backpack_2023-04-20-14-21-21 | 9.8 | Download |
| backpack_2023-04-20-14-32-48 | 7.4 | Download |
| backpack_2023-04-20-14-39-36 | 0.7 | Download |
| backpack_2023-04-20-14-55-15 | 5.6 | Download |
| backpack_2023-04-20-14-59-46 | 6.4 | Download |
| backpack_2023-04-20-15-03-58 | 2.8 | Download |
| backpack_2023-04-20-15-05-25 | 5.6 | Download |
| backpack_2023-04-20-15-13-55 | 8.2 | Download |
| backpack_2023-04-20-15-38-49 | 19.1 | Download |
| Forest-21 | Size (GB) | Download Link |
|---|---|---|
| backpack_2023-04-21-08-51-27 | 6.0 | Download |
| backpack_2023-04-21-09-15-59 | 9.3 | Download |
| backpack_2023-04-21-09-22-05 | 9.2 | Download |
| backpack_2023-04-21-09-31-09 | 16.1 | Download |
| backpack_2023-04-21-09-41-22 | 8.8 | Download |
| backpack_2023-04-21-09-49-58 | 0.3 | Download |
| backpack_2023-04-21-09-52-00 | 5.1 | Download |
| backpack_2023-04-21-09-54-38 | 4.1 | Download |
| backpack_2023-04-21-10-23-23 | 14.6 | Download |
| backpack_2023-04-21-10-32-34 | 13.6 | Download |
| backpack_2023-04-21-10-46-54 | 6.8 | Download |
| backpack_2023-04-21-10-57-59 | 7.9 | Download |
| backpack_2023-04-21-11-03-44 | 5.7 | Download |
| backpack_2023-04-21-11-08-10 | 3.0 | Download |
| backpack_2023-04-21-11-27-52 | 3.0 | Download |
| backpack_2023-04-21-11-30-58 | 9.5 | Download |
| backpack_2023-04-21-11-38-04 | 3.9 | Download |
| backpack_2023-04-21-11-43-19 | 3.6 | Download |
| backpack_2023-04-21-11-47-04 | 12.7 | Download |
| backpack_2023-04-21-12-07-33 | 6.9 | Download |
| backpack_2023-04-21-13-44-26 | 25.2 | Download |
| backpack_2023-04-21-14-01-39 | 6.4 | Download |
| backpack_2023-04-21-14-10-00 | 6.2 | Download |
| backpack_2023-04-21-14-20-06 | 0.4 | Download |
| backpack_2023-04-21-14-59-17 | 9.1 | Download |
| backpack_2023-04-21-15-07-12 | 4.6 | Download |
| backpack_2023-04-21-15-10-54 | 4.9 | Download |
| backpack_2023-04-21-15-16-29 | 7.9 | Download |
If you use our emulator or our BorealHDR dataset in your work, please cite our preprint:
@misc{gamache2023exposing,
title={{Exposing the Unseen: Exposure Time Emulation for Offline Benchmarking of Vision Algorithms}},
author={Olivier Gamache and Jean-Michel Fortin and Matěj Boxan and Maxime Vaidis and François Pomerleau and Philippe Giguère},
year={2024},
eprint={2309.13139},
archivePrefix={arXiv},
primaryClass={cs.RO}
}