met one error when run test.py for scannet dataset. KeyError: Caught KeyError in DataLoader worker process 0. KeyError: 'image_paths'

[jasmine202106]

Hi, when i run test.py for scannet dataset, i met one error.
please help, thanks very much.

root@65eb20bdb6a8:/mmdetection3d# python tools/test.py configs/imvoxelnet/imvoxelnet_scannet.py ./data/checkpoints/scannet.pth --show --show-dir ./data/scannet/show-dir/
Use load_from_local loader
[ ] 0/6, elapsed: 0s, ETA:Traceback (most recent call last):
File "tools/test.py", line 153, in
main()
File "tools/test.py", line 129, in main
outputs = single_gpu_test(model, data_loader, args.show, args.show_dir)
File "/mmdetection3d/mmdet3d/apis/test.py", line 27, in single_gpu_test
for i, data in enumerate(data_loader):
File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 363, in next
data = self._next_data()
File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 989, in _next_data
return self._process_data(data)
File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1014, in _process_data
data.reraise()
File "/opt/conda/lib/python3.7/site-packages/torch/_utils.py", line 395, in reraise
raise self.exc_type(msg)

Original Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 185, in _worker_loop
data = fetcher.fetch(index)
File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/opt/conda/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/mmdetection3d/mmdet3d/datasets/custom_3d.py", line 292, in getitem
return self.prepare_test_data(idx)
File "/mmdetection3d/mmdet3d/datasets/custom_3d.py", line 166, in prepare_test_data
input_dict = self.get_data_info(index)
File "/mmdetection3d/mmdet3d/datasets/scannet_monocular_dataset.py", line 19, in get_data_info
for i in range(len(info['image_paths'])):
KeyError: 'image_paths'

[filaPro]

Hi @jasmine202106 ,

Did you follow our ScanNet instructions in README?

[jasmine202106]

the filenames in the sens_reader/scans/scene0000_00/out extracted from sensreader are 0.jpg, 0.txt, 1.jpg, 1.txt......, not frame-000001.color.jpg .....
is that the reason?

others are all ok with your instruction.

[filaPro]

Are you running with --dataset scannet_monocular? Can you please ensure that this line is executed while running create_data.py? As your missing image_paths key is added there.

One more thing about filenames. If your names are like 0.txt and not like 000001.pose.txt, can you please update these 6 lines accordingly?
Sorry for these inconveniences. Your 0.txt is produced by ScanNet official python script, however they also provide a c++ code with 000001.pose.txt.

[jasmine202106]

[root@65eb20bdb6a8:/mmdetection3d#]python tools/create_data.py scannet --root-path ./data/scannet --out-dir ./data/scannet --extra-tag scannet --dataset scannet_monocular
usage: create_data.py [-h] [--root-path ROOT_PATH] [--version VERSION]
[--max-sweeps MAX_SWEEPS] --out-dir OUT_DIR
[--extra-tag EXTRA_TAG] [--workers WORKERS]
kitti
create_data.py: error: unrecognized arguments: --dataset scannet_monocular

root@65eb20bdb6a8:/mmdetection3d#python tools/create_data.py scannet_monocular --root-path ./data/scannet --out-dir ./data/scannet --extra-tag scannet
train sample_idx: scene0000_00 0/24
Traceback (most recent call last):
File "tools/create_data.py", line 293, in
monocular=True
File "tools/create_data.py", line 127, in scannet_data_prep
root_path, info_prefix, out_dir, workers=workers, monocular=monocular)
File "/mmdetection3d/tools/data_converter/indoor_converter.py", line 42, in create_indoor_info_file
infos_train = train_dataset.get_infos(num_workers=workers, has_label=True)
File "/mmdetection3d/tools/data_converter/scannet_data_utils.py", line 182, in get_infos
infos.append(self.process_single_scene(sample_idx, has_label))
File "/mmdetection3d/tools/data_converter/scannet_data_utils.py", line 148, in process_single_scene
base_file_names.remove('_info')
KeyError: '_info'

i update the lines in scannet_data_utils.py:
base_file_names = {x.split('.')[0] for x in os.listdir(frame_path)}
#base_file_names.remove('_info')
for base_file_name in base_file_names:
#pose = np.loadtxt(osp.join(frame_path, f'{base_file_name}.pose.txt'))
pose = np.loadtxt(osp.join(frame_path, f'{base_file_name}.txt'))
if np.all(np.isfinite(pose)):
#info['image_paths'].append(osp.join(frame_sub_path, f'{base_file_name}.color.jpg'))
info['image_paths'].append(osp.join(frame_sub_path, f'{base_file_name}.jpg'))
info['pose'].append(pose)

then:
root@f3e41478d06b:/mmdetection3d# python tools/create_data.py scannet_monocular --root-path ./data/scannet --out-dir ./data/scannet --extra-tag scannet
train sample_idx: scene0000_00 0/24
Traceback (most recent call last):
File "tools/create_data.py", line 293, in
monocular=True
File "tools/create_data.py", line 127, in scannet_data_prep
root_path, info_prefix, out_dir, workers=workers, monocular=monocular)
File "/mmdetection3d/tools/data_converter/indoor_converter.py", line 42, in create_indoor_info_file
infos_train = train_dataset.get_infos(num_workers=workers, has_label=True)
File "/mmdetection3d/tools/data_converter/scannet_data_utils.py", line 179, in get_infos
infos.append(self.process_single_scene(sample_idx, has_label))
File "/mmdetection3d/tools/data_converter/scannet_data_utils.py", line 152, in process_single_scene
with open(osp.join(frame_path, '_info.txt')) as file:
FileNotFoundError: [Errno 2] No such file or directory: './data/scannet/sens_reader/scans/scene0000_00/out/_info.txt'

i have no _info.txt, the files in out like this:
0.jpg 0.txt 1.jpg 1.txt 2.jpg 2.txt ......5577.jpg 5577.txt

i try to update the filenames to frame-000001.color.jpg frame-000001.pose.txt......, or 000001.color.jpg 000001.pose.txt, but still can not solve the problem.

[filaPro]

This _info.txt is the output of Scannet c++ script. We just use it to read single 4x4 matrix of camera intrinsics per scene. If you are using their python script can you please run it with --export_intrinsics and save the resulting 4x4 matrices for corresponding _info.txt file?

[filaPro]

Also my PR for ScanNet preprocessing is today merged to mmdetection3d master. It is much more documented and supports processing for example 100 images out of 5000 per scene to speed up this step. It also support multiprocessing. You can probably use this for preprocessing and then just simply update these 1, 2, 3 lines in ScanNetMultiViewDataset.

I'm going to update this here when I have some time (very likely on this weekend). Sorry again for your inconveniences.

[filaPro]

PR #21 aims to merge the full ScanNet processing from mmdetection3d master branch. However I need to test it later.