Run custom / Global refine fails

[tomole444]

After the first step completes successfully the second step (which is started automatically) fails. It also fails, when I try to run it manually.
The console error is the following:

project train_images 161/272
project train_images 162/272
project train_images 163/272
project train_images 164/272
project train_images 165/272
project train_images 166/272
project train_images 167/272
Traceback (most recent call last):
  File "run_custom.py", line 247, in <module>
    run_one_video_global_nerf(out_folder=args.out_folder)
  File "run_custom.py", line 152, in run_one_video_global_nerf
    tracker.run_global_nerf(reader=reader, get_texture=True, tex_res=512)
  File "/home/grass/Documents/Leyh/BundleSDF/bundlesdf.py", line 773, in run_global_nerf
    mesh = nerf.mesh_texture_from_train_images(mesh, rgbs_raw=rgbs_raw, train_texture=False, tex_res=tex_res)
  File "/home/grass/Documents/Leyh/BundleSDF/nerf_runner.py", line 1511, in mesh_texture_from_train_images
    locations, distance, index_tri = trimesh.proximity.closest_point(mesh, pts)
  File "/opt/conda/envs/py38/lib/python3.8/site-packages/trimesh/proximity.py", line 152, in closest_point
    all_candidates = np.concatenate(candidates)
  File "<__array_function__ internals>", line 200, in concatenate
ValueError: need at least one array to concatenate

Your can find the full log here:
globalRefine.log

My dataset is here:
https://cloud.thws.de/s/bR2asfPPBjRHrmG

[wenbowen123]

From the log, it seems the 167th or 168th image (among the 272 keyframes) is broken, can you check that and its segmentation?

[tomole444]

These are the two masks for those two keyframes you mentioned.

Seems like the masks are not ideal.

[wenbowen123]

Indeed the mask looks noisy. But I guess it's not the root cause.

File "/home/grass/Documents/Leyh/BundleSDF/nerf_runner.py", line 1511, in mesh_texture_from_train_images
locations, distance, index_tri = trimesh.proximity.closest_point(mesh, pts)

From the log, can you print the shape of pts?

[tomole444]

Alright. Seems like pts is empty:
BookSmallRefine.log

[tomole444]

Here you can find the output-data:
https://cloud.thws.de/s/qWWK4CLKKZGMm3E

[wenbowen123]

Pushed a fix. Can you try again?