Blank viewport when traning

[imAngryPig]

Hi, there is a question when running ns-train splatfacto --data ../../data/dnerf/bouncingballs( my dataset folder ):
when I open browser viewer:

Exception in thread Thread-120:
Traceback (most recent call last):
  File "/home/csong2/anaconda3/envs/shh_4dgsB/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/data/shh/code/4D-Rotor-Gaussians-main/nerfstudio/viewer/render_state_machine.py", line 222, in run
    outputs = self._render_img(action.camera_state)
  File "/home/data/shh/code/4D-Rotor-Gaussians-main/nerfstudio/viewer/render_state_machine.py", line 168, in _render_img
    outputs = self.viewer.get_model().get_outputs_for_camera(camera, obb_box=obb)
  File "/home/csong2/anaconda3/envs/shh_4dgsB/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/data/shh/code/4D-Rotor-Gaussians-main/nerfstudio/models/splatfacto.py", line 1494, in get_outputs_for_camera
    outs = self.get_outputs(camera.to(self.device))
  File "/home/data/shh/code/4D-Rotor-Gaussians-main/nerfstudio/models/splatfacto.py", line 1288, in get_outputs
    output = self.forward_1(camera,background)
  File "/home/data/shh/code/4D-Rotor-Gaussians-main/nerfstudio/models/splatfacto.py", line 903, in forward_1
    world_to_camera = torch.linalg.inv(camera_to_world).to(torch.float)
RuntimeError: linalg.inv: A must be batches of square matrices, but they are 3 by 4 matrices

and the viewer is purely blank:

How to solve?
Thanks.

[ttaoREtw]

Just add a [0,0,0,1] row to camera_to_world should work.

camera_to_world = torch.cat(
    [camera_to_world, torch.tensor([[0, 0, 0, 1]], device=camera_to_world.device)], dim=0
)

[imAngryPig]

Just add a [0,0,0,1] row to camera_to_world should work.

camera_to_world = torch.cat(
    [camera_to_world, torch.tensor([[0, 0, 0, 1]], device=camera_to_world.device)], dim=0
)

@ttaoREtw Thanks a lot!
But when running the lego model(dnerf Dataset), my 24GB , RTX4090 error message memory is insufficient, what is the reason for this, is the batch size too large or a large amount of video memory leakage occurred during code design?How to solve it. Thanks for your help.
Around Step 12000:

Step (% Done) Train Iter (time) ETA (time) Train Rays / Sec

12030 (60.15%) 199.850 ms 26 m, 32 s 3.26 M
12040 (60.20%) 195.978 ms 25 m, 59 s 3.32 M
12050 (60.25%) 197.841 ms 26 m, 12 s 3.29 M
12060 (60.30%) 200.940 ms 26 m, 35 s 3.24 M
12070 (60.35%) 202.773 ms 26 m, 47 s 3.21 M
12080 (60.40%) 203.455 ms 26 m, 51 s 3.20 M
12090 (60.45%) 204.100 ms 26 m, 54 s 3.21 M
12100 (60.50%) 205.624 ms 27 m, 4 s 3.18 M
12110 (60.55%) 237.360 ms 31 m, 12 s 2.78 M
12120 (60.60%) 281.602 ms 36 m, 59 s 2.29 M

File "/home/4D-Rotor-Gaussians/nerfstudio/models/splatfacto.py", line 99, in knn_fast knndist_kx8, knnmask_kx8, knnidx_kx8 = knn_ops.reorder_data_bw(knnidx_bxpx8.int(), num_elements_in_each_bin.int(), knndists_bxpx8, points_reorder_idx, num_points) torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 4.43 GiB. GPU 0 has a total capacty of 23.65 GiB of which 4.17 GiB is free. Including non-PyTorch memory, this process has 19.47 GiB memory in use. Of the allocated memory 17.21 GiB is allocated by PyTorch, and 1.77 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF