How to run my own video?

[zhanghongyong123456]

How can I use it on my own data

[dendenxu]

Hi @zhanghongyong123456 . There are several documentations to look into:

1. We listed some examples to run some baseline methods on a multi-view dataset in the readme here.
2. In the documentation on static scene, we showed how to run EasyVolcap on a video file. This doc includes the full process of extracting camera poses with COLMAP, convert camera parameters and then train models.
3. In this documentation, we showed the full process of how to prepare a custom multi-view dataset from multiple videos, including mask extraction, bounding box & visual hull preparation and background preparation.

[aleatorydialogue]

Is possible to render from a custom dataset generated from monocular video? I understand quality may suffer depending on dataset. I am very excited about the potential here, thank you for all your work.

[dendenxu]

Hi @aleatorydialogue , are you referring to monocular static or dynamic dataset?

For static dataset, we have a complete guide on data preparation, model training and inference here.

For monocular dynamic dataset, it indeed sounds like an interesting direction. Thus we're currently working on open-sourcing official support for deformable algorithms (like DyNeRF, NeRFies) as you can see by the deformation field entries in the default volumetric_video_network.py file. The full data preparation and model training pipeline will be open-sourced along the deformable model. We've already finalized the implementation and are currently working out kinks, so expect an update soon!

[aleatorydialogue]

awesome, appreciate the reply. Yes I am hoping to be able to do monocular dynamic datasets. For now, this will obviously limit quality and probably limit the novel views that are possible to infer, but I imagine over time we will have the ability to infer with less and less input. I look forward to the update, in the meantime i will work on your examples to get up to speed. Very promising project, thank you for open sourcing

[aleatorydialogue]

Just wanted to check back in here. Still very focused on dynamic monocular video, still haven't had too much success. Especially with regard to quality