Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inference with COLMAP processed results #19

Open
bell-one opened this issue May 4, 2023 · 4 comments
Open

Inference with COLMAP processed results #19

bell-one opened this issue May 4, 2023 · 4 comments

Comments

@bell-one
Copy link

bell-one commented May 4, 2023

First of all, Thanks for share your code with great paper,

I think this initial planar mapping will be very useful with another multi view image process,

and I'd like to use your initial planar information with neural radiance fields,
so I tried some edits for COLMAP processed results, with iphone demo reference.

But I cannot get expected results at all,
I tried with NeRF/llff/room dataset which has sufficient planar surfaces,

DJI_20200226_143854_410
like this, but as results, I got like follows,

snapshot00

I think there should be problem with pre-processing of colmap data
and these errors propagated to coordinate systems, scaling, intrinsic params or projection matrix.

Can you help me to apply colmap dataset to your project?

@ymingxie
Copy link
Collaborator

ymingxie commented May 6, 2023

Thank you for your interest in our work! I will take a look soon ...

@ymingxie
Copy link
Collaborator

Hey, I checked the llff data and found that there seems to be a large domain gap between ScanNet (the dataset we trained the model) and llff. The differences mainly lie in the fact that the geometry in llff dataset is far away from the camera, while in ScanNet, the camera usually scans the scene, so the surface won't be that far. We predefined a local volume in PlanarRecon for each fragment, so the surface (about 4 meters far away from the camera) is excluded.

It's not easy to directly adopt the PlanarRecon with ScanNet pretrained model on NeRF dataset. It would be better to finetune the model on such a dataset but it seems that gt plane on NeRF dataset is hard to obtain... I don't have a good idea for now :(

@kwea123
Copy link

kwea123 commented May 13, 2023

@ymingxie thank you for the insight, can you still provide the script to process colmap data anyway? so that we can try on other types of data, not only llff.

Thank you

@bell-one
Copy link
Author

@ymingxie
Thanks for the review, I'm sure there must be a gap in the data domain. I'll do some fine-tuning as you suggested, or I'll make another attempt to apply the callmap processing result.
@kwea123
I made some initial code edit for use colmap data in here,
If you are interested, I think it would be nice to make it together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants