How make custom data?

[lsy-92]

Hello!

Thanks to your great job!

i try to make a custom data.

i already made color image, depth image using Orbbec Femto.

then i make mask image using XMem.

But, BundleSDF doesn't work.

I don't understand other issue. what is different with your custom milk data?

[wenbowen123]

It's hard to tell from the limited info. A common pitfall is the depth data format, which should be uint16 PNG. Or you can use raw npy file and change the loading in the code.

[lsy-92]

i made it Y16 format. it is 16bit image. so it is not problem.
i want to know when you made depth image, did you use mask?
and how can I get cam_K.txt?
i read Femto camera parameter, but it is an identity matrix.

[wenbowen123]

depth should not depend on masks, so no. You might want to make sure after you read the depth image using the code, you are getting the right numbers.

Identity is certainly not right. Please check their manual/github.

[lsy-92]

i check your depth image's numpy.

that maximum value is 4.

i want to know how you nomalized that.

[wenbowen123]

suppose you have the depth data in meters, you can save to PNG by
cv2.imwrite(dir, (depth*1000).astype(np.uint16))

[lsy-92]

my depth image is (480, 640, 3) as numpy.

but it's value is in 0~255.

i want to make this image like your depth image.

[wenbowen123]

How did you obtain this image?

[lsy-92]

I use depth estimate model.

but it doesn't look like your depth image.

did you do postprocess your depth image?

[wenbowen123]

the depth image are the raw data from depth camera. If you use depth estimation model, make sure you are getting the metric depth.

[lsy-92]

I use Zoe Depth metric depth model. but it doesn't work well in BundleSDF. I try tunning parameter RANSAC, or zfar, or depth weight, but it doesn't work. what shoud i do?

[wenbowen123]

using monocular depth estimation to get depth could be highly uncertain for both metric depth and relatively depth quality. I'd suggest to try with a depth camera.

[lsy-92]

Is this problem not related to the performance of the depth estimation model?

Is there any way to solve uncertified depth?

[lsy-92]

I have another question. how can i evaluate a custom data result?

How can i make gt?

I always feel grateful for your kind answer.

[lsy-92]

i make Custom Dataset using Femto camera.

but bottle's nerf back part does not make well.

how can i fix it?

[wenbowen123]

from the GUI, can you tell if the tracking is lost?

[lsy-92]

It's a little wrong, but I think it went well.
and from the GUI, Tracking is not lost. i think it works well.

[JINAOLONG]

Hello!

Thanks to your great job!

i try to make a custom data.

i already made color image, depth image using Orbbec Femto.

then i make mask image using XMem.

But, BundleSDF doesn't work.

I don't understand other issue. what is different with your custom milk data?

hello,how do you solve the problem in the figure below?

[MichaelWangGo]

@JINAOLONG Hi, a solution works for me is to modify the get_depth() in YcbineoatReader class to make sure that depth ranges from 0 to 1:
def get_depth(self,i):
# import ipdb; ipdb.set_trace()
depth = cv2.imread(self.color_files[i].replace('rgb','depth'),-1)/1e5
depth = cv2.resize(depth, (self.W,self.H), interpolation=cv2.INTER_NEAREST)
return depth

[MichaelWangGo]

@lsy-92 Hi bro, how did you make masks for your data? Making a single mask as desribed in authors' paper? Or making mask for each rgb image?

[wenbowen123]

the depth does not have to be in [0,1]. You should make sure it's in metric scale.

You can use SAM2 to create masks.