using non Kitti data with model

[BenMacKenzie]

Any tips on getting model to work with non kitti data?

I ran kittiseg against an image of a road using demo.py and pre-trained weights. it didn't detect anything. might it be sensitive to lighting conditions? aspect ratio (i cropped the image to have similar dimensions, but this did not help)?

works great with all of the images from kitti.

thanks!

[MarvinTeichmann]

Hi Ben,

Any tips on getting model to work with non kitti data?

yes, the easiest solution is to train the model on your data. You could do "fine-tuning" that is, you use the pretrained kittimodel and then perform a few training steps with some (few) labeled images of your dataset. The model generalize very fast from there.

It didn't detect anything. might it be sensitive to lighting conditions?

Yes, the training is optimized to perform well on the benchmark. It is very sensitive to various aspects including lighting conditions and sharpness. (Try bilinear minor up or downsampling of a kitti image. The result will be bad). You can actually build a model which is much better in terms of generalization to new data. A key technique for doing this is data augmentation. Distortions, like random brightness, random resizing (including the change of aspect ratio) and even fancier thinks will force the ignore camera depended hints. Some data augmentation is already implemented in the input-producer (but turned of). But you can also implement much more.

aspect ratio (i cropped the image to have similar dimensions, but this did not help)?

Cropping will not change anything at all. The model consists purely of convolutions which are invariant to cropping (apart from a stride of 32 pixel).

[BenMacKenzie]

Thanks Marvin

Makes sense, though i imagine aspect ratio could have some impact, even with conv layers, at least in extreme cases.

Also, in just quickly reading through the code, it appears that training will update all the weights including the early conv layers - at least i don't see anything to suggest otherwise. Could you confirm?

thanks!

Ben

[MarvinTeichmann]

Yes it does. This is usually useful to stimulate co-adaptation of features. In addition, gradients arriving at early layers are much smaller than at higher layers. So in practice early layers will not chance much.

[BenMacKenzie]

got it. thanks!