ask HowTo do the training and label prediction for CamVid dataset

[amiltonwong]

Hi, @alexgkendall ,

I had installed caffe-segnet successfully. Now I want to practice some examples such as training CamVid dataset which was discussed in paper (SegNet). Would you list the steps and corresponding command for achieving the training and label prediction for CamVid dataset?

Thx~
Milton

[alexgkendall]

Hi Milton,

Have you tried following the instructions in the readme.md file? At which stage did you get stuck in setting up the dataset and training the network?

Cheers,
Alex

[amiltonwong]

Hi , Alex,

I followed the readme.md,

1. First I download dataset from http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamSeq01/������������������������������������������������������������
   Resize them into 480X360 resolution. Put the images under '/root/caffe-segnet/CamVid/CamSeq01/images' and labels under '/root/caffe-segnet/CamVid/CamSeq01/labels'
2. Generate data.txt in the following format:
   CamVid/CamSeq01/images/0016E5_07959.png CamVid/CamSeq01/labels/0016E5_07959_L.png
   CamVid/CamSeq01/images/0016E5_07961.png CamVid/CamSeq01/labels/0016E5_07961_L.png
   CamVid/CamSeq01/images/0016E5_07963.png CamVid/CamSeq01/labels/0016E5_07963_L.png
   CamVid/CamSeq01/images/0016E5_07965.png CamVid/CamSeq01/labels/0016E5_07965_L.png
   CamVid/CamSeq01/images/0016E5_07967.png CamVid/CamSeq01/labels/0016E5_07967_L.png
   ...
3. Modify the last convolution section in file net.prototxt as follows :
   convolution_param {
   num_output: 32 # EDIT ME: set to number of classes
   kernel_size: 1
   weight_filler {
   type: "gaussian"
   std: 0.1
4. Then, I run the training command but came across this issue
   F0828 08:18:06.743408 2925 math_functions.cu:123] Check failed: status == CUBLAS_STATUS_SUCCESS (14 vs. 0) CUBLAS_STATUS_INTERNAL_ERROR

Here is my detailed training log
https://goo.gl/o4zVVt

Which steps am I missing?

Thx~

Milton

[alexgkendall]

Hi Milton, I believe this is a bug with Caffe. See BVLC/caffe#2334 for more details. Their solution is to recompile the code with the following commands:

make clean
make all -j8
make pycaffe
make runtest

Does that work for you?

[kezbreen]

Hi @amiltonwong ,

Another possibility is the label format. Caffe-segnet takes single channel (grayscale) label images as input, so there's an extra conversion step required for the CamVid dataset. Is your label data in this format? If not you can use this script to do the conversions:

#!/usr/bin/env python
import os
import numpy as np
from itertools import izip
from argparse import ArgumentParser
from collections import OrderedDict
from skimage.io import ImageCollection, imsave
from skimage.transform import resize


camvid_colors = OrderedDict([
    ("Animal", np.array([64, 128, 64], dtype=np.uint8)),
    ("Archway", np.array([192, 0, 128], dtype=np.uint8)),
    ("Bicyclist", np.array([0, 128, 192], dtype=np.uint8)),
    ("Bridge", np.array([0, 128, 64], dtype=np.uint8)),
    ("Building", np.array([128, 0, 0], dtype=np.uint8)),
    ("Car", np.array([64, 0, 128], dtype=np.uint8)),
    ("CartLuggagePram", np.array([64, 0, 192], dtype=np.uint8)),
    ("Child", np.array([192, 128, 64], dtype=np.uint8)),
    ("Column_Pole", np.array([192, 192, 128], dtype=np.uint8)),
    ("Fence", np.array([64, 64, 128], dtype=np.uint8)),
    ("LaneMkgsDriv", np.array([128, 0, 192], dtype=np.uint8)),
    ("LaneMkgsNonDriv", np.array([192, 0, 64], dtype=np.uint8)),
    ("Misc_Text", np.array([128, 128, 64], dtype=np.uint8)),
    ("MotorcycleScooter", np.array([192, 0, 192], dtype=np.uint8)),
    ("OtherMoving", np.array([128, 64, 64], dtype=np.uint8)),
    ("ParkingBlock", np.array([64, 192, 128], dtype=np.uint8)),
    ("Pedestrian", np.array([64, 64, 0], dtype=np.uint8)),
    ("Road", np.array([128, 64, 128], dtype=np.uint8)),
    ("RoadShoulder", np.array([128, 128, 192], dtype=np.uint8)),
    ("Sidewalk", np.array([0, 0, 192], dtype=np.uint8)),
    ("SignSymbol", np.array([192, 128, 128], dtype=np.uint8)),
    ("Sky", np.array([128, 128, 128], dtype=np.uint8)),
    ("SUVPickupTruck", np.array([64, 128, 192], dtype=np.uint8)),
    ("TrafficCone", np.array([0, 0, 64], dtype=np.uint8)),
    ("TrafficLight", np.array([0, 64, 64], dtype=np.uint8)),
    ("Train", np.array([192, 64, 128], dtype=np.uint8)),
    ("Tree", np.array([128, 128, 0], dtype=np.uint8)),
    ("Truck_Bus", np.array([192, 128, 192], dtype=np.uint8)),
    ("Tunnel", np.array([64, 0, 64], dtype=np.uint8)),
    ("VegetationMisc", np.array([192, 192, 0], dtype=np.uint8)),
    ("Wall", np.array([64, 192, 0], dtype=np.uint8)),
    ("Void", np.array([0, 0, 0], dtype=np.uint8))
])


def convert_label_to_grayscale(im):
    out = (np.ones(im.shape[:2]) * 255).astype(np.uint8)
    for gray_val, (label, rgb) in enumerate(camvid_colors.items()):
        match_pxls = np.where((im == np.asarray(rgb)).sum(-1) == 3)
        out[match_pxls] = gray_val
    assert (out != 255).all(), "rounding errors or missing classes in camvid_colors"
    return out.astype(np.uint8)


def make_parser():
    parser = ArgumentParser()
    parser.add_argument(
        'label_dir',
        help="Directory containing all RGB camvid label images as PNGs"
    )
    parser.add_argument(
        'out_dir',
        help="""Directory to save grayscale label images.
        Output images have same basename as inputs so be careful not to
        overwrite original RGB labels""")
    return parser


if __name__ == '__main__':
    parser = make_parser()
    args = parser.parse_args()
    labs = ImageCollection(os.path.join(args.label_dir, "*"))
    os.makedirs(args.out_dir)
    for i, (inpath, im) in enumerate(izip(labs.files, labs)):
        print i + 1, "of", len(labs)
        # resize to caffe-segnet input size and preserve label values
        resized_im = (resize(im, (360, 480), order=0) * 255).astype(np.uint8)
        out = convert_label_to_grayscale(resized_im)
        outpath = os.path.join(args.out_dir, os.path.basename(inpath))
        imsave(outpath, out)

[amiltonwong]

Hi, Alex,

You're right, the labels is in original format, which contains rgb labels. Therefore, I should convert them into grayscale first. I use the code( https://goo.gl/gNx2nl ) you suggested above. I try it as follows:

cd ~/caffe-segnet/Camvid/CamSeq01
./convertGrayscale.py labels/ labels_gray/
However, after I run the command above, nothing happens and the mouse pointer became a strange '+' pointer (the picture shows here: https://goo.gl/n2o20s ) , it seems a screen area selector. And the convert process was not performed.

Which steps am I missing?

Thx~

Milton

[kezbreen]

@amiltonwong ,

If you're using the above script, I missed the #!/usr/bin/env python so either add it or run with python convert_camvid.py to get it to work

[amiltonwong]

Hi, Alex,

Yes, it works now and the conversion script is definitely useful. THX!
Here is my train log:
https://goo.gl/CeIL9C

[amiltonwong]

Hi, Alex,

Now I'm practicing the training for CamVid dataset. Do you know which sequence is evaluated in SegNet paper. As in the original SegNet paper, it says "367 training images of 360x480" for training, For the original dataset, it contains 701 images from "701_StillsRaw_full.zip", download link: http://web4.cs.ucl.ac.uk/staff/g.brostow/MotionSegRecData/files/
but I couldn't find any corresponding segmentation groundtruth labeled images from there.

However, I could find both raw images and corresponding segementation groundtruth (101 images) of camvid_seq1 from http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamSeq01/

So my second question is : Where is the repository for the corresponding segmentation groundtruth labeled image of those 701 images)?

Thanks in advance!
Milton

[alexgkendall]

The dataset is still available for download on the website. I found the data here:
http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/data/LabeledApproved_full.zip

All the relevant information, such as class color codes, is here:
http://web4.cs.ucl.ac.uk/staff/g.brostow/MotionSegRecData/

Cheers,
Alex

[defenceVT]

hello, @alexgkendall ,

I had run the segnet successfully. Now I want to training CamVid dataset , the Camvid dataset have 32 semantic classes, but the segnet just classify 11 or 12 classes, the label category is not equal to the output category, how it work ?

Thx~
kevin

[sriramvasu]

@defenceVT The segnet was trained on a smaller version of Camvid dataset with only 12 classes. If you want to train it on the complete camvid dataset, you might want to finetune it again.

[jay98]

@kezbreen I ran the script you provided here, but it gives me the following error:

Traceback (most recent call last):
File "convertGrayscale.py", line 79, in
out = convert_label_to_grayscale(resized_im)
File "convertGrayscale.py", line 52, in convert_label_to_grayscale
assert (out != 255).all(), "rounding errors or missing classes in camvid_colors"
AssertionError: rounding errors or missing classes in camvid_colors

I would really appreciate if you could reply asap I have to give a demo in 2 days

[jonVolta]

Hi Alex;
I followed the segnet-tutorial and the network works correctly. But when i tried to train my own data set of faces to segment 6 labels (like: eye,nose,mouse,hair,...) the network fails to train. it reads all my images and its labels then the moment start iteration it breaks and gives a bunch of strange errors that looks like memory crash. why is this happening?
please i really need your response asp. i have to compile my final year paper at the end of this month.

[tom-bu]

@jay98 it means that there are colors in your labeled RGB picture that aren't assigned a class in your conversion script

[mokii]

Where can I download the full Camvid images and labels? I only find the labels here:http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamVid/
can anyone send me the images?

[unnir]

@mokii http://mi.eng.cam.ac.uk/research/projects/VideoRec/CamSeq01/

[mokii]

@unnir Thanks very much@unnir. But there seems only 101 images.The whole set is 701_StillsRaw_full.zip

[unnir]

@mokii what about this repo https://github.com/mostafaizz/camvid?