Can't get the cuda test working !

[Emie31]

Hello !

Thanks a lot for the nice code you provide ! I've just installed it, and I want to use the cuda version of it.. So I first use the cmake option -DBUILD_CUDA_LIB, and I add to change the CUDA_NVCC_FLAGS so that it actually compiles with my architecture (-arch=sm_20). Now I try to get the tests working, but I'm running into an issue the HDF5 lib.. and I don't know how to deal with it !

Here is the error I get:

[100%] Runnint gtest test(s) flann_cuda_test
[==========] Running 8 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from Flann_3D
[ RUN      ] Flann_3D.KDTreeSingleTest
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.KDTreeSingleTest (3 ms)
[ RUN      ] Flann_3D.KDTreeCudaTest
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.KDTreeCudaTest (2 ms)
[ RUN      ] Flann_3D.TestRadiusSearch
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.TestRadiusSearch (0 ms)
[----------] 3 tests from Flann_3D (5 ms total)

[----------] 5 tests from Flann_3D_Random_Cloud
[ RUN      ] Flann_3D_Random_Cloud.Test1NN
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.797032 seconds)
Building kd-tree index...done (0.028326 seconds)
Searching KNN...done (0.000142 seconds)
/home/guy/builds/flann/test/flann_cuda_test.cu:252: Failure
Expected: (precision) >= (0.99), actual: 0 vs 0.99
Precision: 0
[  FAILED  ] Flann_3D_Random_Cloud.Test1NN (832 ms)
[ RUN      ] Flann_3D_Random_Cloud.Test4NN
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.795608 seconds)
Building kd-tree index...done (0.004154 seconds)
Searching KNN...done (0.000139 seconds)
/home/guy/builds/flann/test/flann_cuda_test.cu:272: Failure
Expected: (precision) >= (0.99), actual: 5e-05 vs 0.99
Precision: 5e-05
[  FAILED  ] Flann_3D_Random_Cloud.Test4NN (801 ms)
[ RUN      ] Flann_3D_Random_Cloud.Test4NNGpuBuffers
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.794307 seconds)
Building kd-tree index...done (0.003875 seconds)
Searching KNN...Segmentation fault (core dumped)
make[3]: *** [test/CMakeFiles/test_flann_cuda_test] Error 139
make[2]: *** [test/CMakeFiles/test_flann_cuda_test.dir/all] Error 2
make[1]: *** [test/CMakeFiles/test_flann_cuda_test.dir/rule] Error 2
make: *** [test_flann_cuda_test] Error 2

I've tried to install HDF5 using apt-get install libhdf5-mpi-dev, and also by compiling the source (from the library github) but nothing worked!

What am I missing ? I'm really new to this lib.. so I guess I missed some stupid step somewhere...
Could you help me on how to have this test working ?

Many Thanks!

[hoding123]

Hi. It seems that you have written to the wrong guy. Sorry for not being
able to help.

Good luck with you hdf5 installation!

2015-05-13 18:42 GMT+08:00 Emie31 [email protected]:

Hello !

Thanks a lot for the nice code you provide ! I've just installed it, and I
want to use the cuda version of it.. So I first use the cmake option
-DBUILD_CUDA_LIB, and I add to change the CUDA_NVCC_FLAGS so that it
actually compiles with my architecture (-arch=sm_20). Now I try to get the
tests working, but I'm running into an issue the HDF5 lib.. and I don't
know how to deal with it !

Here is the error I get:

[100%] Runnint gtest test(s) flann_cuda_test
[==========] Running 8 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from Flann_3D
[ RUN ] Flann_3D.KDTreeSingleTest
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
#000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
major: Dataset
minor: Object not found
#1: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
major: Symbol table
minor: Object not found
#2: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found
#3: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed
#4: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
major: Symbol table
minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[ FAILED ] Flann_3D.KDTreeSingleTest (3 ms)
[ RUN ] Flann_3D.KDTreeCudaTest
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
#000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
major: Dataset
minor: Object not found
#1: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
major: Symbol table
minor: Object not found
#2: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found
#3: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed
#4: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
major: Symbol table
minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[ FAILED ] Flann_3D.KDTreeCudaTest (2 ms)
[ RUN ] Flann_3D.TestRadiusSearch
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
#000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
major: Dataset
minor: Object not found
#1: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
major: Symbol table
minor: Object not found
#2: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
major: Symbol table
minor: Object not found
#3: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
major: Symbol table
minor: Callback failed
#4: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
major: Symbol table
minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[ FAILED ] Flann_3D.TestRadiusSearch (0 ms)
[----------] 3 tests from Flann_3D (5 ms total)

[----------] 5 tests from Flann_3D_Random_Cloud
[ RUN ] Flann_3D_Random_Cloud.Test1NN
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.797032 seconds)
Building kd-tree index...done (0.028326 seconds)
Searching KNN...done (0.000142 seconds)
/home/guy/builds/flann/test/flann_cuda_test.cu:252: Failure
Expected: (precision) >= (0.99), actual: 0 vs 0.99
Precision: 0
[ FAILED ] Flann_3D_Random_Cloud.Test1NN (832 ms)
[ RUN ] Flann_3D_Random_Cloud.Test4NN
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.795608 seconds)
Building kd-tree index...done (0.004154 seconds)
Searching KNN...done (0.000139 seconds)
/home/guy/builds/flann/test/flann_cuda_test.cu:272: Failure
Expected: (precision) >= (0.99), actual: 5e-05 vs 0.99
Precision: 5e-05
[ FAILED ] Flann_3D_Random_Cloud.Test4NN (801 ms)
[ RUN ] Flann_3D_Random_Cloud.Test4NNGpuBuffers
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.794307 seconds)
Building kd-tree index...done (0.003875 seconds)
Searching KNN...Segmentation fault (core dumped)
make[3]: *** [test/CMakeFiles/test_flann_cuda_test] Error 139
make[2]: *** [test/CMakeFiles/test_flann_cuda_test.dir/all] Error 2
make[1]: *** [test/CMakeFiles/test_flann_cuda_test.dir/rule] Error 2
make: *** [test_flann_cuda_test] Error 2

I've tried to install HDF5 using apt-get install libhdf5-mpi-dev, and also
by compiling the source (from the library github) but nothing worked!

What am I missing ? I'm really new to this lib.. so I guess I missed some
stupid step somewhere...
Could you help me on how to have this test working ?

Many Thanks!

—
Reply to this email directly or view it on GitHub
#240.

[Emie31]

Hi again,

Sorry, for my unclear question ! I'm trying to use the GPU version of flann to get all the 3D points inside a sphere. The CPU version of the library is working well, but I can't get the GPU version working.

for the GPU version In my code I just add :

#define FLANN_USE_CUDA
#include <flann/flann.h>
std::vector<double> v_ptr(vertices.size() * 3);
    for (unsigned int i = 0; i < vertices.size(); i++)
    {
        for (unsigned int j = 0; j < 3; j++)
        {
            v_ptr[i * 3 + j] = vertices[i].p()[j];
        }
    }
    flann::Matrix<double> dataset(&v_ptr[0], vertices.size(), 3);
    flann::Matrix<double> query(&v_ptr[0], vertices.size(), 3);

    flann::Matrix<int> indices(new int[query.rows * 3], query.rows, 3);
    flann::Matrix<double> dists(new double[query.rows * 3], query.rows, 3);
    // construct an randomized kd-tree index using 4 kd-trees
    flann::Index<L2_Simple<double> > index(dataset, flann::KDTreeCuda3dIndexParams());
    index.buildIndex();

and I linked to libflann, libflann_cpp and libflann_cuda but I keep getting the 2 following errors:

In function `flann::KDTreeCuda3dIndex<flann::L2_Simple<double> >::~KDTreeCuda3dIndex()':
tmpxft_0000492e_00000000-4_Cuda_GLS.cudafe1.cpp:(.text._ZN5flann17KDTreeCuda3dIndexINS_9L2_SimpleIdEEED2Ev[_ZN5flann17KDTreeCuda3dIndexINS_9L2_SimpleIdEEED5Ev]+0x3f): undefined reference to `flann::KDTreeCuda3dIndex<flann::L2_Simple<double> >::clearGpuBuffers()'
In function `flann::KDTreeCuda3dIndex<flann::L2_Simple<double> >::buildIndex()':
tmpxft_0000492e_00000000-4_Cuda_GLS.cudafe1.cpp:(.text._ZN5flann17KDTreeCuda3dIndexINS_9L2_SimpleIdEEE10buildIndexEv[_ZN5flann17KDTreeCuda3dIndexINS_9L2_SimpleIdEEE10buildIndexEv]+0xaf): undefined reference to `flann::KDTreeCuda3dIndex<flann::L2_Simple<double> >::uploadTreeToGpu()'

So I was guessing the cuda library might not be working. To check that I compile the flann_cuda_test (and it compiles fine). However when I try to run it, the test doesn't pass:

guy@chewbacca:~/builds/flann/test$ ./flann_cuda_test 
[==========] Running 8 tests from 2 test cases.
[----------] Global test environment set-up.
[----------] 3 tests from Flann_3D
[ RUN      ] Flann_3D.KDTreeSingleTest
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.KDTreeSingleTest (7 ms)
[ RUN      ] Flann_3D.KDTreeCudaTest
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.KDTreeCudaTest (2 ms)
[ RUN      ] Flann_3D.TestRadiusSearch
Reading test data...HDF5-DIAG: Error detected in HDF5 (1.8.11) thread 0:
  #000: ../../../src/H5D.c line 334 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: ../../../src/H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: ../../../src/H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: ../../../src/H5Gloc.c line 385 in H5G_loc_find_cb(): object 'indices' doesn't exist
    major: Symbol table
    minor: Object not found
unknown file: Failure
C++ exception with description "Error opening dataset in file." thrown in SetUp().
[  FAILED  ] Flann_3D.TestRadiusSearch (1 ms)
[----------] 3 tests from Flann_3D (10 ms total)

[----------] 5 tests from Flann_3D_Random_Cloud
[ RUN      ] Flann_3D_Random_Cloud.Test1NN
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.727318 seconds)
Building kd-tree index...done (0.035195 seconds)
Searching KNN...done (0.000147 seconds)
/home/guy/builds/flann/test/flann_cuda_test.cu:252: Failure
Expected: (precision) >= (0.99), actual: 0 vs 0.99
Precision: 0
[  FAILED  ] Flann_3D_Random_Cloud.Test1NN (764 ms)
[ RUN      ] Flann_3D_Random_Cloud.Test4NN
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.727468 seconds)
Building kd-tree index...done (0.004335 seconds)
Searching KNN...done (0.000138 seconds)
/home/guy/builds/flann/test/flann_cuda_test.cu:272: Failure
Expected: (precision) >= (0.99), actual: 2.5e-05 vs 0.99
Precision: 2.5e-05
[  FAILED  ] Flann_3D_Random_Cloud.Test4NN (733 ms)
[ RUN      ] Flann_3D_Random_Cloud.Test4NNGpuBuffers
creating random point cloud (10000 points)...done
Building linear index...done (1e-06 seconds)
Searching KNN...done (0.725263 seconds)
Building kd-tree index...done (0.004063 seconds)
Searching KNN...Segmentation fault (core dumped)

but it looks like it is a different issue.. Do you have any idea of what I'm doing wrong and on how I can get the cuda lib working ?

Thanks,
Emilie.

[Emie31]

Sorry for all the messages.. I finally found my mistake in my code.
We cannot used the cuda kdtree with double, but only with float (as there is no implementation of the class for the double template in the library)
so I changed my code so that :

 flann::Index<L2_Simple<double> > index(dataset, flann::KDTreeCuda3dIndexParams());

becomes :

 flann::Index<L2_Simple<float> > index(dataset, flann::KDTreeCuda3dIndexParams());

and it compiles now.

[alexsusu]

Hello,
I followed your advice from this thread and I was able to compile the flann CUDA library.

Basically I gave:
  unset CPLUS_INCLUDE_PATH
  export CUDA_NVCC_FLAGS=-arch=sm_20
  cmake -DBUILD_CUDA_LIB=ON .
  make

(I also need to have installed Google Test framework - e.g., from https://github.com/google/googletest )

I am curious - if I understand correctly (from the source file name kdtree_cuda_3d_index.cu, etc) the KDtree is written for 3 dimensions. Can I easily extend it to 4 dimensions?

Thanks,
Alex

[Leo-LiHao]

Hello, I got the same error!
I found that in flann_cuda_test.cu:
117 flann::load_from_file(match,"cloud.h5","indices");
but

$ h5ls -r cloud.h5 
/                        Group 
/dataset                 Dataset {177264, 3}
/dataset_padded          Dataset {177264, 5}
/distances               Dataset {19696, 5}
/match                   Dataset {19696, 5}
/query                   Dataset {19696, 3}

There doesn't exist 'indices'