Testing depends on remote content fetching

[ghisvail]

I successfully managed to build the latest tag (0.10.1.0) of clSPARSE in Debian and was looking for a way to run the testsuite from the corresponding tarball.

However, it seems the testsuite requires additional files for its execution. These are not provided in the tarball and are probably downloading by the SuperBuild (correct me if I am wrong). This is an issue for Debian packaging, since the policy forbids remote fetching of content during the build process to avoid potential tempering.

I am not asking for the complete testsuite to be re-engineered, but I am sure some unit testing could be achieved with generated data. This would allows testing and CI be performed in Debian and other Linux distributions. Please consider making the core unit testing part of the testsuite independent of remote content fetching.

[kknox]

HI @ghisvail
You are correct; the superbuild has cmake code to fetch input data. The data is typically measured in megabytes, but can go into gigabytes, so it's best left out of packaging.

I hear your request and it's a valid suggestion, but it's not a task I can commit to right now. I am of the mind now that our unit test and bench programs are of limited use in packages, I'm thinking that I should probably default BUILD_TESTS and BUILD_BENCHMARKS to OFF. If I look at more common libraries like FFTW, Boost or googletest, they do not ship their unit tests in their packages. We behave abnormally by defaulting those on.

Does your packaging problem implicitly go away if you don't build the tests/benches?

[ghisvail]

The data is typically measured in megabytes

Which would be fine in terms of size to embed in the source tree...

but can go into gigabytes

less fine...

Does your packaging problem implicitly go away if you don't build the tests/benches?

Well until now for, I had testing turned off clBLAS, clFFT and now clSPARSE because the OpenCL stack in Debian was behind Now we caught up. And with latest version of pocl, we can do CI-testing of our OpenCL packages. such as ArrayFire and the clMath libraries, against the rest of the archive.

Debian and other Linux distribution are actively pushing towards getting more and more of their respective archive being tested. The level of coverage however does not matter too much. Hence, myself asking whether part of the testsuite could be made data independent. They don't make testing a requirement (yet), but the trend seems to be going in that direction.

[ghisvail]

Perhaps embedding some representative sample .mtx files to play along with the executables under clsparse/samples could be enough for testing on our end. Hopefully, this should not require too much work on your end, as opposed to reorganizing the codebase of the testsuite.

What do you think?

[bkmgit]

Hi, Trying to build for Fedora and put in repositories. Have similar issue with remote dependencies, in particular OpenCL-CLHPP which itself uses a different testing infrastructure. Is it possible to package the necessary header files directly with clSPARSE?

[pavanky]

@bkmgit cl2.hpp should ideally be packaged into fedora as well.

[kknox]

Hi @ghisvail
My recommendation is to proceed with turning testing off for now.

@bkmgit
I concur with pavanky.