Add steps to install multi-threaded OpenBLAS on Ubuntu

[kloudkl]

Multi-threaded OpenBLAS makes a huge performance difference. The benchmarks with and without it in comments to #16 demonstrated more than 5 times speed-up for boost-eigen and MKL on a machine with 4 Hyper-Threading CPU cores (supporting 8 threads).

This fixes #79.

[Yangqing]

Are you sure when using boost-eigen, you are compiling with multi-thread enabled? boost-eigen naturally comes with multithreaded gemm, which would probably account for most of the gain you are observing.

[Yangqing]

http://stackoverflow.com/questions/14783219/how-to-speed-up-eigen-librarys-matrix-product

[kloudkl]

To make it clear whether OpenBLAS or Eigen contributed to the performance improvements in the boost-eigen branch, three groups of benchmark experiments with different compilation flags are conducted using the lenet*.prototxt. For all the experiments, max iter is set to 3,000 and solver_mode is 0.

cf_id	compilation flags
1	-latlas -lcblas -fopenmp
2	-lopenblas
3	-lopenblas -fopenmp

To check the effects of threads number, three runtime environment variables combinations are tested.

rev_id	runtime environment variables
1	``
2	OPENBLAS_NUM_THREADS=4 OMP_NUM_THREADS=4
3	OPENBLAS_NUM_THREADS=8 OMP_NUM_THREADS=8

In all the experiments, max iter is set to 3,000 and solver_mode is set to 0 in lenet_solver.prototxt.

cf_id	rev_id	real time	user time	system time
1	1	500.638	500.559	0.328
1	2	501.15	501.37	0.26
2	1	99.787	230.694	166.238
2	2	99.42	228.74	166.25
2	3	100.56	232.78	166.66
3	1	99.915	231.802	165.206
3	2	99.34	229.79	165.15
3	3	99.73	232.86	163.89

Comparing the results of compilation flags 1 and 3, it is evident that the multi-threaded OpenBLAS runs about 5 times faster than the normal ATLAS. The similar performances of compilation flags 2 and 3 prove that enabling OpenMP for Eigen does not help at all in this setting.

[Yangqing]

I still do not think you are using the multithreaded version of eigen3.
With benchmarks as follows:

https://plafrim.bordeaux.inria.fr/doku.php?id=people:guenneba

it would be extremely unlikely that eigen itself is bad in multithreading.
Could you double-check with a gemm call that your eigen version is using
multiple threads (by e.g. looking at top)?

Again, using lenet is not a good idea to benchmark things, use
net_speed_test instead, which fits real-world use cases better.

Yangqing

On Fri, Feb 7, 2014 at 10:39 AM, kloudkl [email protected] wrote:

To make it clear whether OpenBLAS or Eigen contributed to the performance
improvements in the boost-eigen branch, three groups of benchmark
experiments with different compilation flags are conducted using the
lenet*.prototxt. For all the experiments, max iter is set to 3,000 and
solver_mode is 0.
cf_id compilation flags 1 -latlas -lcblas -fopenmp 2 -lopenblas 3 -lopenblas
-fopenmp

To check the effects of threads number, three runtime environment
variables combinations are tested.
rev_id runtime environment variables 1 `` 2 OPENBLAS_NUM_THREADS=4
OMP_NUM_THREADS=4 3 OPENBLAS_NUM_THREADS=8 OMP_NUM_THREADS=8

In all the experiments, max iter is set to 3,000 and solver_mode is set to
0 in lenet_solver.prototxt.
cf_id rev_id real time user time system time 1 1 500.638 500.559 0.328
1 2 501.15 501.37 0.26 2 1 99.787 230.694 166.238 2 2 99.42 228.74
166.25 2 3 100.56 232.78 166.66 3 1 99.915 231.802 165.206 3 2 99.34
229.79 165.15 3 3 99.73 232.86 163.89

Comparing the results of compilation flags 1 and 3, it is evident that the
multi-threaded OpenBLAS runs about 5 times faster than the normal ATLAS.
The similar performances of compilation flags 2 and 3 prove that enabling
OpenMP for Eigen does not help at all in this setting.

Reply to this email directly or view it on GitHubhttps://github.com//pull/80#issuecomment-34486255
.

[Yangqing]

I'd like to make my arguments clear:

(1) I am not comparing ATLAS with OpenBLAS - it is known that ATLAS is
inherently single-threaded. Eigen compiled with Atlas backend is not what I
mean here - I mean native implementations of blas inside eigen.

(2) small datasets like MNIST does not reflect actual use cases such as
imagenet. In imagenet experiments, more than 80% computation time is spent
on gemm, so it really boils down to the point whether gemm is parallelized
or not - and I believe eigen does have that parallelized. Please provide a
more detailed analysis where the speedup comes from and why, rather than an
end-to-end run (honestly, that may not reveal too much information).

Yangqing

On Fri, Feb 7, 2014 at 10:44 AM, Yangqing Jia [email protected] wrote:

I still do not think you are using the multithreaded version of eigen3.
With benchmarks as follows:

https://plafrim.bordeaux.inria.fr/doku.php?id=people:guenneba

it would be extremely unlikely that eigen itself is bad in multithreading.
Could you double-check with a gemm call that your eigen version is using
multiple threads (by e.g. looking at top)?

Again, using lenet is not a good idea to benchmark things, use
net_speed_test instead, which fits real-world use cases better.

Yangqing

On Fri, Feb 7, 2014 at 10:39 AM, kloudkl [email protected] wrote:

To make it clear whether OpenBLAS or Eigen contributed to the performance
improvements in the boost-eigen branch, three groups of benchmark
experiments with different compilation flags are conducted using the
lenet*.prototxt. For all the experiments, max iter is set to 3,000 and
solver_mode is 0.
cf_id compilation flags 1 -latlas -lcblas -fopenmp 2 -lopenblas 3 -lopenblas
-fopenmp

To check the effects of threads number, three runtime environment
variables combinations are tested.
rev_id runtime environment variables 1 `` 2 OPENBLAS_NUM_THREADS=4
OMP_NUM_THREADS=4 3 OPENBLAS_NUM_THREADS=8 OMP_NUM_THREADS=8

In all the experiments, max iter is set to 3,000 and solver_mode is set
to 0 in lenet_solver.prototxt.
cf_id rev_id real time user time system time 1 1 500.638 500.559 0.328
1 2 501.15 501.37 0.26 2 1 99.787 230.694 166.238 2 2 99.42 228.74
166.25 2 3 100.56 232.78 166.66 3 1 99.915 231.802 165.206 3 2 99.34
229.79 165.15 3 3 99.73 232.86 163.89

Comparing the results of compilation flags 1 and 3, it is evident that
the multi-threaded OpenBLAS runs about 5 times faster than the normal
ATLAS. The similar performances of compilation flags 2 and 3 prove that
enabling OpenMP for Eigen does not help at all in this setting.

Reply to this email directly or view it on GitHubhttps://github.com//pull/80#issuecomment-34486255
.

[Yangqing]

I looked at the code more closely and now I have a little clearer picture on what caused this. in caffe/util/math_functions.cpp the gemm calls are still made using cblas_gemm instead of the eigen function, making the framework effectively still using atlas rather than eigen to carry out gemm. I will close this issue and open a separate issue indicating this necessary change for boost-eigen. If you would like to do a more detailed comparison please feel free to. Thanks for finding this bug!

[shelhamer]

Thank you for all this benchmarking work!

[shelhamer]

INSTALL.md has been replaced with a pointer to the online installation documentation to avoid the overhead of duplication, so refer to #81.

[jeffhammond]

This statement is categorically false: "it is known that ATLAS is inherently single-threaded." ATLAS has been threaded for 5+ years

http://math-atlas.sourceforge.net/faq.html#tnum
http://math-atlas.sourceforge.net/timing/newThr395/index.html