Skip to content

Home

Jump to bottom
Laura Murgatroyd edited this page Feb 22, 2023 · 4 revisions

Welcome to the CCPi-Framework wiki!

Machine precision

Some calculations should return the same value but are different due to machine precision.

With Python is not so clear what type the numbers are stored in. Additionally NumPy seems to use different strategies. We have encountered problems related to reductions like calculating the norm of an array (possibly multidimensional).

One would expect the following to return the same number for 1D arrays (just for simplicity):

a = numpy.asarray([some array], dtype=numpy.float32)
res1 = (a**2).sum()
res2 = a.dot(a)
res3 = functools.reduce(lambda x,y: x + y*y, a, 0)

Now, NumPy stores the result of sum in a container of the same type of the array it's summing. It is advisable, instead, to use a container of higher precision if possible. In the third method Python seems to be smart enough to store the result in a float64 by itself.

See https://github.com/vais-ral/CCPi-Framework/pull/273 and https://github.com/vais-ral/CCPi-Framework/issues/292

Clone this wiki locally