unit test failure on command line #366

tylerjereddy · 2015-07-25T14:51:14Z

For commit: ce8657d

Passes on travis but fails on Mac OS X command line when run from path mdanalysis/testsuite/MDAnalysisTests:

nosetests --processes=8 --process-timeout=120

final report:

Ran 561 tests in 32.567s

FAILED (SKIP=3, errors=42, failures=101)

Sample failure snippet:

FAIL: test_coordinates (MDAnalysisTests.test_coordinates.TestXTCNoConversion)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/treddy/python_modules/MDAnalysis/MDA_dev/mdanalysis/testsuite/MDAnalysisTests/test_coordinates.py", line 2485, in test_coordinates
    assert_equal(self.ts.frame, 2, "failed to step to frame 3")
  File "/Users/treddy/anaconda/lib/python2.7/site-packages/numpy/testing/utils.py", line 334, in assert_equal
    raise AssertionError(msg)
AssertionError: 
Items are not equal: failed to step to frame 3
 ACTUAL: 3
 DESIRED: 2

Can anyone reproduce this on a mac with the latest development branch? There were a lot of recent changes to unit testing framework, but my impression is that they were done quite carefully to avoid inhibiting the usual command line behaviour.

The text was updated successfully, but these errors were encountered:

richardjgowers · 2015-07-25T18:02:29Z

That looks like it's issue #310 not working, which is strange.

On 3:51PM, Sat, 25 Jul 2015 Tyler Reddy [email protected] wrote:

For commit: ce8657d

Passes on travis but fails on Mac OS X command line when run from path
mdanalysis/testsuite/MDAnalysisTests:

nosetests --processes=8 --process-timeout=120

final report:

Ran 561 tests in 32.567s

FAILED (SKIP=3, errors=42, failures=101)

Sample failure snippet:

FAIL: test_coordinates (MDAnalysisTests.test_coordinates.TestXTCNoConversion)

Traceback (most recent call last):
File "/Users/treddy/python_modules/MDAnalysis/MDA_dev/mdanalysis/testsuite/MDAnalysisTests/test_coordinates.py", line 2485, in test_coordinates
assert_equal(self.ts.frame, 2, "failed to step to frame 3")
File "/Users/treddy/anaconda/lib/python2.7/site-packages/numpy/testing/utils.py", line 334, in assert_equal
raise AssertionError(msg)
AssertionError:
Items are not equal: failed to step to frame 3
ACTUAL: 3
DESIRED: 2

Can anyone reproduce this on a mac with the latest development branch?
There were a lot of recent changes to unit testing framework, but my
impression is that they were done quite carefully to avoid inhibiting the
usual command line behaviour.

—
Reply to this email directly or view it on GitHub
#366.

orbeckst · 2015-07-25T21:57:49Z

I just did a parallel test on Max OS X 10.6.8 as

./mda_nosetests -v --processes=2 --process-timeout=120

and got 10 failures in test_timestep_api.py whereas travis-ci does not flag any of them (note: serial testing gives the same failures):

======================================================================
FAIL: MDAnalysisTests.test_timestep_api.TestTimestepEquality.test_check_equal
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 729, in test_check_equal
    assert_equal(ts1, ts2)
  File "/Users/oliver/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal:
 ACTUAL: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >
 DESIRED: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >

======================================================================
FAIL: MDAnalysisTests.test_timestep_api.TestTimestepEquality.test_check_forces
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 839, in test_check_forces
    assert_equal(ts1, ts2)
  File "/Users/oliver/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal:
 ACTUAL: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >
 DESIRED: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >

======================================================================
FAIL: MDAnalysisTests.test_timestep_api.TestTimestepEquality.test_check_vels
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 810, in test_check_vels
    assert_equal(ts1, ts2)
  File "/Users/oliver/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal:
 ACTUAL: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >
 DESIRED: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >

======================================================================
FAIL: MDAnalysisTests.test_timestep_api.TestTimestepEquality.test_other_timestep(< Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, "Failed on <class 'MDAnalysis.coordinates.DCD.Timestep'>")
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 733, in _check_ts
    assert_equal(a, b, err_msg=err_msg)
  File "/Users/oliver/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal: Failed on <class 'MDAnalysis.coordinates.DCD.Timestep'>
 ACTUAL: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >
 DESIRED: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >

======================================================================
FAIL: MDAnalysisTests.test_timestep_api.TestTimestepEquality.test_other_timestep(< Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, "Failed on <class 'MDAnalysis.coordinates.TRJ.Timestep'>")
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 733, in _check_ts
    assert_equal(a, b, err_msg=err_msg)
  File "/Users/oliver/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal: Failed on <class 'MDAnalysis.coordinates.TRJ.Timestep'>
 ACTUAL: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >
 DESIRED: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >

======================================================================
FAIL: MDAnalysisTests.test_timestep_api.TestTimestepEquality.test_other_timestep(< Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, "Failed on <class 'MDAnalysis.coordinates.DMS.Timestep'>")
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 733, in _check_ts
    assert_equal(a, b, err_msg=err_msg)
  File "/Users/oliver/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal: Failed on <class 'MDAnalysis.coordinates.DMS.Timestep'>
 ACTUAL: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >
 DESIRED: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >

======================================================================
FAIL: MDAnalysisTests.test_timestep_api.TestTimestepEquality.test_other_timestep(< Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, "Failed on <class 'MDAnalysis.coordinates.GRO.Timestep'>")
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 733, in _check_ts
    assert_equal(a, b, err_msg=err_msg)
  File "/Users/oliver/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal: Failed on <class 'MDAnalysis.coordinates.GRO.Timestep'>
 ACTUAL: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >
 DESIRED: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >

======================================================================
FAIL: MDAnalysisTests.test_timestep_api.TestTimestepEquality.test_other_timestep(< Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, "Failed on <class 'MDAnalysis.coordinates.TRZ.Timestep'>")
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 733, in _check_ts
    assert_equal(a, b, err_msg=err_msg)
  File "/Users/oliver/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal: Failed on <class 'MDAnalysis.coordinates.TRZ.Timestep'>
 ACTUAL: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >
 DESIRED: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >

======================================================================
FAIL: MDAnalysisTests.test_timestep_api.TestTimestepEquality.test_other_timestep(< Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >, "Failed on <class 'MDAnalysis.coordinates.xdrfile.core.Timestep'>")
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 733, in _check_ts
    assert_equal(a, b, err_msg=err_msg)
  File "/Users/oliver/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal:
Failed on <class 'MDAnalysis.coordinates.xdrfile.core.Timestep'>
 ACTUAL: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >
 DESIRED: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >

======================================================================
FAIL: MDAnalysisTests.test_timestep_api.TestTimestepEquality.test_trr_timestep
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 761, in test_trr_timestep
    self._check_ts(ts1, ts2, "Failed on TRR Timestep")
  File "/Volumes/Data/oliver/Biop/Library/python/mdanalysis/testsuite/MDAnalysisTests/test_timestep_api.py", line 733, in _check_ts
    assert_equal(a, b, err_msg=err_msg)
  File "/Users/oliver/.local/lib/python2.7/site-packages/numpy/testing/utils.py", line 317, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal: Failed on TRR Timestep
 ACTUAL: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >
 DESIRED: < Timestep -1 with unit cell dimensions [ 0.  0.  0.  0.  0.  0.] >

----------------------------------------------------------------------
Ran 2028 tests in 445.089s

FAILED (KNOWNFAIL=8, failures=10)

richardjgowers · 2015-07-26T09:06:06Z

They're all the __eq__ method on a Timestep not working.

Does something like this work?

import numpy as np
from MDAnalysis.coordinates.base import Timestep

a = Timestep(10)
b = Timestep(10)

a.positions = np.arange(30).reshape(10, 3)
b.positions = np.arange(30).reshape(10, 3)

assert a == b

tylerjereddy · 2015-07-26T14:47:21Z

That code block works just fine for me Richard.

orbeckst · 2015-07-26T15:27:26Z

I'm currently traveling so I won't be able to test this immediately.

Am Jul 26, 2015 um 2:06 schrieb Richard Gowers [email protected]:

They're all the eq method on a Timestep not working.

Does something like this work?

import numpy as np
from MDAnalysis.coordinates.base import Timestep

a = Timestep(10)
b = Timestep(10)

a.positions = np.arange(30).reshape(10, 3)
b.positions = np.arange(30).reshape(10, 3)

assert a == b
—
Reply to this email directly or view it on GitHub.

orbeckst · 2015-07-26T23:56:46Z

Hi @richardjgowers ,

Yes, the code snippet below works (a == b evaluates to True).

Oliver

On 26 Jul, 2015, at 03:06, Richard Gowers wrote:

They're all the eq method on a Timestep not working.

Does something like this work?

import numpy as np

from MDAnalysis.coordinates.base import
Timestep

a = Timestep(10)
b = Timestep(10)

a.positions = np.arange(30).reshape(10, 3)
b.positions = np.arange(30).reshape(10, 3)

assert a == b

richardjgowers · 2015-07-27T09:34:43Z

So that's confusing because the first test that failed for you is basically the code snippet. Maybe it's a nose version issue? The test (line 729 of test_timestep_api) is currently assert_equal(ts1, ts2), if you change it to assert_equal(ts1 == ts2, True) that should definitely pass?

@tylerjereddy
Your test failure looks like you're running old code against the new tests. Timestep.frame was changed to 0 based, so u.trajectory[i] is frame i, not i+1

orbeckst · 2015-08-07T00:53:59Z

With the changes

        #assert_equal(ts1, ts2)
        assert_equal(ts1 == ts2, True)
        #assert_equal(ts2, ts1)
        assert_equal(ts2 == ts1, True)

the test passes.

Or I can use assert_(ts1 == ts2) and

    def _check_ts(self, a, b, err_msg):
        assert_(a == b, err_msg)
        assert_(b == a, err_msg)

Not sure what to make of this. I am using nosetests version 1.3.7 on Mac OS X 10.6.8.

richardjgowers · 2015-08-07T11:25:23Z

Maybe a mac specific bug? Easiest thing is just to add the fixes you've found.

- replaced assert_equal(ts1, ts2) with assert_(ts1 == ts2) and also assert_not_equal() with assert_( != ) - fixes #366 - tests in test_timestep_api.TestTimestepEquality failed on Mac OS X 10.6.8 python 2.7, numpy 1.8.1, nose 1.3.7 but passed in Travis CI and Linux; the fix makes them pass everywhere

tylerjereddy added the testing label Jul 25, 2015

orbeckst mentioned this issue Aug 8, 2015

fixed spurious test failures in Timestep API tests #390

Merged

orbeckst self-assigned this Aug 8, 2015

orbeckst added this to the 0.11 milestone Aug 8, 2015

richardjgowers closed this as completed in #390 Aug 8, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

unit test failure on command line #366

unit test failure on command line #366

tylerjereddy commented Jul 25, 2015

richardjgowers commented Jul 25, 2015

FAIL: test_coordinates (MDAnalysisTests.test_coordinates.TestXTCNoConversion)

orbeckst commented Jul 25, 2015

richardjgowers commented Jul 26, 2015

tylerjereddy commented Jul 26, 2015

orbeckst commented Jul 26, 2015

orbeckst commented Jul 26, 2015

richardjgowers commented Jul 27, 2015

orbeckst commented Aug 7, 2015

richardjgowers commented Aug 7, 2015

unit test failure on command line #366

unit test failure on command line #366

Comments

tylerjereddy commented Jul 25, 2015

richardjgowers commented Jul 25, 2015

FAIL: test_coordinates (MDAnalysisTests.test_coordinates.TestXTCNoConversion)

orbeckst commented Jul 25, 2015

richardjgowers commented Jul 26, 2015

tylerjereddy commented Jul 26, 2015

orbeckst commented Jul 26, 2015

orbeckst commented Jul 26, 2015

richardjgowers commented Jul 27, 2015

orbeckst commented Aug 7, 2015

richardjgowers commented Aug 7, 2015