Python dev test in non-dev mode failure: test_closing_while_disconnected

[jsarenik]

Issue and Steps to Reproduce

EDIT: Testing on latest master - i.e. 56046d4 as of today (2018-06-11)

When I configure without parameters and then make check there is a test error which is trying to run a developer test in a (default) non-developer mode. See the full log.

DEBUG:root:Calling dev-memleak with payload None
DEBUG:root:lightningd-2: 2018-06-11T13:31:42.945Z lightningd(29222): Adding block 184: 0985104458bb7daaedb4b70cb5130ba3f758c233f0b74014a9f0c148ab39e1ae
DEBUG:root:lightningd-1: 2018-06-11T13:31:43.169Z lightningd(29095):jcon fd 15: Failing: Unknown command 'dev-memleak'
DEBUG:root:Received response for dev-memleak call: {'jsonrpc': '2.0', 'error': {'code': -32601, 'message': "Unknown command 'dev-memleak'"}, 'id': 0}
DEBUG:root:Calling stop with payload None

getinfo output

Not relevant.

[jsarenik]

Full log from Alpine Linux at https://gist.github.com/jsarenik/36611568a1645b3ce1dea98aadf3797b (updated on 2018-06-11)

[cdecker]

Have you tried rerunning it? Some of these tests are rather flaky.

[jsarenik]

Yes, tried at least 3 times and it's always the same.

Here's what I am doing:

#!/bin/sh

set -x
rm -rf lightning
git clone https://github.com/rustyrussell/lightning -b guilt/configure
cd lightning
./configure --disable-developer --disable-valgrind
make -j4
make check

And here's the full log (after running above with 2>&1 | tee log): https://pastebin.com/dGfbyawM (or raw → https://pastebin.com/raw/dGfbyawM)

[jsarenik]

And the same happens also on Ubuntu 18.04. make check log: https://gist.github.com/jsarenik/557f9b2240461369579e76ad4ac91ce8

(log updated on 2016-06-11)

[cdecker]

On ubuntu do you have bitcoind installed?

FileNotFoundError: [Errno 2] No such file or directory: 'bitcoind': 'bitcoind'

And on alpine you seem to be executing with DEVELOPER=1, but didn't compile it with that flag:

DEBUG:root:lightningd-2: 2018-06-06T16:11:06.501Z lightningd(5123):jcon fd 15: Failing: Unknown command 'dev-memleak'

That can be fixed by running make DEVELOPER=1 clean default

[jsarenik]

Sorry for that. It seems I have pasted wrong logs. See this one for Ubuntu: https://gist.github.com/jsarenik/557f9b2240461369579e76ad4ac91ce8

Now I am running the tests on 4773176, but also before I have used the branch with ./configure script and started from a fresh close, so there should be no issues with DEVELOPER on command line. I am running it again on Alpine, if it shows again, it is something that needs to be fixed.

[jsarenik]

This is what I run currently to produce the log:

#!/bin/sh

set -x

{

rm -rf lightning
git clone https://github.com/ElementsProject/lightning.git
cd lightning
./configure --disable-developer --disable-valgrind
make -j4
make check

} 2>&1 | tee log

[jsarenik]

Yes, it's there: https://gist.github.com/jsarenik/557f9b2240461369579e76ad4ac91ce8#file-ubuntu-full-log-txt-L3234 and AFAIU it means that in non-developer mode a test is run which requires developer mode.

[jsarenik]

I am adding TIMEOUT=120 to the make check line as @rustyrussell suggested (https://gist.github.com/jsarenik/36611568a1645b3ce1dea98aadf3797b#gistcomment-2612293).

[jsarenik]

Here is latest Alpine Linux log (still without TIMEOUT=120): https://gist.github.com/jsarenik/e07be20339fb34f6710838a730b9b5a1

Errs on missing dev-memleakcommand.

[jsarenik]

Rusty, BTW it seems to me that the TIMEOUT variable is not propagated in the environment of Python tests.

diff --git a/Makefile b/Makefile
index 63113a17..eaada7d7 100644
--- a/Makefile
+++ b/Makefile
@@ -215,7 +215,7 @@ ifndef PYTEST
 	exit 1
 else
 # Explicitly hand DEVELOPER and VALGRIND so you can override on make cmd line.
-	PYTHONPATH=contrib/pylightning:$$PYTHONPATH TEST_DEBUG=1 DEVELOPER=$(DEVELOPER) VALGRIND=$(VALGRIND) $(PYTEST) tests/ $(PYTEST_OPTS)
+	PYTHONPATH=contrib/pylightning:$$PYTHONPATH TEST_DEBUG=1 DEVELOPER=$(DEVELOPER) VALGRIND=$(VALGRIND) TIMEOUT=$(TIMEOUT) $(PYTEST) tests/ $(PYTEST_OPTS)
 endif
 
 # Keep includes in alpha order.

[cdecker]

Nevermind, using `make check` should propagate the config.vars `DEVELOPER` to `pytest`.

[jsarenik]

Yes, that's the point of this issue actually.

[cdecker]

This should have been addressed by #1572 can you check please? The issue was that the tests that were split out were still relying on the environment variable, not the config.vars.

[jsarenik]

Sure. Checking now. Thank you for comment!

[jsarenik]

@cdecker Now I see an issue with test_lockin_between_restart. Both on Alpine and Ubuntu.

Shall I start a new issue and close this #1548 or continue here?

[jsarenik]

I am rerunning with TIMEOUT=120...

[jsarenik]

Closing. The lockin_between_restart test is not failing with increased TIMEOUT. Closing this issue and excuse me for false-positive noise.

[cdecker]

No problem :-)