Shutdown fixes

[rustyrussell]

Builds on #841

This is basically a rewrite of closingd to do negotiation properly. Doing so revealed a spec bug, so I've created a patch for that too, and assumed it's going to be accepted. If not, we don't really care; by default we just don't enforce sane negotiation anyway.

[rustyrussell]

Looks like this needs more testing...

[ZmnSCPxj]

Fixes: #801

[cdecker]

Has some valgrind errors:

------------------------------- Valgrind errors --------------------------------
Valgrind error file: valgrind-errors.5732
==5732== Invalid read of size 8
==5732==    at 0x4149FD: remove_cmd_from_hout (pay.c:292)
==5732==    by 0x468BAB: notify (tal.c:237)
==5732==    by 0x469077: del_tree (tal.c:400)
==5732==    by 0x4690C7: del_tree (tal.c:410)
==5732==    by 0x46948A: tal_free (tal.c:509)
==5732==    by 0x40F1EA: main (lightningd.c:362)
==5732==  Address 0x69df148 is 1,512 bytes inside a block of size 1,544 free'd
==5732==    at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5732==    by 0x469150: del_tree (tal.c:421)
==5732==    by 0x46948A: tal_free (tal.c:509)
==5732==    by 0x4198F2: free_htlcs (peer_control.c:1281)
==5732==    by 0x40EBA9: shutdown_subdaemons (lightningd.c:209)
==5732==    by 0x40F1DE: main (lightningd.c:360)
==5732==  Block was alloc'd at
==5732==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5732==    by 0x468C30: allocate (tal.c:250)
==5732==    by 0x4691F7: tal_alloc_ (tal.c:448)
==5732==    by 0x40A279: new_htlc_out (htlc_end.c:143)
==5732==    by 0x41FD64: send_htlc_out (peer_htlcs.c:397)
==5732==    by 0x41511C: send_payment (pay.c:388)
==5732==    by 0x41589E: json_sendpay (pay.c:513)
==5732==    by 0x40D9B1: parse_request (jsonrpc.c:600)
==5732==    by 0x40DCAC: read_json (jsonrpc.c:667)
==5732==    by 0x45C706: next_plan (io.c:59)
==5732==    by 0x45D1DD: do_plan (io.c:387)
==5732==    by 0x45D21B: io_ready (io.c:397)
==5732== 
==5732== Invalid write of size 8
==5732==    at 0x414A27: remove_cmd_from_hout (pay.c:293)
==5732==    by 0x468BAB: notify (tal.c:237)
==5732==    by 0x469077: del_tree (tal.c:400)
==5732==    by 0x4690C7: del_tree (tal.c:410)
==5732==    by 0x46948A: tal_free (tal.c:509)
==5732==    by 0x40F1EA: main (lightningd.c:362)
==5732==  Address 0x69df148 is 1,512 bytes inside a block of size 1,544 free'd
==5732==    at 0x4C2EDEB: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5732==    by 0x469150: del_tree (tal.c:421)
==5732==    by 0x46948A: tal_free (tal.c:509)
==5732==    by 0x4198F2: free_htlcs (peer_control.c:1281)
==5732==    by 0x40EBA9: shutdown_subdaemons (lightningd.c:209)
==5732==    by 0x40F1DE: main (lightningd.c:360)
==5732==  Block was alloc'd at
==5732==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5732==    by 0x468C30: allocate (tal.c:250)
==5732==    by 0x4691F7: tal_alloc_ (tal.c:448)
==5732==    by 0x40A279: new_htlc_out (htlc_end.c:143)
==5732==    by 0x41FD64: send_htlc_out (peer_htlcs.c:397)
==5732==    by 0x41511C: send_payment (pay.c:388)
==5732==    by 0x41589E: json_sendpay (pay.c:513)
==5732==    by 0x40D9B1: parse_request (jsonrpc.c:600)
==5732==    by 0x40DCAC: read_json (jsonrpc.c:667)
==5732==    by 0x45C706: next_plan (io.c:59)
==5732==    by 0x45D1DD: do_plan (io.c:387)
==5732==    by 0x45D21B: io_ready (io.c:397)
==5732==

This is test_closing_different_fees, the failing node is l1 with port 16331.

[rustyrussell]

Yeah, suspect that's an unrelated issue, but I've fixed it once and for all, with the final commit.

And rebased...

[cdecker]

Had to fix a small conflict, will merge if CI is happy.