Skip to content
This repository has been archived by the owner on Sep 12, 2023. It is now read-only.

Taker panics when triggering two contract setups in parallel #1049

Closed
da-kami opened this issue Jan 9, 2022 · 1 comment
Closed

Taker panics when triggering two contract setups in parallel #1049

da-kami opened this issue Jan 9, 2022 · 1 comment
Assignees
@thomaseizinger
Labels
bug Something isn't working
Milestone
0.4.0 - "Rickroll...

Comments

@da-kami
Copy link
Contributor

da-kami commented Jan 9, 2022 •
edited
Loading

What I did:

Taker and maker running locally. Click Long button twice and confirm. Accept both setups in maker.
The taker logs actually report insufficient funds - that is likely related to #965. The taker has sufficient funds - and the balance was displyed in the UI! The taker should have also had enough outputs to be used for creating another Cfd. The only reason I can imainge is, that the both setups try to pick the same output, that was locked by one setup and then not available in the other. I thought that #793 would have fixed that though.

The fact that the taker panics with Disconnected is defininitely worrying.

It seems the maker was unable to read a message from the taker and consequently closed the connection:

2022-01-09 19:11:09 ERROR daemon::maker_inc_connections: Failed to read incoming messages from taker taker_id=ba4a7001bb1debc4e355f971e7e87409281865691ae64cbd9c8cfac1a07da442
2022-01-09 19:11:09  INFO daemon::maker_inc_connections: Dropping connection

Maker logs:

2022-01-09 19:10:58  INFO rocket::server: POST /api/order/sell application/json:    
2022-01-09 19:10:58 TRACE daemon::maker_inc_connections: Sent new order: Some(OrderId(Hyphenated(4c7aeac8-da47-44e2-8d7b-4df46e3503d9))) taker_id=ba4a7001bb1debc4e355f971e7e87409281865691ae64cbd9c8cfac1a07da442
2022-01-09 19:10:59 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:02 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:02 DEBUG daemon::maker_cfd: Taker wants to take an order taker_id=ba4a7001bb1debc4e355f971e7e87409281865691ae64cbd9c8cfac1a07da442 quantity=100 order_id=4c7aeac8-da47-44e2-8d7b-4df46e3503d9
2022-01-09 19:11:02 TRACE daemon::maker_inc_connections: Sent new order: None taker_id=ba4a7001bb1debc4e355f971e7e87409281865691ae64cbd9c8cfac1a07da442
2022-01-09 19:11:02 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:04 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:07  INFO rocket::server: POST /api/cfd/87d06c92-7bf5-4f27-9c00-92d8b2f0a0dd/acceptOrder:    
2022-01-09 19:11:07 DEBUG daemon::maker_cfd: Maker accepts order order_id=87d06c92-7bf5-4f27-9c00-92d8b2f0a0dd
2022-01-09 19:11:07  INFO daemon::setup_maker: Maker accepts an order order_id=87d06c92-7bf5-4f27-9c00-92d8b2f0a0dd
2022-01-09 19:11:07  INFO daemon::setup_contract: Exchanged setup parameters
2022-01-09 19:11:07 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:07 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:08  INFO rocket::server: POST /api/cfd/4c7aeac8-da47-44e2-8d7b-4df46e3503d9/acceptOrder:    
2022-01-09 19:11:08 DEBUG daemon::maker_cfd: Maker accepts order order_id=4c7aeac8-da47-44e2-8d7b-4df46e3503d9
2022-01-09 19:11:08  INFO daemon::setup_maker: Maker accepts an order order_id=4c7aeac8-da47-44e2-8d7b-4df46e3503d9
2022-01-09 19:11:09 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:09 ERROR daemon::maker_inc_connections: Failed to read incoming messages from taker taker_id=ba4a7001bb1debc4e355f971e7e87409281865691ae64cbd9c8cfac1a07da442
2022-01-09 19:11:09  INFO daemon::maker_inc_connections: Dropping connection taker_id=ba4a7001bb1debc4e355f971e7e87409281865691ae64cbd9c8cfac1a07da442
2022-01-09 19:11:09  INFO rocket::server: POST /api/cfd/ab1e0e6b-14c3-4048-bd29-c53b6300eb5c/acceptOrder:    
2022-01-09 19:11:09 DEBUG daemon::maker_cfd: Maker accepts order order_id=ab1e0e6b-14c3-4048-bd29-c53b6300eb5c
2022-01-09 19:11:10  INFO daemon::setup_contract: Created CFD transactions
2022-01-09 19:11:12 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:13 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:14 TRACE daemon::monitor: Updating status of 3 transactions
2022-01-09 19:11:14 TRACE daemon::monitor: RefundFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have unseen
2022-01-09 19:11:14 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 1 remaining for this script
2022-01-09 19:11:14 TRACE daemon::monitor: LockFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have in mempool
2022-01-09 19:11:14 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 1 remaining for this script
2022-01-09 19:11:14 TRACE daemon::monitor: CommitFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have unseen
2022-01-09 19:11:14 TRACE daemon::monitor: CetTimelockExpired(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 12 blocks and we have unseen
2022-01-09 19:11:14 TRACE daemon::monitor: RefundTimelockExpired(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 216 blocks and we have unseen
2022-01-09 19:11:14 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 3 remaining for this script
2022-01-09 19:11:14 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:19 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:20 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:20 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:24 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:25 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:25 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:29 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:30 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:31 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:34 TRACE daemon::monitor: Updating status of 3 transactions
2022-01-09 19:11:34 TRACE daemon::monitor: RefundFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have unseen
2022-01-09 19:11:34 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 1 remaining for this script
2022-01-09 19:11:34 TRACE daemon::monitor: LockFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have in mempool
2022-01-09 19:11:34 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 1 remaining for this script
2022-01-09 19:11:34 TRACE daemon::monitor: CommitFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have unseen
2022-01-09 19:11:34 TRACE daemon::monitor: CetTimelockExpired(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 12 blocks and we have unseen
2022-01-09 19:11:34 TRACE daemon::monitor: RefundTimelockExpired(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 216 blocks and we have unseen
2022-01-09 19:11:34 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 3 remaining for this script
2022-01-09 19:11:34 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:36 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:36 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:39 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:41 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:41 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:44 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:46 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:47 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:49 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:52 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:52 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:54 TRACE daemon::monitor: Updating status of 3 transactions
2022-01-09 19:11:54 TRACE daemon::monitor: RefundFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have unseen
2022-01-09 19:11:54 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 1 remaining for this script
2022-01-09 19:11:54 TRACE daemon::monitor: LockFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have in mempool
2022-01-09 19:11:54 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 1 remaining for this script
2022-01-09 19:11:54 TRACE daemon::monitor: CommitFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have unseen
2022-01-09 19:11:54 TRACE daemon::monitor: CetTimelockExpired(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 12 blocks and we have unseen
2022-01-09 19:11:54 TRACE daemon::monitor: RefundTimelockExpired(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 216 blocks and we have unseen
2022-01-09 19:11:54 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 3 remaining for this script
2022-01-09 19:11:54 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:57 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:57 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:11:59 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:12:02 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:12:02 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:12:04 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:12:07 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:12:08 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:12:09 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:12:13 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:12:13 TRACE daemon::bitmex_price_feed: Received pong
2022-01-09 19:12:14 TRACE daemon::monitor: Updating status of 3 transactions
2022-01-09 19:12:14 TRACE daemon::monitor: RefundFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have unseen
2022-01-09 19:12:14 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 1 remaining for this script
2022-01-09 19:12:14 TRACE daemon::monitor: LockFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have in mempool
2022-01-09 19:12:14 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 1 remaining for this script
2022-01-09 19:12:14 TRACE daemon::monitor: CommitFinality(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 1 blocks and we have unseen
2022-01-09 19:12:14 TRACE daemon::monitor: CetTimelockExpired(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 12 blocks and we have unseen
2022-01-09 19:12:14 TRACE daemon::monitor: RefundTimelockExpired(OrderId(Hyphenated(c92b84bd-94b2-49a9-9972-0f9f6c528a52))) requires confirmed with 216 blocks and we have unseen
2022-01-09 19:12:14 TRACE daemon::monitor: 0 subscriptions reached their monitoring target, 3 remaining for this script
2022-01-09 19:12:14 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:12:18 ERROR daemon::model::cfd: Contract setup failed: Expected Msg0 within 70 seconds: deadline has elapsed

Taker logs:

2022-01-09 19:11:08  INFO daemon::setup_taker: Order got accepted order_id=4c7aeac8-da47-44e2-8d7b-4df46e3503d9
2022-01-09 19:11:08 ERROR daemon::model::cfd: Contract setup failed: Failed to build party params: InsufficientFunds { needed: 118284, available: 47555 }
thread 'rocket-worker-thread' panicked at 'we checked that we are connected: Disconnected', daemon/src/address_map.rs:79:22
stack backtrace:
   0: rust_begin_unwind
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:517:5
   1: core::panicking::panic_fmt
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panicking.rs:100:14
   2: core::result::unwrap_failed
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/result.rs:1616:5
   3: core::result::Result<T,E>::expect
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/result.rs:1258:23
   4: daemon::address_map::AddressMap<K,A>::send_fallible::{{closure}}
             at ./src/address_map.rs:76:27
   5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/mod.rs:80:19
   6: <daemon::connection::Actor as xtra::Handler<daemon::connection::MakerStreamMessage>>::handle::{{closure}}
             at ./src/connection.rs:456:20
   7: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/mod.rs:80:19
   8: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/future.rs:119:9
   9: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.19/src/future/future/map.rs:55:37
  10: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.19/src/lib.rs:91:13
  11: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/future.rs:119:9
  12: xtra::context::Context<A>::tick::{{closure}}
             at /Users/dakami/.cargo/git/checkouts/xtra-27dfd36cb04c9fa2/413dc80/src/context.rs:266:17
  13: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/mod.rs:80:19
  14: xtra::context::Context<A>::run::{{closure}}
             at /Users/dakami/.cargo/git/checkouts/xtra-27dfd36cb04c9fa2/413dc80/src/context.rs:242:19
  15: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/mod.rs:80:19
  16: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::future::future::Future>::poll
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panic/unwind_safe.rs:289:9
  17: <futures_util::future::future::catch_unwind::CatchUnwind<Fut> as core::future::future::Future>::poll::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.19/src/future/future/catch_unwind.rs:36:42
  18: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panic/unwind_safe.rs:271:9
  19: std::panicking::try::do_call
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:403:40
  20: <unknown>
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:431:6
  21: std::panicking::try
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:367:19
  22: std::panic::catch_unwind
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panic.rs:133:14
  23: <futures_util::future::future::catch_unwind::CatchUnwind<Fut> as core::future::future::Future>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.19/src/future/future/catch_unwind.rs:36:9
  24: <futures_util::future::future::remote_handle::Remote<Fut> as core::future::future::Future>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.19/src/future/future/remote_handle.rs:105:29
  25: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/future.rs:119:9
  26: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:161:17
  27: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/loom/std/unsafe_cell.rs:14:9
  28: tokio::runtime::task::core::CoreStage<T>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:151:13
  29: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:461:19
  30: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panic/unwind_safe.rs:271:9
  31: std::panicking::try::do_call
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:403:40
  32: <unknown>
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:431:6
  33: std::panicking::try
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:367:19
  34: std::panic::catch_unwind
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panic.rs:133:14
  35: tokio::runtime::task::harness::poll_future
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:449:18
  36: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:98:27
  37: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:53:15
  38: tokio::runtime::task::raw::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:113:5
  39: tokio::runtime::task::raw::RawTask::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:70:18
  40: tokio::runtime::task::LocalNotified<S>::run
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/mod.rs:343:9
  41: tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:443:21
  42: tokio::coop::with_budget::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/coop.rs:102:9
  43: std::thread::local::LocalKey<T>::try_with
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/thread/local.rs:399:16
  44: std::thread::local::LocalKey<T>::with
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/thread/local.rs:375:9
  45: tokio::coop::with_budget
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/coop.rs:95:5
  46: tokio::coop::budget
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/coop.rs:72:5
  47: tokio::runtime::thread_pool::worker::Context::run_task
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:419:9
  48: tokio::runtime::thread_pool::worker::Context::run
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:386:24
  49: tokio::runtime::thread_pool::worker::run::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:371:17
  50: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/macros/scoped_tls.rs:61:9
  51: tokio::runtime::thread_pool::worker::run
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:368:5
  52: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:347:45
  53: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/blocking/task.rs:42:21
  54: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:161:17
  55: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/loom/std/unsafe_cell.rs:14:9
  56: tokio::runtime::task::core::CoreStage<T>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:151:13
  57: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:461:19
  58: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panic/unwind_safe.rs:271:9
  59: std::panicking::try::do_call
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:403:40
  60: <unknown>
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:438:2
  61: std::panicking::try
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:367:19
  62: std::panic::catch_unwind
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panic.rs:133:14
  63: tokio::runtime::task::harness::poll_future
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:449:18
  64: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:98:27
  65: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:53:15
  66: tokio::runtime::task::raw::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:113:5
  67: tokio::runtime::task::raw::RawTask::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:70:18
  68: tokio::runtime::task::UnownedTask<S>::run
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/mod.rs:379:9
  69: tokio::runtime::blocking::pool::Inner::run
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/blocking/pool.rs:264:17
  70: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/blocking/pool.rs:244:17
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'rocket-worker-thread' panicked at 'watch channel should outlive the future: RecvError(())', /Users/dakami/CoBloX/repos/hermes/daemon/src/connection.rs:604:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:517:5
   1: core::panicking::panic_fmt
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panicking.rs:100:14
   2: core::result::unwrap_failed
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/result.rs:1616:5
   3: core::result::Result<T,E>::expect
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/result.rs:1258:23
   4: daemon::connection::connect::{{closure}}
             at ./src/connection.rs:601:9
   5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/future/mod.rs:80:19
   6: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::future::future::Future>::poll
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panic/unwind_safe.rs:289:9
   7: <futures_util::future::future::catch_unwind::CatchUnwind<Fut> as core::future::future::Future>::poll::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.19/src/future/future/catch_unwind.rs:36:42
   8: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panic/unwind_safe.rs:271:9
   9: std::panicking::try::do_call
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:403:40
  10: <unknown>
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:431:6
  11: std::panicking::try
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:367:19
  12: std::panic::catch_unwind
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panic.rs:133:14
  13: <futures_util::future::future::catch_unwind::CatchUnwind<Fut> as core::future::future::Future>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.19/src/future/future/catch_unwind.rs:36:9
  14: <futures_util::future::future::remote_handle::Remote<Fut> as core::future::future::Future>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.19/src/future/future/remote_handle.rs:105:29
  15: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:161:17
  16: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/loom/std/unsafe_cell.rs:14:9
  17: tokio::runtime::task::core::CoreStage<T>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:151:13
  18: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:461:19
  19: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panic/unwind_safe.rs:271:9
  20: std::panicking::try::do_call
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:403:40
  21: <unknown>
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:431:6
  22: std::panicking::try
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:367:19
  23: std::panic::catch_unwind
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panic.rs:133:14
  24: tokio::runtime::task::harness::poll_future
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:449:18
  25: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:98:27
  26: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:53:15
  27: tokio::runtime::task::raw::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:113:5
  28: tokio::runtime::task::raw::RawTask::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:70:18
  29: tokio::runtime::task::LocalNotified<S>::run
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/mod.rs:343:9
  30: tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:420:13
  31: tokio::coop::with_budget::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/coop.rs:102:9
  32: std::thread::local::LocalKey<T>::try_with
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/thread/local.rs:399:16
  33: std::thread::local::LocalKey<T>::with
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/thread/local.rs:375:9
  34: tokio::coop::with_budget
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/coop.rs:95:5
  35: tokio::coop::budget
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/coop.rs:72:5
  36: tokio::runtime::thread_pool::worker::Context::run_task
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:419:9
  37: tokio::runtime::thread_pool::worker::Context::run
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:386:24
  38: tokio::runtime::thread_pool::worker::run::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:371:17
  39: tokio::macros::scoped_tls::ScopedKey<T>::set
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/macros/scoped_tls.rs:61:9
  40: tokio::runtime::thread_pool::worker::run
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:368:5
  41: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/thread_pool/worker.rs:347:45
  42: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/blocking/task.rs:42:21
  43: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:161:17
  44: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/loom/std/unsafe_cell.rs:14:9
  45: tokio::runtime::task::core::CoreStage<T>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/core.rs:151:13
  46: tokio::runtime::task::harness::poll_future::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:461:19
  47: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panic/unwind_safe.rs:271:9
  48: std::panicking::try::do_call
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:403:40
  49: <unknown>
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:438:2
  50: std::panicking::try
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panicking.rs:367:19
  51: std::panic::catch_unwind
             at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/std/src/panic.rs:133:14
  52: tokio::runtime::task::harness::poll_future
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:449:18
  53: tokio::runtime::task::harness::Harness<T,S>::poll_inner
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:98:27
  54: tokio::runtime::task::harness::Harness<T,S>::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/harness.rs:53:15
  55: tokio::runtime::task::raw::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:113:5
  56: tokio::runtime::task::raw::RawTask::poll
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/raw.rs:70:18
  57: tokio::runtime::task::UnownedTask<S>::run
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/task/mod.rs:379:9
  58: tokio::runtime::blocking::pool::Inner::run
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/blocking/pool.rs:264:17
  59: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}
             at /Users/dakami/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.15.0/src/runtime/blocking/pool.rs:244:17
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
2022-01-09 19:11:10  INFO daemon::setup_contract: Created CFD transactions
2022-01-09 19:11:10 ERROR daemon::model::cfd: Contract setup failed: Failed to send Msg1: Actor address disconnected
2022-01-09 19:11:12 TRACE daemon::oracle: Skipping /x/BitMEX/BXBT/2022-01-10T09:00:00.price?n=20 because it likely hasn't occurred yet
2022-01-09 19:11:12 TRACE daemon::bitmex_price_feed: No message from BitMex in the last 5 seconds, pinging
2022-01-09 19:11:13 TRACE daemon::bitmex_price_feed: Received pong
@da-kami da-kami added the bug Something isn't working label Jan 9, 2022
@da-kami da-kami added this to the 0.4.0 - "Rickrolled Rick" milestone Jan 9, 2022
@thomaseizinger thomaseizinger self-assigned this Jan 10, 2022
@thomaseizinger
Copy link
Contributor

This is a race condition between checking that we are disconnected and actually attempting to send a message. We can fix that pretty easily!

thomaseizinger added a commit that referenced this issue Jan 10, 2022
@thomaseizinger
Avoid race condition in address_map
cbba28d 
The check for being connected and sending a message is not atomic. To
not trigger a panic, we need to attempt sending the message and react to
the error accordingly.

Fixes #1049.
@bors bors bot closed this as completed in a0ece19 Jan 10, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@thomaseizinger thomaseizinger

Labels
bug Something isn't working
Projects
None yet
Milestone
0.4.0 - "Rickrolled Rick"
Development

No branches or pull requests
2 participants
@thomaseizinger @da-kami