Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Segwit for BSQ #5000

Merged
merged 20 commits into from
Mar 22, 2021

Commits on Jan 21, 2021

  1. Bring ScryptType enum up to date with latest bitcoind

    Add 'witness_v1_taproot' script type to the enum and proto.pb, so that
    it doesn't cause any problems when Taproot is activated and the new
    script type starts showing up in RPC getBlock(..) responses (including
    possibly BSQ transactions).
    
    Also change the Java enum order (which shouldn't cause any problems as
    the ordinal isn't used directly in hashCode calculations) and add the
    missing 'witness_unknown' enum value to pb.proto to bring it in sync.
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    5990fcf View commit details
    Browse the repository at this point in the history
  2. Add replacement bitcoind RPC client using jsonrpc4j

    Create a new 'BitcoindClient' interface and a corresponding builder, to
    replace the old 'com.neemre.btcdcli4j.core.client.BtcdClientImpl' class
    from the btcdcli4j library. This is instantiated by jsonrpc4j using a
    dynamic proxy. It provides only a cut down version of the bitcoind RPC
    API, exposing the methods 'getblock', 'getblockcount' & 'getblockhash',
    as they are the only ones currently being used by RpcService.
    
    Add corresponding Jackson-annotated DTO classes to model the JSON
    structures returned by bitcoind, very similar to the classes provided by
    btcdcli4j. Note that we use Double instead of BigDecimal to represent
    fractional fields (difficulties + coin amounts in BTC), as they have
    more consistent Jackson (de)serialisation and appear to be able to
    faithfully round-trip numeric fields produced by bitcoind. Also note
    that doubles can faithfully represent any valid decimal BTC amount (that
    is, with 8 d.p. of precision) up to 21 million.
    
    For now, keep the old BtcdClientImpl instance used by RpcService in
    place, as the btcdcli4j block notification daemon is dependent upon it
    and would also need to be replaced.
    
    Also add unit tests for BitcoindClient which test against sample regtest
    responses, using a mock HttpURLConnection.
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    dbe4953 View commit details
    Browse the repository at this point in the history
  3. Add replacement bitcoind block notification daemon

    Provide a new 'BitcoindDaemon' block notification socket server, to
    replace 'com.neemre.btcdcli4j.daemon.BtcdDaemonImpl'. This starts a
    single service thread to listen for raw block hashes on localhost port
    512*, sent by the specified 'blocknotify' shell/batch script, delegating
    to a pool of worker threads to run the supplied BlockListener handler.
    Unlike the original BtcdDaemonImpl class, a call to the 'getblock' RPC
    method is not made automatically to supply a complete block to the
    handler, instead requiring a separate, manual BitcoindClient.getBlock
    invocation from within RpcService.
    
    Also provide unit tests using a mock ServerSocket + Socket.
    
    TODO: Use the new Bitcoind(Client|Daemon) implementations in RpcService,
     in place of btcdcli4j Btcd(Client|Daemon)Impl & remove the old library.
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    b7c7eaf View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    863392b View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    f34231a View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    4470af8 View commit details
    Browse the repository at this point in the history
  7. Improve exception handling in BitcoindDaemon

    Wrap any exception that occurs during socket IO or within the supplied
    BlockListener with a new 'BlockNotificationException'. This brings the
    exception handling more in line with that of the old BtcdDaemonImpl and
    makes it easier to match them downstream in FullNode.handleError.
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    ac78639 View commit details
    Browse the repository at this point in the history
  8. Use new Bitcoind(Client|Daemon) & remove btcd-cli4j

    Migrate RpcService over to the new block notification daemon and RPC
    client based on jsonrpc4j. Drop in own DTO classes in place of the ones
    defined by btcd-cli4j and rename requestBtcBlock & addNewBtcBlockHandler
    to requestDtoBlock & addNewDtoBlockHandler respectively.
    
    Also remove now redundant filtering from the logback config and update
    grade-witness.
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    8104301 View commit details
    Browse the repository at this point in the history
  9. Enable extraction of segwit pubkeys from raw tx inputs

    Factor out a new RpcService.extractPubKeyAsHex method, to take public
    keys from the inputs of the raw transactions returned by the RPC client,
    when building TxInput objects to incorporate into the DAO state. Enhance
    the method to additionally support segwit (P2WPKH & P2SH-P2WPKH) inputs
    (but only the first input for backwards compatibility - see code
    comment). Also fix a bug when handling non-SIGHASH_ALL input signatures.
    
    This will allow segwit BSQ to be used in proof-of-burn and issuance txs,
    which need a public key associated with the tx to establish ownership of
    it, when signing messages with a proof-of-burn or staking merit awarded
    from a compensation issuance, respectively.
    
    Also add unit tests for the factored-out method and add a missing RawTx
    toString() method, to aid debugging the TxInput fields within the
    processed block returned by RpcService.
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    6ca42c0 View commit details
    Browse the repository at this point in the history
  10. Fix Codacy issues & deduplicate code in RpcService

    Factor out shared construction logic to a new 'getBlockFromRawDtoBlock'
    method in RpcService. Also add some 'NOPMD' comments in an attempt to
    suppress unfixable Codacy warnings about qualified imports.
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    93b46e3 View commit details
    Browse the repository at this point in the history
  11. Fix data race in BitcoindDaemonTest

    Prevent intermittent test failures, caused by a race between checking
    whether the mock socket is closed upon accepting a new connection and
    setting 'socketClosed' to true during shutdown. Waiting to accept and
    then checking the flag needs to be done in a synchronized block.
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    796097a View commit details
    Browse the repository at this point in the history
  12. Make special case for irregular tx with segwit BSQ inputs

    Selectively disable pubkey extraction from segwit inputs of a particular
    tx at block height 660384 (2020-12-07), which spends spuriously created
    segwit BSQ (later burned), to prevent a change in the DAO state hashes
    from that point.
    
    (Since a tx with a given ID can only appear on one chain, a fixed global
    exclusion list of IDs should not cause any issues on testnet/regtest
    versus mainnet. This is simpler than conditioning by block height.)
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    a850ada View commit details
    Browse the repository at this point in the history
  13. Add getNetworkInfo & getBestBlockHash RPC client methods

    Provide a 'NetworkInfo' DTO class (with associated nested DTO classes),
    returned by the 'getnetworkinfo' RPC method call to bitcoind. This will
    be used during startup of RpcService to determine if Bitcoin Core is
    available and which version it is using. Add a unit test to round-trip a
    sample NetworkInfo JSON response.
    
    Also add the missing 'getbestblockhash' RPC method, which will be needed
    by RpcService to determine the Bitcoin Core node health.
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    d595cac View commit details
    Browse the repository at this point in the history
  14. Add Bitcoin Core version & health check to RpcService

    Make a 'getnetworkinfo' RPC call to bitcoind immediately upon startup,
    to check that the node is up (and throw a ConnectException to ensure
    that the user is presented with an appropriate warning popup otherwise).
    Log a warning if the node version is outside the 0.18.0 - 0.20.1 range.
    
    Additionally, call 'getbestblockhash' to check that the chain tip is not
    stale (> 6 hours old). As part of this, make sure the 'getblock' RPC
    call works correctly with verbosity < 2, by fixing JSON deserialisation
    of the response when the block or txs are in summary (hex string) form.
    
    (These version & health checks are almost identical to the ones done by
    the original btcd-cli4j library during RPC client startup.)
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    4b0711b View commit details
    Browse the repository at this point in the history
  15. Exclude segwit pubkeys by block height instead of a blacklist

    Since extraction of segwit pubkeys technically represents a hard fork,
    activate it by block height in the same way as the fork defined in
    TxOutputParser, instead of relying on the absence of premature segwit
    BSQ inputs outside of a blacklist. This also means we no longer need to
    exclude all but the first tx input from segwit pubkey extraction to
    maintain backwards compatibility, which is a little safer and consistent
    with the original behaviour of extracting every available P2PKH pubkey.
    
    Provisionally activate this (2nd) DAO hard fork at block height 672646,
    which should be 6 weeks from now, just under 5 weeks from the planned
    1.5.5 release on 2021/01/27. (Block 1906689 for testnet - 2 weeks from
    now assuming an average block time of 10 minutes, but it's erratic.)
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    1abf4c5 View commit details
    Browse the repository at this point in the history
  16. Ensure RPC client works with Bitcoin Core 0.21.0

    Add missing 'connections_(in|out)' JSON properties to NetworkInfo that
    came with the recent 0.21.0 release of Bitcoin Core. Also ensure that
    unrecognised JSON properties are ignored, so that future changes to the
    RPC API are less likely to break our client.
    
    Also, for the benefit of the tests, change the JSON property order of
    RawInput to better match observed 'getblock' responses. (It appears that
    in 0.21.0, extra "txinwitness" fields have started appearing in coinbase
    inputs, which may be a bug.)
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    a8b0863 View commit details
    Browse the repository at this point in the history
  17. Rename rpc.dto.RawX to RawDtoX to avoid confusion

    Avoid name clashes between the Json RPC client DTOs & the corresponding
    raw (un-parsed) DAO state entities. Also prepend the other DTO classes
    with 'Dto' for consistency.
    stejbac committed Jan 21, 2021
    Configuration menu
    Copy the full SHA
    e0aa76e View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2021

  1. Upgrade jsonrpc4j & Jackson; simplify BitcoindClient

    Change jsonrpc4j version from 1.5.3 to 1.6.0.bisq.1, forked to the Bisq
    repo from the recent 1.6.0 release. The forked version changes the class
    'com.googlecode.jsonrpc4j.HttpException' to be public, instead of (prob.
    mistakenly) package private, so we can avoid using reflection to catch
    it and re-throw as a 'bisq.network.http.HttpException'. Remove the now
    unused constructors from the latter.
    
    As part of this, upgrade Jackson to the latest stable (2.12.1) release,
    since jsonrpc4j now depends on a newer version than the previous 2.8.10.
    stejbac committed Feb 4, 2021
    Configuration menu
    Copy the full SHA
    b4ad6bf View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2021

  1. Activate Segwit DAO hard fork 4 weeks after v1.6.0 release

    Push back the (2nd) DAO hard fork activation block height to 680300,
    which is 4 weeks after the planned 1.6.0 release around 2021/03/25.
    (Also push back the testnet block activation height to 1943000 - 2 weeks
    from now assuming an average block time of 10 minutes.)
    stejbac committed Mar 20, 2021
    Configuration menu
    Copy the full SHA
    05ea293 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    f213791 View commit details
    Browse the repository at this point in the history