Fix tradestatistics #3469
Merged
Fix tradestatistics #3469
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was done earlier when only maker was publishing trade statistics. Now both traders do it so we get already higher resilience.
Forgot in prev. commit to remove also the method.
The 2 arbitrator related fields in Trade are only set by the maker and not used anymore for reading, so it can be removed. The whole arbitrator domain should be cleaned out some day, but because of backward compatibility issues it id difficult to do it entirely at release date. With release after v 1.2. when no old offers are out anymore we are able to clean up that domain.
…x-tradestatistics # Conflicts: # core/src/main/java/bisq/core/trade/statistics/TradeStatisticsManager.java
|
@chimp1984 Could you please verify, that my conflict resolving in TradeStatisticsManager was correct. Thanks! |
ripcurlx
added a commit
that referenced
this pull request
Oct 31, 2019
* New trade protocol (#3333) * Remove arbitration key, cleanup * Add BuyerAsMakerProcessDepositTxAndDelayedPayoutTxMessage * Adopt trade protocol - Add handler for DepositTxAndDelayedPayoutTxMessage - Change handler for DepositTxPublishedMessage - Add MakerSetsLockTime - Rename MakerProcessPayDepositRequest to MakerProcessPayDepositRequest - Rename MakerSendPublishDepositTxRequest to MakerSendsProvideInputsForDepositTxMessage - Rename DepositTxPublishedMessage to DelayedPayoutTxSignatureRequest - Rename MakerProcessDepositTxPublishedMessage to MakerAsBuyerProcessSignDelayedPayoutTxMessage * Remove arbitratorKey * Add new classes * Add new message classes * Add new task classes * Renamed classed (no functional change yet) * Add lockTime * Add delayedPayoutTxSignature field * Add useReimbursementModel field * Add new classes * Add setting.preferences.useReimbursementModel * Apply renamed classes (new classes not added yet) * Add useReimbursementModel * Add preferences param * Add new methods, cleanup * Add daoFacade param, apply renaming * Add delayedPayoutTx, lockTime and delayedPayoutTxId - Support daoFacade param * Remove DirectMessage interface * Rename emergencySignAndPublishPayoutTx method, add new one for 2of2 MS * Apply new protocol * Apply new protocol * Add renaming (no functional change yet) * Add new messages, apply renaming * Remove unneeded P2SHMultiSigOutputScript * Remove PREFERRED_PROJECT_CODE_STYLE * Refactor: Rename class * Use InputsForDepositTxRequest instead of TradeMessage in handleTakeOfferRequest * Do not sign deposit tx if maker is seller We change behaviour that the maker as seller does not send the pre signed deposit tx to the taker as the seller has more to lose and he wants to control the creation process of the delayed payout tx. * Apply new trade protocol to seller as maker version * Apply new trade protocol Delayed payout tx are now working for all scenarios but we use a small hack to get around an issue with not receiving confirmations and the peers tx. We add a tiny output to both peers, so we see the tx and confirmation. Without that only the publisher sees the tx and confirmations are not displayed. Need further work to get that working without that extra outputs. * Set TRADE_PROTOCOL_VERSION to 2 * Add PeerPublishedDelayedPayoutTxMessage We need add the delayed payout tx to the wallet once the peer publishes it. We will not see the confidence as we do not receive or sent funds from our address. Same is with dispute payouts where one peer does not receive anything. Then the confidence is not set. It seems that is a restriction in BitcoinJ or it requires some extra handling. We set the confidence indicator invisible in the dispute case and that might be an acceptable option here as well. * Add refund agent domain * Add refundAgentNodeAddress * Apply refund domain * Add refund views * Apply refundAgent domain * Support refundAgent * Remove useReimbursementModel field We dont need in the offer anymore the decision if reimbursement or arbitration is chosen. * Apply refundAgent payout * Handle tx info and balances * Remove mediation activation * Add new tac accepted flag for v1.2.0 and adjust text * Fix params for test classes * Signed witness trading (#3334) * Added basic UI for account signing for arbitrators * Add domain layer for signed account age witnesses (credits ManfredKarrer and oscarguindzberg) * Remove testing gridlines * Arbitrator sign accountAgeWitnesses Automatically filter to only sign accounts that - have chargeback risk - bought BTC - was winner in dispute * Handle chargebackrisk by currency * Check winners only for closed disputes * Show sign status of paymentaccounts in AccountsView * Rename service to accountAgeWitnessService * Refactor: Move account signing helpers to AccountAgeWitnessService * Refactor: rename hasSignedWitness to myHasSignedWitness * Show if witness is signed in offerbook view * Use witness sign age for age comparison * Refactor: rename to isTaker... to isMyTaker... * Allow trading with signed witnesses * Use witness age for showing account age icon * Move AccountAgeRestrictions into AccountAgeWitnessService * Handle trade limit of unverified accounts as normal case * Avoid optional as argument * Set trade limit depending on trade direction * Avoid optional arguments * Add text for seller as signer * Seller with signer privilege signs buyer witness * Fix merge issues * Remove explicit check for risky offers * Remove sellers explicit account age check * Add limit check based on common accountAgeWitness function * Fix arbitrator key event handling * Filter accounts on tradelimit instead of maturity * Fix test * Buyer sign seller account Add SIGNED_ACCOUNT_AGE_WITNESS capability * Fix checks for signing at end of trade Get correct valid accounts for offer * Rename BuyerDataItem -> TraderDataItem * Arbitrator sign both parties in a buyer payout dispute * Only sign unsigned accountAgeWitnesses * Remove unused code * Add demo for material design icons * Use different account age limits for sell/buy * Fix signing interface for arbitrator * Add signing state column to offer book * Add signing state to fiat accounts overview * Add signing state to selected fiat account * Fix popover padding * Add account signing state to peer info popup * Retrieve only unsigned witnesses for arbitrator to sign * Accounts signed by arbitrators are signers * Disable test due to travis issues * Improve witness handling (#3342) * Fix comparison * Add display strings for witness sign state * Fix immaturity check * Use accountAgeWitness age for non risky payment methods * Show information about non risky account types * Fix peer info icon account age text * Complete new trade protocol (#3340) * Improve handling of adding tx to wallet * Add delayedPayoutTx to dispute * Fix test * Use RECIPIENT_BTC_ADDRESS from DAO for trade fee * Set lockTime to 10 days for altcoins, 20 days others. - Devmode uses 1 block * Fix params * Update text * Update docs * Update logging if (log.isDebugEnabled()) only matches if logLevel is debug not if it is INFO * Remove log * Remove arbitrator checks * Remove arbitrator address - It works not if not legacy arbitrator is registered. We cannot remove too much from arbitration as we would risk to break account signing and display of old arbitration cases. Though if testing time permits we should try to clean out more of arbitration domain what is not needed anymore. * Use account signing state in accounts view (#3365) * Add account signing icons to signing state in account display * Remove unnecessary "." that caused layout issues in the past * Add additional warning in the received payment popup for account signer * Fix Revolut padding issues for currencies * Hide signing icon for non-high-risk payment methods * Add correct icon state and info text for account signing state * Remove not implemented notification part * Test self signing witnesses * Change verified account limit factor to 0.5 * Account Signing: Add information popups for signing state (#3374) * Add account signing icons to signing state in account display * Remove not implemented notification part * Hide time since signing column when not needed * Remove fiat rounding popup as feature was introduced a long time ago already * Add information popups for new signed states (only shown once for user) and minor clean-ups * Update core/src/main/resources/i18n/displayStrings.properties Co-Authored-By: sqrrm <[email protected]> * Account Signing: Improve signed state notificaton (#3388) * Remove new badge from Altcoin instant feature * Remove new badge from percentage user deposit feature * Fix line break issues in received payment confirmation popup * Check if received payload fulfills signing state condition and not any personal witness * Show additional badge for account sections to guide user to check out new signing states * Fix account signing state in offer book (#3390) * Account Signing: Fix verified usage (#3392) * Rename witnessHash -> accountAgeWitnessHash * Add enum for SignedWitness verification method * Fix usage of isValidAccountAgeWitness * Revert icon for signstate change * Account signing: add signing state to payment account selection (#3403) * Clean up dead code parts * Add account signing state to payment account selection * Account signing: revert dev date setting for trusted accounts (#3404) * Revert temporary value for dev testing * Only enable button if there are accounts to be signed * Add trade limit exceptions (#3406) * Remove dead code * Add trade limit exception for accounts signed by arbitrator * Update translations to adapt to new unified delay (#3409) * NTP: Fix a couple of UI issues in the New Trade Protocol (#3410) * Add badge support for refund agent (new arbitrator) tickets * Fix translation typo * Clean up arbitrator issues in translation * Only show refund agent label to support staff Every user should still see this role as arbitration * NTP: Improve differentiation between mediation and new arbitration (#3414) * Clean up property exposure * Improve differentiation between mediation and arbitration cases * Go to new refund view if it is no mediation and not open mediation notification if refund is already in progress * Don't sign filtered accounts * NTP: merge with master (#3420) * Temporarily disable onion host for @KanoczTomas's BTC node * Add Ergo (ERG) without Bouncy Castle dependency. See #3195. * List CTSCoin (CTSC) * Tweak the English name of Japan Bank Transfer payment method * Add mediator prefix to trade statistics * List Faircoin (FAIR) * List uPlexa (UPX) * Remove not used private methods from BisqEnvironment * Add onInitP2pNetwork and onInitWallet to BisqSetupListener - Rename BisqSetupCompleteListener to BisqSetupListener - Add onInitP2pNetwork and onInitWallet to BisqSetupListener - make onInitP2pNetwork and onInitWallet default so no impl. required * Start server at onInitWallet and add wallet password handler - Add onInitWallet to HttpApiMain and start http server there - Add onRequestWalletPassword to BisqSetupListener - Override setupHandlers in HttpApiHeadlessApp and adjust setRequestWalletPasswordHandler (impl. missing) - Add onRequestWalletPassword to HttpApiMain * Add combination (Blockstream.info + Mempool.space) block explorer * Revert "Temporarily disable onion host for @KanoczTomas's BTC node" This reverts commit d333520. * Temporarily disable KanoczTomas btcnode on both onion and clearnet * Refactor BisqApp - update scene size calculation * Refactor BisqApp - update error popup message build * Refactor BisqApp - move icon load into ImageUtil * Remove unused Utilities * Increase minimum TX fee to 2 sats/vByte to fix #3106 (#3387) * Fix mistakes in English source (#3386) * Fix broken placeholders * Replace non existing pending trades screen with open trades screen * Update core/src/main/resources/i18n/displayStrings.properties Co-Authored-By: Steve Jain <[email protected]> * Update message in failed trade popup * Refactor BisqEnvironment * Account Signing: Improve arbitrator signing flow (#3421) * Pre-select a point of time 2 months in the past So all arbitrator signed payment accounts will have their limits lifted completely * Only show payment methods with high chargeback risk to be signed * Show connected Bitcoin network peer info * List Ndau (XND) - Official project URL: https://ndau.io/ - Official block explorer URL: https://explorer.service.ndau.tech * List Animecoin (ANI) * Apply rule to not allow BSQ outputs after BTC output for regular txs (#3413) * Apply rule to not allow BSQ outputs after BTC output for regular txs * Enforce exactly 1 BSQ output for vote reveal tx * Fix missing balance and button state update * Refactor isBtcOutputOfBurnFeeTx method and add comments and TODOs No functional change. * Handle asset listing fee in custom method We need to enforce a BSQ change output As this is just tx creation code it has no consequences for the hard fork. * Use getPreparedBurnFeeTxForAssetListing * Update comments to not use dust output values * Fix missing balance and button state update * Use same method for asset listing fee and proof of burn Use same method for asset listing fee and proof of burn as tx structure is same. Update comments to be more general. * Use getPreparedProofOfBurnTx * Require mandatory BSQ change output for proposal fee tx. We had in the doc stated that we require a mandatory BSQ change output but it was not enforced in the implementation, causing similar issues as in Asset listing and proof of burn txs. * Add fix for not correctly handled issuance tx * Use new method for issuance tx // For issuance txs we also require a BSQ change output before the issuance output gets added. There was a // minor bug with the old version that multiple inputs would have caused an exception in case there was no // change output (e.g. inputs of 21 and 6 BSQ for BSQ fee of 21 BSQ would have caused that only 1 input was used // and then caused an error as we enforced a change output. This new version handles such cases correctly. * Handle all possible blind vote fee transactions * Move check for invalid opReturn output up * Add dust check at final sign method * Fix incorrect comments * Refactor - Remove requireChangeOutput param which is always false - Remove method which is used only by one caller - Cleanup * Add comment * Fix comments, rename methods * Move code of isBlindVoteBurnedFeeOutput to isBtcOutputOfBurnFeeTx * Update account signing strings for v1.2 release (#3435) * Update account signing strings for v1.2 release * Add minor corrections from ripcurlx review * Adjust tradeLimitDueAccountAgeRestriction string So that it describes why an account isn't signed (in general) instead of why it wasn't signed by an arbitrator. * Account Signing/NTP: More improvements and fixes (#3436) * Select the the correct sub view when a dispute is created * Require capability REFUND_AGENT to receive RefundAgent Messages * Remove unused return type for account signing * Add new feature popup for account signing and new trade protocol * Return void from account signing * Fix bug with not updating vote result table at vote result block * NTP: improve backwards compatibility for mediation (#3439) * Improve readability of offer update * Add type safeguard for dispute lists * Set not existing dispute support type for clients < 1.2.0 from message support type * Enable handling of mediation cases for old trade protocol disputes in 1.2.0 clients * Remove unnecessary forEach * Use correct formatter and add missing value for placeholder * Bump version number * Add sign all checkbox. Fix list entry display (#3450) * Add sign all checkbox. Fix list entry display * Add summary to log and clipboard * Use safe version for seednodes (#3452) * Apply shutdown and memory check again To not risk issues with the release and seed nodes we merge back the old code base for handling memory check and shutdowns. The newly added changes for cross connecting between seed nodes cause out of memory issues and require more work and testing before it can be used. * Revert code change for periodic updates between seed nodes. The periodic updates code caused out of memory issues and require more work and testing before it can be used. * Arbitrator republish signedWitnesses on startup (#3448) * Arbitrator republish signedWitnesses on startup * Keep republish internal to SignedWitnessService * Improve new feature popup for ntp and account signing (#3453) * Do not commit delayedPayoutTx to avoid publishing at restart Fixes #3463 BitcoinJ publishes automatically committed transactions. We committed it to the wallet to be able to access it later after a restart. We stored the txId in Trade and used that to request the tx from the wallet (as it was committed). Now we store the bitcoin serialized bytes of the tx and do not commit the tx before broadcasting it (if a trader opens refund agent ticket). * [1.2.0] Update client resources (#3456) * Update bitcoinj checkpoint file * Update data stores * Update translations * [1.2.0] Improve new feature popup (#3465) * Improve layout of new feature popup * Extract external hyperlinks into component to make it easier to update * Comment in necessary showAgain check * Add Raspberry Pi to build process (#3466) * Add Raspberry Pi to build process * Rename deploy variable to improve readability * Update informational prompt upon creating fiat account with account signing details (#3467) * Update informational prompt upon creating fiat account with account signing details * Fix wrong buyer limit for first 30 days * Set delayedPayoutTxBytes when setting delayedPayoutTx Fixes #3473 The delayedPayoutTx is not committed to the wallet as long it is not published. The seller who creates the delayedPayoutTx has not stored the delayedPayoutTxBytes which caused a nullpointer after restart. * Minor updates (#3474) * Remove unnecessary log statement This seems to be a left over log statement from debugging. * Use a small delay for MakerSetsLockTime on regtest When testing on regtest, not in devmode, we want a relatively short delay to be able to test the delay period. * Clarify payment limits up to 30 days after signing * Update RECIPIENT_BTC_ADDRESS for regtest (#3478) Use an address that is owned by the regtest wallet in the dao-setup.zip file. This allows for easily verifying BTC trading fees are sent to this address correctly. Also, it helps verify spending of the time lock payout. * Remove btc nodes from Manfred Karrer (#3480) * Avoid null objects (#3481) * Avoid null objects * Remove check for type Historical data can be arbitration instead of mediation (arbitration was fallback at last update), so we need to tolerate the incorrect type here. Is only for tickets from pre 1.2. * Display appropriate account age info header Depending on charge back risk type, accounts should show accountAgeWitness age or time since signing * Set amount for delayed payout tx to 0 (#3471) We have shown the spent funds from the deposit tx to the bisq donation address before. But that was incorrect from the wallet perspective and would have lead to incorrect summary of all transaction amounts. We set it now to 0 as we are not spending funds nor receiving any in our wallet. * Check for result phase at activate method Fixes #3487 * Only show warning for risky payment menthods (#3497) * Fix style issues with dark mode (#3495) * Addresses issues mentioned in #3482 (comment) (#3496) * Clean up trade statistics from duplicate entries (#3476) * Clean up trade statistics from duplicate entries At software updates we added new entries to the extraMap which caused duplicate entries (if one if the traders was on the new and the other on the old version or at republishing). We set it now json exclude so avoid that in future and clean up the map. * Avoid repeated calls to addPersistableNetworkPayloadFromInitialRequest For trade stat cleanup we don't want to apply it multiple times as it is a bit expensive. We get from each seed node the initial data response and would pollute with the second response our map again and if our node is a seed node, the seed node itself could not get into a clean state and would continue pollution other nodes. * Refactor Remove not used param Rename method Inline method Cleanups * Change unsigned to N/A * [1.2.0] Update data stores and adding SignedWitnessStore (#3494) * Update data stores and adding SignedWitnessStore * Update translations * Update cleaned TradeStatistics2Store and changes in other stores * VoteResultView update results on any block in result phase Avoid updating the result more than once per result phase but make sure it's done if activated during the result phase * [1.2.0] Format maker fee for BTC and BSQ correctly (#3498) * Format maker fee for BTC and BSQ correctly * Update tests * Only automatically open popup if result wasn't accepted and disable action button when being accepted (#3503) * Fix tradestatistics (#3469) * Remove delayed re-publishing of tradeStatistics This was done earlier when only maker was publishing trade statistics. Now both traders do it so we get already higher resilience. * Remove unused method Forgot in prev. commit to remove also the method. * Remove support for TradeStatistics2.ARBITRATOR_ADDRESS * Add comment and set ARBITRATOR_ADDRESS deprecated * Remove setting of arbitrator data from makers side The 2 arbitrator related fields in Trade are only set by the maker and not used anymore for reading, so it can be removed. The whole arbitrator domain should be cleaned out some day, but because of backward compatibility issues it id difficult to do it entirely at release date. With release after v 1.2. when no old offers are out anymore we are able to clean up that domain. * Remove dev log * Update translations * [1.2.0] Improve dispute section (#3504) * Improve wording for mediation summary and add specific next steps for refund agent case * Select the first dispute case when entering the support section * Revert to SNAPSHOT version * Fix but with initialRequestApplied (#3512) * Fix resource name (#3514) * Remove minor version number in news popup * Fix copy SignedWitnessStore db script * Not show payment account details for blocked offers * Use age of accountAgeWitness as basis for sell limits * Bump version number * Revert to SNAPSHOT version * Merge v1.2.0/v1.2.1 with master (#3521) * List Krypton (ZOD) * Temporarily disable onion host for @KanoczTomas's BTC node * Add Ergo (ERG) without Bouncy Castle dependency. See #3195. * List CTSCoin (CTSC) * Tweak the English name of Japan Bank Transfer payment method * List Animecoin (ANI) * Add mediator prefix to trade statistics * List Faircoin (FAIR) * List uPlexa (UPX) * Remove not used private methods from BisqEnvironment * Add onInitP2pNetwork and onInitWallet to BisqSetupListener - Rename BisqSetupCompleteListener to BisqSetupListener - Add onInitP2pNetwork and onInitWallet to BisqSetupListener - make onInitP2pNetwork and onInitWallet default so no impl. required * Start server at onInitWallet and add wallet password handler - Add onInitWallet to HttpApiMain and start http server there - Add onRequestWalletPassword to BisqSetupListener - Override setupHandlers in HttpApiHeadlessApp and adjust setRequestWalletPasswordHandler (impl. missing) - Add onRequestWalletPassword to HttpApiMain * Add combination (Blockstream.info + Mempool.space) block explorer * Revert "Temporarily disable onion host for @KanoczTomas's BTC node" This reverts commit d333520. * Temporarily disable KanoczTomas btcnode on both onion and clearnet * Refactor BisqApp - update scene size calculation * Refactor BisqApp - update error popup message build * Refactor BisqApp - move icon load into ImageUtil * Remove unused Utilities * Increase minimum TX fee to 2 sats/vByte to fix #3106 (#3387) * Fix mistakes in English source (#3386) * Fix broken placeholders * Replace non existing pending trades screen with open trades screen * Update core/src/main/resources/i18n/displayStrings.properties Co-Authored-By: Steve Jain <[email protected]> * Update message in failed trade popup * Refactor BisqEnvironment * List Ndau (XND) - Official project URL: https://ndau.io/ - Official block explorer URL: https://explorer.service.ndau.tech * Show connected Bitcoin network peer info * Not show payment account details for blocked offers (#3425) * Add GitHub issue template for user reported bugs (#3454) * Add issue template with steps to reproduce and actual/expected behavior * Fix typo in .github/ISSUE_TEMPLATE.md * Fix wrong auto merge * Add CapabilityRequiringPayload to TradeStatistics2 With v1.2.0 we changed the way how the hash is created. To not create too heavy load for seed nodes from requests from old nodes we use the SIGNED_ACCOUNT_AGE_WITNESS capability to send trade statistics only to new nodes. As trade statistics are only used for informational purpose it will not have any critical issue for the old nodes beside that they don't see the latest trades. * Fix tradestat hash issue (#3529) * Recreate hash from protobuf data To ensure all data are using the new hash method (excluding extraMap) we do not use the hash field from the protobug data but pass null which causes to create the hash new based on the new hash method. * Add filter.toString method and log filter in case of wrong signature We have atm a invalid filter (prob. some dev polluted a test filter to mainnet) * Change log level, add log * Refactor: Move code to dump method * Add TRADE_STATISTICS_HASH_UPDATE capability We changed the hash method in 1.2.0 and that requires update to 1.2.2 for handling it correctly, otherwise the seed nodes have to process too much data. * Add logs for size of data exchange messages * Add more data in log * Improve logs * Fix wrong msg in log, cahnge log level * Add check for depositTxId not empty * Remove check for duplicates As we recreate the hash for all trade stat objects we don't need that check anymore. * Add logs * Temporarily remove this part of the statistics It prevents merging with master because through auto merge a duplication of this part of the code is happening and prevents Travis from succeeding
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #3468
See commit comments for more details