server: Filter out duplicated transactions while synchronizing the le…

…dger

Bitcoin allows duplicated transactions that are already spent, this causes
conflicts when retrieving them from the txindex because the older ones are lost.

Filtering the lost transactions reduces the consistency but allows to keep syncing.

server/app/com/xsn/explorer/services/LedgerSynchronizerService.scala

@@ -186,8 +186,14 @@ class LedgerSynchronizerService @Inject() (
       data <- transactionCollectorService.collect(rpcBlock.transactions, ExcludedTransactions).toFutureOr
       (transactions, contracts) = data
       validContracts <- getValidContracts(contracts).toFutureOr
+      filteredTransactions = transactions.filter(_.blockhash == rpcBlock.hash)
     } yield {
-      val block = toPersistedBlock(rpcBlock, extractionMethod).withTransactions(transactions)
+      if (transactions.size != filteredTransactions.size) {
+        // see https://github.com/bitpay/insight-api/issues/42
+        logger.warn(s"The block = ${rpcBlock.hash} has phantom ${transactions.size - filteredTransactions.size} transactions, they are being discarded")
+      }
+
+      val block = toPersistedBlock(rpcBlock, extractionMethod).withTransactions(filteredTransactions)
       (block, validContracts)
     }