Implement security incident victim repayment tracking mechanism

[cbeams]

As described in bisq-network/proposals#209:

A mechanism will be developed to track how much BTC has been received by each address over time
[...]
The repayment tracking mechanism mentioned above will be developed such that the USD value of each payment made to each address is calculated according to the average daily price on the day the payment was made

This means that we must implement a repayment tracking mechanism that meets the following requirements:

• tracks each utxo paid to each victim's repayment address (see Collect victim repayment details #76)
• tracks the date of each payment (may not be necessary to do ourselves; see Implement security incident victim repayment tracking mechanism #77 (comment))
• tracks the USD value of the bitcoin repaid according to the btcusd average on the day of the repayment

No specific proposals have been laid out as to how to do this, but we should focus on doing the simplest thing that can possibly work in an automated fashion. This should be a fun little effort. If you're interested in implementing this, please speak up here and lay out how you would do it. Ideally, we should have this solution in place before we merge bisq-network/bisq#4150 and begin doing actual repayments, but given that the data is all there on the blockchain forever, we technically should be able to implement this after the fact as well. Just not too far after the fact, because we should be transparent about how the repayment process is going, and because we need to remove addresses once they've been fully repaid, and this tracking mechanism is how we'll know that.

[ghost]

@cbeams are you looking for a script, a report, or something built into the existing bisq java framework itself?

The simplest way to obtain the information would be the following API query to a blockchain explorer, for example: https://api.blockchair.com/bitcoin/dashboards/addresses/1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa,12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX

This returns information about the cumulative BTC and USD amount received on each address.

• received — total received in satoshi
• received_usd — total received in USD

[cbeams]

Something external to the app. There’s no need to build this into the ui or on top of anything in the core. How is blockchair’s `received_usd` calculated? It can’t be the naive product of `btc_received` and the current usd price. It must add up the usd value of each btc payment received at the time it was received. It need not be super precise; a daily average price will suffice.

…

On May 1, 2020, at 9:03 PM, James Cox ***@***.***> wrote:  @cbeams are you looking for a script, a report, or something built into the existing bisq java framework itself? The simplest way to obtain the information would be the following API query to a blockchain explorer, for example: https://api.blockchair.com/bitcoin/dashboards/addresses/1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa,12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX This returns information about the cumulative BTC and USD amount received on each address. received — total received in satoshi received_usd — total received in USD — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[ghost]

How is blockchair’s received_usd calculated?

From experimentation it is based on the USD price at the time each transaction was made. From the above example, 50 BTC was apparently worth $0.50 in 2009.

The most recent deposit to that address, on 2020-04-04, was 700 sats. The coinmarketcap price on that day was around 6800 so (0.00000700 * 6800 == $0.047688) which rounds up to $0.05 -> displayed by blockchair.

I don't know exactly what their algorithm is, but it seems to match what you're asking for.

[cbeams]

From experimentation it is based on the USD price at the time each transaction was made

That sounds promising. I'd be happy with a simple script that does the following:

$ ./track-repayments
victim_id, repayment_address, btc_lost, btc_received, usd_value, usd_received
[outputs one line of values for each of the 6 victims]

where victim_id, repayment_address, btc_lost and usd_value are hard-coded values corresponding to the entries in the table at #76, and where btc_received and usd_received are pulled from the blockchair API's received and received_usd fields, with the satoshi-denominated received value being reformatted as bitcoin to match the btc_lost field.

As above, output can be simple comma or tab-separated values, one line per address. Can be JSON as well if that is preferred or more convenient.

Implementation language doesn't matter so long as the result is dead simple run on a typical *nix machine. Should ideally be published as a single-file gist.

Note that there are no automation requirements. We will just have someone own running this script manually on a regular basis so we know when we're getting close to full repayment on any given address. The script source will be public (as the aforementioned gist or otherwise), so anyone can run it for themselves as well.

Interested in implementing that, @jmacxx (or anyone else)?

Please indicate an estimate in USD (to be paid out as BSQ in a normal compensation request).

[cbeams]

Would also be good to see blockchair's transaction_count field included in the script output, just to sanity check that our random selection of repayment addresses is working as expected, i.e. that each address has roughly the same number of payments at any given time (until fully repaid and taken out of the rotation).

[cbeams]

So to be clear, if blockchair's API works as expected, we can strike the following requirement from the original description above:

• tracks the date of each payment

The script will still implicitly do this, but by delegating the work to the blockchair API (which is great, less for us to do).

[ghost]

Yes I'd like to do this.

Please indicate an estimate in USD (to be paid out as BSQ in a normal compensation request).

$50.

[cbeams]

Excellent, assigned!

[ghost]

First stab at the report, in a self-contained html file. No library dependencies other than a single call to api.blockchair.com

It is available at https://gist.github.com/jmacxx/ea2f021a2b87ecd8755e7010cb74c183
Suggestions/feedback welcome.

[Two of the addresses provided have already received some funds. If this was not part of the repayment, I'd suggest obtaining unused addresses in those cases].

Screenshot:

[cbeams]

This looks great, @jmacxx. I just tried it out locally and everything works as expected. I've asked victims 1 and 2 to provide a fresh address.

It's perfect that you provided this as a gist to start. It'll be good, though, to have this in a repository where we can make changes as necessary without necessarily needing to go through you. Could you submit a pull request to https://github.com/bisq-network/security? Please first change the name of the file from BisqRepaymentRpt.html to track-repayments.html. Please commit the file to the root of the repository. Thanks.

[cbeams]

And @jmacxx, please indicate in your PR commit message body that it Resolves bisq-network/admin#77 in order to properly link it to this issue and to close it when the PR is merged. Perhaps you would already have done this, I don't know.

[ghost]

Regarding addresses provided by users, it would obviously be important for them not to deposit to the addresses themselves. It looks like they provided an unused address from their wallet and then later that address was used in normal day-to-day wallet use. Electrum allows you to freeze an address which would prevent this.

[ghost]

Currently experiencing a paywall problem with blockchair API. Not sure if this is going to be an issue for whoever runs this report, I suspect it might. It seems blockchair are trying to monetize.

Your IP address is temporary blacklisted due to exceeding usage of API resources. Please apply for an API key by contacting us at [email protected]

For now I've updated my gist of the report to use https://sochain.com/api. Its not quite as useful as the blockchair one in that it only provides the BTC amounts, not USD.

I'd be happy to make a PR if you need it. @m52go

[m52go]

I haven't been following this in detail -- I think @ripcurlx is leading this initiative now, might be best to ask for his opinion.

[ripcurlx]

Currently experiencing a paywall problem with blockchair API. Not sure if this is going to be an issue for whoever runs this report, I suspect it might. It seems blockchair are trying to monetize.

Your IP address is temporary blacklisted due to exceeding usage of API resources. Please apply for an API key by contacting us at [email protected]

For now I've updated my gist of the report to use https://sochain.com/api. Its not quite as useful as the blockchair one in that it only provides the BTC amounts, not USD.

I'd be happy to make a PR if you need it. @m52go

@jmacxx I would run your script every other day to check the current values. I just ran it again and it did work (did get a 404, but that is because of the not existing transactions. But I saw also in the response Beginning July 19th, 2020 we'll start enforcing request cost formulas, see the changelog for details. So I guess there might be a problem starting from this date. I think we would need in the end also a USD value tracking if it is decided that the USD amount is compensated and not the BTC amount.

[ghost]

I ran it just now, the output checks out ok -> 6 repayment transactions so far.

How would you like the USD tracking to be done? To roll our own we would need a historical source of daily BTC price in USD - I don't know if this is available. Or we pay blockchair for an API subscription ($10 per year).

[ghost]

@ripcurlx two of the addresses are not receiving any payments.

[wiz]

Why do you need a blockchair subscription? You can use mempool API for free let me know what you need

[ghost]

We need the total USD value of funds received to an address, value marked to market at the date it was broadcast. All UTXOs received, regardless if they were later spent.

As specified by cbeams:

How is blockchair’s received_usd calculated? It can’t be the naive product of btc_received and the current usd price. It must add up the usd value of each btc payment received at the time it was received. It need not be super precise; a daily average price will suffice.

The API we currently use: https://api.blockchair.com/bitcoin/dashboards/addresses/{:address}₀,...,{:address}ᵩ

It returns received and received_usd for each address which are used to populate the table shown above.

[ghost]

@ripcurlx @wiz The functionality can almost be achieved using Electrum (& EPS), as shown below. The only downside is that we want to see the recevied values, not the balances. If there are any spends from a repayment address, the balance no longer makes sense. Still, a watching wallet is useful to keep track of the repayment mechanism's progress. As noted above, it seems odd to me that two addresses are not being paid to, perhaps we need to check the bisq configuration.