Losing tokens without downloading

[noughtmare]

I have no downloads but I am still losing tokens, how can that be?

I can reproduce this on version 7.2.2 and the 'next' branch from github.

The screenshot is made while I was using revision 407afd3

[synctext]

Yes, it is strange behavior.

We are aware of the unexpected issue of running into negative credits. We very much want to fix this.
The issue seems to be the seeding to non-Tribler peers through the Tor-like exit node network. That leaks more then we expected.

We'll be making the feature: "only seed for credit income". Without receiving a credit report, you will not upload. Probably should be on by default.

Duplicate of prior report: #4161

[noughtmare]

But I have downloaded nothing, my download list is completely empty, so I don't think anything can be seeding.

[synctext]

Ehhhh, actually.. I have no idea. Anybody any theories?

[MeadowHillSoftware]

Bug filer probably didn't look at the credit mining tab.

[noughtmare]

I have disabled credit mining and I did look at the credit mining tab, it was empty.

[noughtmare]

@MeadowHillSoftware I do not mean for this issue to be about the credit mining.

I am now trying to reproduce this on a clean installation where all downloading and credit mining is disabled from the beginning.

[synctext]

More info and details are helpfull, thnx! Indeed the status of the credit mining tab is important. You can also upload/earn + download&invest from there.

[noughtmare]

I have now removed my ~/.Tribler folder and restarted and reconfigured tribler and unchecked the token mining option in the settings. But I have again "24 MBytes taken from community" and I expect that this number will grow if I keep tribler running for longer.

So I guess anybody could reproduce this issue by just installing a clean version of tribler or removing their settings folder and then just letting tribler run for a few hours. That will result in taking MBytes from the community without downloading anything.

The full steps I took were:

1. remove ~/.Tribler folder
2. start tribler
3. edit settings:
   3.1. disable "family filter enabled?"
   3.2. disable "download anonymously using proxies"
   3.3. disable "encrypted anonymous seeding using proxies"
   3.4. set seeding options to "Unlimited seeding"
   3.5. enable "allow tribler to be an exit node"
   3.6. disable "enable token mining"
   3.7. click save settings
4. add port = 7759 to ~/.Tribler/triblerd.conf as suggested here
5. make sure port 7759 is forwarded in router and port is allowed in firewall
6. restart tribler
7. wait for a few hours

After that my trust statistics look like this:

Update, now my trust statistics look like this:

So my total amount of tokens is positive, but the amount I "took" from the community still grows.

[devos50]

I will try to debug this one. First, I'm trying to reproduce it myself by running Tribler idle for an extended period of time.

If anyone wants to help me out, it would be helpful if one could send me a copy of their TrustChain database if the issue is reproduced (otherwise I will be debugging TrustChain records which balances should be correct). This database can be found in the Tribler state, under the sqlite directory and is named trustchain.db.

@noughtmare the graph you linked shows an interesting thing. The first transaction correctly increases your balance with around 9MB. In the next transaction however, it seems that a transaction is initiated where you only take MBs. Do you still have this database around?

[noughtmare]

@devos50 This is a zip of my trustchain.db file (I can't upload .db files to github directly, so I zipped it). I believe that still includes those first transactions.

trustchain.zip

[devos50]

Thank you! I will inspect the file 👍

[devos50]

@noughtmare I need one more piece of information, namely your public identity that TrustChain uses. You can find that identifier if you enable the developer mode in the settings panel (under the 'debug' panel). Then you should open the debug pane in the left menu. If you then navigate to the 'Trustchain' pane, you will see a field named id. Could you post that one here?

It should look something like this: https://gyazo.com/5ed5e2a2abfbe1422c8e6e3aed51c339

[noughtmare]

4c69624e61434c504b3a2fd335ccde4a9c298c1a69c1db7da7b633890bc644fadfdb16ea8ee1f04cde1954702643419ef7e1b7e95c7c38424c16bf417be783a1d6b55aa7daba9368596a

Maybe you should make it easier to copy those to the clipboard (it took me some time to type it out right).

[devos50]

Thanks!

I looked into this and it seems that there are occasionally transactions created which transfers tokens from you to some counterparty. The transaction contents of one of them looks like this:

a4d4bdown8i1355035210btotal_down8i135503522bup1i08btotal_up7i9756671

It's a bit hard to read but there is nothing uploaded and around 13.5MB downloaded, which matches with the graph you linked on this issue. Why you would initiate such a transaction without any active downloads is still unknown to me. I have to check the code to see how this is possible.

[devos50]

@noughtmare you mentioned that you did 3\. 3.5. enable "allow tribler to be an exit node". Please be really careful with this option, since you will start to serve (possibly copyrighted) material to others. I would highly recommend you to remove your full Tribler state directory and start Tribler again.

[noughtmare]

@devos50 Is it not just enough to turn of the option in the settings? I originally turned it on to see how much of a difference it would make to the amount of tokens that I would receive, especially since in the beginning I mostly lost tokens by staying connected to the network.

[devos50]

Yes that should also be enough but removing the state dir also removes any data attached to Tribler so it's more like the nuclear option.

[devos50]

I'm still not sure what's causing this. I will add an assert to the payout method and run some idle Tribler instances to reproduce this. The log output should give some more information then.

[synctext]

Please do not run as an exit node. There are some very bad torrents out there, worse then copyright violations.

[qstokkink]

Closely related, gaining tokens without uploading:

      {
         "public_key":"4c69624e61434c504b3a1abc3c31c1f94f75a4b0f57a30b23b7f577c1e35ffe5535d4317036c17b4ac1a28321cf14e024d4afbd56b2ef83e3d41afb36c73d2c5b02ac8856b2068079d35",
         "transaction":{
            "down":1368725,
            "total_down":4349145,
            "up":0,
            "total_up":1086161671
         },
         "hash":"85fc596207a84d71611808eb8a21f201c445c6a3071279b6879aba6e360fd854",
         "timestamp":1551100149971,
         "link_public_key":"4c69624e61434c504b3a2cac2c87405b47450200c435480a7217da3777c5dcf5a57ff197ca45608ccf755a2c7258990eeb31a27125b51965583d7096fd498c1ba215380fb5614c08ef3e",
         "type":"tribler_bandwidth",
         "insert_time":"2019-02-25 13:09:09",
         "signature":"268f778584fea0a81c8bbffdff5e8ffa5054c103063b60a5ef3859d4f98524537297675cabe646ce0f1590998bf21bcb505ac6bb5271690f3688e1e94ac02709",
         "previous_hash":"73f3cc074084146320bfed2370895c15ca7a3f796ffdb8b1ae5dbf7f124858ad",
         "link_sequence_number":0,
         "sequence_number":16
      },
      {
         "public_key":"4c69624e61434c504b3a1abc3c31c1f94f75a4b0f57a30b23b7f577c1e35ffe5535d4317036c17b4ac1a28321cf14e024d4afbd56b2ef83e3d41afb36c73d2c5b02ac8856b2068079d35",
         "transaction":{
            "down":0,
            "total_down":2980420,
            "up":48254400,
            "total_up":1086161671
         },
         "hash":"73f3cc074084146320bfed2370895c15ca7a3f796ffdb8b1ae5dbf7f124858ad",
         "timestamp":1551099880972,
         "link_public_key":"4c69624e61434c504b3ab0135c0dff5156147f4bc01bab3941390e173097ccdc35a4bb9768248864e30853a5f26f89a0607e3b18c1f7f29ed48e2060764c88355b4e2a4eef8e76bfd709",
         "type":"tribler_bandwidth",
         "insert_time":"2019-02-25 13:04:40",
         "signature":"e96172aeb577189b2b8e527dd359472ce5e64f8445f1de519e6bbb120cf1a3c1329c8a8f970ef93b9ac375a43a2a0d64e9c2fe42c0585a8b17118cc075354c09",
         "previous_hash":"59868cbebb53556b6ffc0abb3938bfdc02ceff3b394aa21105125a80df912056",
         "link_sequence_number":178,
         "linked":{
            "public_key":"4c69624e61434c504b3ab0135c0dff5156147f4bc01bab3941390e173097ccdc35a4bb9768248864e30853a5f26f89a0607e3b18c1f7f29ed48e2060764c88355b4e2a4eef8e76bfd709",
            "transaction":{
               "down":48254400,
               "total_down":8752378215,
               "up":0,
               "total_up":27086901
            },
            "hash":"4f71cce5179c0d9ec83c53d6bd03a32359ccbee35d592e50017ad37ad7f3e84d",
            "timestamp":1551099881944,
            "link_public_key":"4c69624e61434c504b3a1abc3c31c1f94f75a4b0f57a30b23b7f577c1e35ffe5535d4317036c17b4ac1a28321cf14e024d4afbd56b2ef83e3d41afb36c73d2c5b02ac8856b2068079d35",
            "type":"tribler_bandwidth",
            "insert_time":"2019-02-25 13:04:39",
            "signature":"1de2d3f804496bc4378384c0c925ee695355f840020673b8b8e8d584a4848ca185cb3386207b9197fa97721b90dd48c302f4d455d8fdeb4b3380da4cea5b3006",
            "previous_hash":"89252bfd9c25e837f2c09506027ab7f440febc9fa760d82c17c26bfab94f3159",
            "link_sequence_number":0,
            "sequence_number":178
         },
         "sequence_number":15
      }

I sent 1.3 MB of payout to the next hop and got paid out 48 MB.

EDIT: Maybe even more interesting, my total up/down of the IPv8 community (175 MB) is less than the >200 MB of bandwidth tokens I have earned this session:

[devos50]

Today, I fixed two issues related to payouts (in #4248 and #4250). These changes should make the payout system more robust.

That said, I still haven't figured out how (and especially why) balance is drained when only doing relaying activities. I have setup a server which has been running Tribler idle for a few hours. After these few hours, it has accumulated a balance of 659MB. I also print every single payout made. I've written a script that parses all payouts and reconstructs the final balances. This yields me roughly 968MB. So somewhere in the process, I have lost 309MB. I should do a deeper inspection of the database and backtrace every single payout made to further debug this issue.

[devos50]

After some extensive debugging, I think I might have found the bug! 🎉

If there is a downloader D, a relay R and an exit node E, the following could happen. If D has to do a payout of 100MB, it will send a half block to R, say that this half block has sequence number 10 in the chain of D. Now, if R receives this half block, it will try to validate the half block by requesting blocks 5-9 in the chain of D (assuming that R does not have these blocks already). Simultaneously, R continues the payout process (without full validation of the chain of D) and pays 100MB to E by creating and sending a half block to the exit node. Now, if D does not respond properly with blocks 5-9 as requested by R, the balance of R will be -100MB. I have seen this behaviour in the logs of Tribler.

So how can we fix this? I can think of two solutions:

• Wait for the transaction between D and R to complete before proceeding the payout process. This requires me to twistify process_block in IPv8 which should not be that hard.
• Disable verification of prior blocks in ones chain when doing payouts in the tunnels. This increases the risk of the double spend attack but this is already an issue anyway I think.

Any suggestions or other solutions for this?

[synctext]

Congrats!
Interesting bug. You respond to a signature-request with a crawl-request that is not allowed to fail. Can we schedule the latest blocks request, say, 300 seconds after tunnel has been established? Plus not block on the outcome. In further iterations we need to make this more resilient, reliable and accurate.

[noughtmare]

I'm still experiencing this on the tribler-devel branch, revision 897f548.

[noughtmare]

The trust statistics graph is pretty interesting:

The "taken from community" line is very jagged, indicating that those events happen right after every "given to community" event.

[devos50]

That’s unexpected. There’s something else apparently. I will look at it tomorrow...

[devos50]

After some thinking about this, I believe this is the way the mechanism should work. Let me first note that the graph might be slightly confusing since it implies that you take MBs from the communities. This is an implication of the way the payout mechanism works. We should improve the visualization a bit and discuss this.

When paying out bandwidth tokens, the downloader (which is at one end of the circuit) pays tokens to the subsequent hops in the circuit. Intermediate hops take these tokens and transfer a part of this full token amount to the next hop. So first you receive some tokens and then transfer some of them to the next hop, which will happen at roughly the same time (which explain the occurrence of the two orangish dots right above each other). The most important thing here is that the MBs given should always be bigger than the MBs taken when there are no active downloads.

You can find more technical details about the payouts in this document. I will do some more testing to see whether everything works as expected.