First Credit mining prototype

[ardhipoetra]

This PR handles first protoype of auto credit mining in Tribler. Actually, it has been mentioned in #573.

Some GUI components change in home screen. Instead of showing most recent/popular video, list of popular channels is shown. Selecting (or unselect) those checkboxes will affect what channel that we will mine.

A new GUI page called credit mining is there. Basically it didn't change much from #23. I just changed the adding/removing sources part (especially channels) to be done in single click. And now, each source has it's own statistics, and I keep the total statistics in the top part of the screen. Some source info also available in the right part.

More in the core:
When Tribler start the credit mining process, it will (by default) look for swarm that has less seeder ratio compared to all the peers. Tribler will auto-download those swarm, and maximize the ratio credit if possible. We also determined the ratio we want.

Some progress shown in #1842. It includes (also some works that need to be done):

• revamped some GUI's (cc80e26)
• Fixing known bugs from previous work (415efe5)
• Dead tracker reduces performance (blacklisting OR completely integrate with central tracker management)
• DHT-only support (not perfect but will do for now) (ec2586b).
  Basically, it translates information we can get from peers discovered by DHT to swarm health (seeder/leecher)
• Main download (in download screen) aware mechanism (cee0d5d).
  We don't want main downloading activity become slower because of mining. Also, continue to download some stuff that already mined.
• DHT Performance evaluation -beta (nothing to do with Tribler code)

TODO list on feedbacks :

Core :

• called checkpoint twice feedback. Inherited from @whirm branch. I have pointed out (probably) the reason above.
• pointless touch channel instance? this. Inherited from @whirm branch.

Test :

• Not to boot up complete session in here. I've tried but this is the least I can get it to work.
• Use MockObject here. I do have some mock object by my own. It's quite complicated to use lambda so I stick with mine for now.

Other :

• twistd plugin on (deleted) boostchannel.py

[tribler-ci]

Can one of the admins verify this patch?

[devos50]

add to whitelist

[devos50]

@ardhipoetra nice! You should probably rebase since I see some commits that have been merged into devel already 👍

[ardhipoetra]

@devos50 I'm thinking on rebasing after #2038 is merged. I will check something with that part anyway

[leahparsuidualc-zz]

Though it isnt the right place to ask, would you point out the most necessary steps to generate this explicit build on ubuntu-based systems? So to say, a generic step-by-step that can be used to test a build like this, for one that has no overview of the given infrastructure but wishes to contribute in terms of testing with usable reports.

I am right now switching forth and back for some days already between Ubuntu Trusty and Xenial in 32- and 64- -bit in different flavors ( Gnome | Mate | Unity | XFCS | LXDE | Cinnamon, mixed in installed variants for the official releases, as also live ones ( w/wo persistence ) and will grow my personal testing grounds to pi- and pi2- based distros in future and add linux mint rosa and latest, as also win10, 8.1 and 8 to the setups; hardware-base will span Core2Duo | Athlon X2 64 | i7 | P4 | pi-arm-v7 and whenever i find the time a galaxy s4-I9505(Krait)|Note3 running diverse CustomROMS (Omni|CM|PAC);

So the detailed question would be:

• what is the perfect testing setup, explicitly for Tribler-RELEASE and Tribler-TESTING-Builds (like the one mentionened in here)
• how to organize the BUILDS, so (if possible) an automated test-run-batch would deliver results to you developers
• how to BUILD and COLLECT RESULTS manually, so you the developers can profit from it?
• if possible, can that in anyway combined with the github-account or tribler-name, so it can be found easily afterwards

I hope my request is understandable.

Background is, i am running tribler, for a (lil bit longer :) ) while now VDSL|ADSL|ISDN|GPRS|EDGE|UMTS|LTE-based and am constantly looking for setups that are working, as i wish to use tribler as THE ONLY tool to leech (forever) and as soon as possible seed (forever);

Just to mention it if it helps, space given is 750GB 15k Barracuda SATA|SAS per above listed CPU-Architecture that i am willed and right now spending for these test-|use-cases. (up to 8 Drives are unused/available for Tests)

Please, point me in the right direction or make use of this request to overhaul your knowledge and compress it with precise guides into a as-simple-as-possible-follow-up for the WILLED.

From my side: i'm interested in rolling by your side as long as possible, no time-limit.

Thanks for your interest.

[ardhipoetra]

@claudiusraphael, what do you mean by "this explicit build"? are you interested in this particular (credit mining) work? I see your request rather general than specific. I suggest you to create new issue discussing your request

[synctext]

@claudiusraphael quite an interesting post! Early in-depth testing of our releases is something we've struggled with for 10 years, really. Ever since the start of Tribler the most difficult task has been reproducing bugs, especially the once that occur only occasionally or in very special cases.

It would be great if you could spend hours trying out all the various features of Tribler manually and on multiple OSes, making you the official Quality Assurance Release Officer :-)

It might take quite some back and forth emailing to reproduce bugs. For example, if you had an outdated version of VLC on your Windows computer, Tribler would give a very peculiar error. With a lowest-level possible 50MByte trace file our chief developer tracked this down to a special place in windows DLL hell. Seems Windows didn't respect DLL loading orders. Tribler could use more testing in general with anti-virus tools and firewall stuff that tries to do smart things and messes with Internet connectivity.

[synctext]

@ardhipoetra Impressive work! Solid master thesis material.

[whirm]

Though it isnt the right place to ask,

Please, create a new issue and move this discussion there.

I'll try to respond your questions there.

Thanks!

[whirm]

@ardhipoetra also, don't waste time on improving the gui as we are ditching it in favor of the new Qt one @devos50 is working on, I would just do the minimal necessary to show your work.

[leahparsuidualc-zz]

@ardhipoetra : You asked, whether the guideline for building a specific branch | pull request | Version is aimed at Tribler in general or this specific instance (credit mining);

One as well as the other, should be the correct answer from my side.

As i already tried to describe: i am interested in compiling Tribler builds and (if available) run (hopefully batch ...) scripted Unit-Tests or do manual Testing programmatically, so a result can be gained, that can be of worth to you, the developers, for 'Tribler in General' and for those builds, where i (or you) think they should be tested in much detail as possible, for sure especially those that wake (my) special interest, like the 'credit mining'- -variant.

Why would one do that?

• Because i like to learn about the workflow you guys established for tribler and how to automate parts of that to make testing for generic, as also specific builds, easier | possible even for those (that like to help and .. ) that have absolutely no idea what has to be done.
• Because i think a very specific build like the credit-mining one given here, should end up finding themselves in general tribler; So it might be helpful to have some testing done, that one (or many) could easily setup and run or manually do step by step, to gather rich results for you, the developers.
• Because i wish to see Tribler as the-one-cross-platform-anonymized-torrent-sucker, with tasty features and a heck of possibilities to ruin global players day.

Just as an explanation ...

To answer your question in necessary detail: I'd like to know about the exact differences between the generic tribler and the credit-mining-wip and the instructions that enable one to build | patch | publish results in developer-friendly manner (and learn how to interpret the results in forefront to ease the creation of new issues).

P.s.:
"Btw.: Sorry, that it took some time to get back to this; My youngest (4 month old) daughter is a lil' bit sick right now for nearly 2 weeks already, so it might be that it takes time again to look into github."

@synctext @whirm @ardhipoetra However new Issue opened for the request for testing-environment-setup.

When the mentioned generic build- | patch- | publish- -guidelines will be given, it would be helpful if at this place (because it's the origin of my interest and the original question) some differential and relative steps in comparison to the generic-approach, could be left for future followers and testing-willed, so these can easily find the steps that have to be taken.

[ardhipoetra]

@devos50 I just did the rebasing with the current devel branch, can you check it out maybe I have done something silly in my branch?

[devos50]

@ardhipoetra you probably made a mistake during rebase since I see commits made by other team members. Make sure you pick the right commits to replay during the rebase.

[ardhipoetra]

@devos50 commits performed by other team members is exist because I continued the work from whirm branch, so there are some commits by him that not merged yet. What do you think about the overall content?

[devos50]

@ardhipoetra ah I see. I will do a code review as soon as possible.

[tribler-ci]

Can one of the admins verify this patch?

[ardhipoetra]

@devos50, I have fixed some violations in pylint. There's still some, though. Can you take a look at it?

[devos50]

I think these lines should be removed. It's probably not good to specify custom pylint behaviour (we have a nice .pylintrc file for that).

[ardhipoetra]

That's what I thought as well. But it's already there before I'm writing the code. But, yeah, will do.

[ardhipoetra]

@whirm, The squashing is halfway done. I expect around 10-15 commits at the end. I also separated GUI and Core commit. I will finish this before tomorrow afternoon. Is that okay with you?

[synctext]

OK. Tuesday is sort of our last point, as we need to do the experimenral release real soon.

[ardhipoetra]

@whirm, @synctext, it's ready now. I trim it up until 16 commits excluding your stuff (8 commits). I rebased it several times, and I think it quite stable.