Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scaling collected_torrents #3960

Closed
Dmole opened this issue Oct 13, 2018 · 7 comments
Closed

scaling collected_torrents #3960

Dmole opened this issue Oct 13, 2018 · 7 comments

Comments

@Dmole
Copy link
Contributor

Dmole commented Oct 13, 2018

> du -sc ~/.Tribler/collected_torrents 
973M	~/.Tribler/collected_torrents
> strings ~/.Tribler/collected_torrents/*.ldb | grep files | wc -l
19877

That is not going to scale well, I recommend only storing torrents for subscribed channels.
Or share the Max Disk space setting from token mining.

@devos50
Copy link
Contributor

devos50 commented Oct 14, 2018

We recently fixed an issue where torrents were not collected at all, which explains why the size of this file is growing. IIRC, we had some code to remove old torrents from the leveldb store but I'm not sure what happened to that. Also, there used to be a configuration option to limit the total number of torrents in the store (this was 50.000 by default I think).

Storing only torrents for subscribed channels is a good idea. Another solution could be to only store popular torrents since they are more likely to be downloaded by the user. Maybe we could combine this with the popularity overlay that @xoriole made?

@Dmole
Copy link
Contributor Author

Dmole commented Oct 14, 2018

Maybe 2 options;

  • prioritize [recent, popular, subscribed]
  • limit up to [X] MB

and a "clear cache" button

@Dmole
Copy link
Contributor Author

Dmole commented Oct 14, 2018

rm -r .Tribler/collected_*

reduced the size from 1.2 GB to 7MB.

@Dmole
Copy link
Contributor Author

Dmole commented Oct 14, 2018

.Tribler/collected_* should be moved to $Downloads with credit_mining.

@devos50
Copy link
Contributor

devos50 commented Oct 16, 2018

Related to #1150

@Dmole
Copy link
Contributor Author

Dmole commented Feb 5, 2019

#4199

@ichorid
Copy link
Contributor

ichorid commented Feb 23, 2019

Fixed by #4090

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

3 participants