question : dht_get_peers python binding?

[ardhipoetra]

Is there any way to do get_peers query explicitly in python binding? I saw dht_get_peers definition in bindings/python/src/session.cpp that (maybe) call dht_get_peers in session_impl.

Is this function can be called from session object? how to expose it so I can call it (from where doesn't matter). To be honest I can't understand how the binding works. The way it's defined is the same as is_dht_running (picked up as example), but it just won't work.

Thanks!

[aldenml]

I think you should be able to call this function the same way as others, this method is async and the corresponding alert is dht_get_peers_reply_alert

[ardhipoetra]

Unfortunately, it seems not the case. I run the program below and it shows dht_get_peers is not available in python binding.

Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import libtorrent as lt
>>> lt.version
'1.0.9.0'
>>> s = lt.session()
>>> s.start_dht()
>>> s.dht_get_peers()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'session' object has no attribute 'dht_get_peers'
>>> s.is_dht_running()
True
>>> 

[aldenml]

I see your version is 1.0.9.0, this is for version >= 1.1.0, branch RC_1_1 or master.

[ardhipoetra]

will take a look for that, thanks!

[ardhipoetra]

That's weird, I can't get any alerts you mentioned. Full code in here, but my step is :

import libtorrent as lt
import time

infohash_str = "5aec5ee9f2d044316fe1de29d221452103ceb958"
ses = lt.session()
ses.start_dht()
ses.add_dht_router('router.bittorrent.com', 6881)
ses.add_dht_router('router.utorrent.com', 6881)
ses.add_dht_router('router.bitcomet.com', 6881)

ihash_sh1 = lt.sha1_hash(infohash_str)

ses.dht_get_peers(ihash_sh1)

stopped = False
count = 0
while(not stopped):
    al = ses.pop_alerts()

    if not len(al):
        time.sleep(5)

        # furiously ask for peers!
        ses.dht_get_peers(ihash_sh1)
    else:
        for a in al:
            print a
            count += 1

        time.sleep(1)

    # we've seen enough
    if count >= 30:
        stopped = True

What do you think?

[aldenml]

Well, I don't have the environment to test with python, but I just coded a java unit test (see https://github.com/frostwire/frostwire-jlibtorrent/blob/libtorrent-RC_1_1/src/test/java/com/frostwire/jlibtorrent/DhtGetPeersTest.java) and it works.

Try to get alerts related to DHT stats to see if indeed you are populating your DHT tables

[ardhipoetra]

Find it, I have to put alert mask in session settings before I can get the alerts. Updated my gist

Next problem is (probably specific to python binding) :

• it doesn't have type/alert category dht_notification, dht_operation_notification (or maybe it's just me who missed it). Filtering with raw value seems work though.
• alert retrieved from dht_notification still in alert type. Therefore, I can't call peers and num_peers function.

[aldenml]

Yes, a lot missing, working in a PR to add support for dht_get_peers_reply_alert.

[aldenml]

Ok, it's merged #608, give it a try

[arvidn]

Does that PR cover all issues raised in here?

[ardhipoetra]

I still can't call peers(), and num_peers act as a attribute instead of function.

It does have type which is libtorrent.dht_get_peers_reply_alert though

[aldenml]

Does it make a difference if they are a function or a property for you? I added both as properties following other alerts as example. Are you able to see them in the properties of the Python object?

[arvidn]

I think it would make sense for them to be functions, to map 1:1 to C++ (the fewer differences between the C++ and python API, the less important it is to have separate documentation for the python bindings)

[aldenml]

Well, I was thinking about this (API design wise). I agree with the documentation point, but I perceive that one reason to have num_peers() as a method, is the lack of convenient read-only property definition in C++ (the version in use) and the reason for peers(std::vector<tcp::endpoint>& peers) is mainly performance, since it involves deserialization of addresses (done only on demand). It seems to me more natural to have them mapped as properties in python (same for other languages).

Of course, this is completely subjective and I don't mind looking for the 1:1 map if you think otherwise.

[arvidn]

actually, when I looked at this earlier today, I realized that peers() really should return the vector. At some point I'll deprecate this one and add an overload that does that.

[arvidn]

@ardhipoetra could you give this a try? #617

[arvidn]

#617 was merged

[ardhipoetra]

Sorry for the late response. I can confirm that the new patch fix this issue.

However, talking about performance, it's very hard to get peer reply on the request.

In approximately 30 minute with several swarms, I just get 1 reply of the get_peers request. Is there something wrong with my code?

[aldenml]

Hi @arvidn, where do you like a possible PR to deprecate the current peers signature, RC_1_1 or master?

@ardhipoetra have you tried a very popular hash, like the one from a recent ubuntu distribution?

[ardhipoetra]

@aldenml, yes I have. I've tried some from linuxtracker and other common torrent sharing site. I picked up with > 1500 seeder and > 500 leecher

[aldenml]

I can't give you an ETA, but I will try to setup my python environment. In my java tests I get peers in less than 5 seconds.

[arvidn]

@aldenml RC_1_1 would be great

[Ueland]

Did you get around to check the missing replies @aldenml , i did some quick testing on the code mentioned in this thread it appears that more or less no replies comes back. (ports are open etc)

[aldenml]

@Ueland this issue is very old, but I just tested again and I have no problems. Do you have a minimal code to look at it. Async calls and polling alerts can be a little tricky.

[Ueland]

@aldenml Sorry about that, but since the isue was discussed here i took a chance to ask here, let me know if i should move to a new issue :)

I have updated OPs gist with a new popular hash for a Ubuntu release:
https://gist.github.com/Ueland/934e87898b362c474f1a7f5cf23d29d9

There appears to be a couple of thousand peers for that hash.

I let it run for 2-3 minutes, in that time it sent 327 outgoing get_peers, and it recieved 0 dht_get_peers_reply_alert back for that hash. (but i do get other get_peer requests, announces etc)

[aldenml]

@Ueland is this running with 1.1? because I only test (and deal with)master for a while now

[Ueland]

@aldenml version : 1.1.6.0 to be correct. Is there any specific branch i should try and build to check?

[aldenml]

try master and let me know, my tests are in java

[Ueland]

Ran it for two minutes now, the results are after two minutes 10s:
version : 1.2.0.0
outgoing dht get_peers: 291 times
incoming dht get_peers: 34 times
incoming dht announce:: 2 times

Of the 34 incoming dht get_peers, 20 was for the same hash that i requested, could it be that the messages that should be dht_get_peers_reply_alert are treaded as an incomin dht get_peers?

[aldenml]

strange, I get the dht_get_peers_reply_alert like immediately in my java app. That # furiously ask for peers! comment in your code does not inspire me confidence. I will try to code something in python to verify.

[aldenml]

Hi @Ueland I looked at it, but unfortunately I can't get it work, the problem is that your script uses deprecated API, I tried to create the session with the new settings API, but something is seriously wrong, the settings are not applied. I don't have a proper python environment to continue in a short time, but I can confirm again that this works with my tests in java.

[Ueland]

I see, can i see the java test code somewhere? To see if i can figure it out :)

[aldenml]

I use this tool: https://github.com/frostwire/frostwire-jlibtorrent/blob/master/src/test/java/com/frostwire/jlibtorrent/demo/DhtShell.java