Bug: Paired devices are not showing unless on same network

[acupofjose]

Describe the bug
I get the error "Paired devices are not persistent." when attempting to link two instances that have been installed as a Chrome PWA.

In the Console it says: "This browser does not support IndexedDB. Paired devices will be gone after the browser is closed."

To Reproduce

• Install PWA on Computer 1
• Install PWA on Computer 2
• Get Code from Computer 2
• Pair Code on Computer 1
• Message: "Paired devices are not persistent."

Expected behavior

• Devices should pair and persist connections.

Desktop (please complete the following information):

• OS: Mac and Windows
• Browser: Chrome
• Version: 110.0.5481.177

Self-Hosted
Yes

Self-Hosted Setup
Proxy: Nginx Proxy Manager
Deployment: docker-compose

[schlagmichdoch]

Is there simply a message or are they in fact not persistent? If you ever see the unpair button it means that the devices are persistent and it‘s a wrong error message.

[acupofjose]

hm. Some more debugging shows the following:

WS: {type: 'pair-device-initiated', roomSecret: '----', roomKey: '430048'}
network.js:66 WS: {type: 'peers', peers: Array(0), roomType: 'secret', roomSecret: '----'}
network.js:66 WS: {type: 'pair-device-joined', roomSecret: '----', peerId: '47016234-f559-4234-aae4-669f389a6eed'}
network.js:66 WS: {type: 'peer-joined', peer: {…}, roomType: 'secret', roomSecret: '----'}
ui.js:1766 Request successful. RoomSecret added: ----
ui.js:1697 This browser does not support IndexedDB. Paired devices will be gone after the browser is closed.

The devices are indeed persisted through page load and browser close. And discovery happens while on different networks:

ICE Gathering connected
network.js:655 RTC: state changed: connected
network.js:584 RTC: channel opened with 0c0cc604-1658-455c-a62b-4339f15215c7
network.js:597 RTC: {type: 'display-name-changed', displayName: 'Moiraine'}
network.js:597 RTC: {type: 'display-name-changed', displayName: 'Moiraine'}
network.js:66 WS: {type: 'peer-left', peerId: '47016234-f559-4234-aae4-669f389a6eed', roomType: 'ip', roomSecret: '', disconnect: false}
network.js:66 WS: {type: 'peer-left', peerId: '47016234-f559-4234-aae4-669f389a6eed', roomType: 'secret', roomSecret: '-----', disconnect: false}
network.js:66 WS: {type: 'peer-joined', peer: {…}, roomType: 'ip', roomSecret: ''}
network.js:66 WS: {type: 'peer-joined', peer: {…}, roomType: 'secret', roomSecret: '-----'}

However the page shows no devices unless they are on the same network.

My docker-compose.yml is as follows:

version: "3.3"

services:
  server:
    container_name: pairdrop
    image: lscr.io/linuxserver/pairdrop
    restart: unless-stopped
    networks:
      - nginx
    ports:
      - 127.0.0.1:3000:3000
    environment:
      - WS_FALLBACK=true
      - PUID=1000
      - PGID=1000
      - RATE_LIMIT=false
      - TZ=America/Chicago
      - RTC_CONFIG=/rtc_config.json
    volumes:
      - ./rtc_config.json:/rtc_config.json

networks:
   nginx:
     external: true

[acupofjose]

Sorry, as I started debugging things got muddied!

• No, it is not specific to the PWA. Same thing happens on chrome and safari
• The device paired devices while on different networks are recognized in the console, but don’t show up on the page with the green dot under them unless on the same network

I’ll try the regular hosted one and see if it happens there!

[acupofjose]

Hm. Same thing happens on the hosted version too. I wonder if it’s the local network blocking something? It is a school network.

[acupofjose]

@schlagmichdoch thanks for debugging with me! I'll try this route.

[marcogreiveldinger]

As an idea for the VPS, you could start with an oracle always free instance. They go up to 24gigs of RAM. But I think the bandwidth is limited per month. @schlagmichdoch

[Robin-Sch]

Pairdrop with websockets enabled shouldn't require a TURN server? But still getting the same error

[Robin-Sch]

@Robin-Sch the websocket fallback is only used if one of the two devices does not support WebRTC as is the case on some vpn connections. In your case WebRTC seems to be supported by both devices but the connection fails as the turn server is not reachable. If you want the fallback to be used additionally if the rtc connection fails i would appreciate if you would create another issue as a feature request

I'm using my own pairdrop instance, websockets enabled. One browser with webrtc disabled, one browser with webrtc enabled (so it should use websockets). It's still getting that error.

[schlagmichdoch]

That's weird.. When both of your devices are on the same network and they are shown to each other, are they shown with a red border indicating the websocket fallback? Otherwise WebRTC would be used.

[Robin-Sch]

That's weird.. When both of your devices are on the same network and they are shown to each other, are they shown with a red border indicating the websocket fallback? Otherwise WebRTC would be used.

Same network: WebRTC enabled: does work
Same network: WebRTC disabled: does work (red border)
Different network: WebRTC enabled: doesn't work (gives the error Paired devices are not persistent.)
Different network: WebRTC disabled: does work (but gives the error Paired devices are not persistent.) (red border)

[schlagmichdoch]

Okay, thanks a lot for checking back! That confirmes the current state, that the TURN server is broken but everything else works. I will have to look into why the error message about paired devices not being persistent is showing (are you using Firefox private tabs?). Otherwise your issue will be resolved as soon as the TURN server is up and running.

[schlagmichdoch]

I have since deployed a new TURN server for pairdrop.net that enables sending between devices that are behind different NATs. Does this resolve your issue @Robin-Sch ?

@acupofjose Have you tried deploying your own TURN server?

Try following this guide: https://gabrieltanner.org/blog/turn-server/
You can either install coturn directly on your system (Step 1) or deploy it via docker-compose (Step 5).

[xundeenergie]

i just made a pull-request #106 for the docker-compose-coturn.yml with some changes around to get the turn, turns, stun and stuns working.

Now you can deploy a pairdrop and coturn-server which are wired together (you still can use the coturn with other services too!).

What's not working is to listen on port 443 for the turns coturn-part, because the nginx is listening there too..

Maybe this fixes this bug too...

[Robin-Sch]

I have since deployed a new TURN server for pairdrop.net that enables sending between devices that are behind different NATs. Does this resolve your issue @Robin-Sch ?

I still get the error when linking the two devices

[schlagmichdoch]

So you’re getting the error, that device pairing is not persistent. Does everything else work though? So is it a false warning? I guess the false warning about persistence is probably a different issue

[Robin-Sch]

I get the error and it doesn't work, just like before

Same network: WebRTC enabled: does work
Same network: WebRTC disabled: does work (red border)
Different network: WebRTC enabled: doesn't work (gives the error Paired devices are not persistent.)
Different network: WebRTC disabled: does work (but gives the error Paired devices are not persistent.) (red border)

[schlagmichdoch]

Have you specified a turn server?

[Robin-Sch]

I'm getting the error also on the official instance (which has a turn server enabled?)

[schlagmichdoch]

I could reproduce your issue on chromium based browsers and debugged my turn server.
Not only is it now working with chromium browsers but the turn server is now served via tls (turns:).

Could you please test again whether the official instance does now work for you? I will then update the turnserver_example.conf to include the changes.

Additionally, #106 will fix the possibility to host your turn server easily via docker-compose together with PairDrop.

[xundeenergie]

You can also try my server https://pairdrop.schuerz.at

Coturn is listening for turns on port 443.

[Robin-Sch]

Could you please test again whether the official instance does now work for you? I will then update the turnserver_example.conf to include the changes.

Still gives the error, but it does connect

[xundeenergie]

Could you please test again whether the official instance does now work for you? I will then update the turnserver_example.conf to include the changes.

Still gives the error, but it does connect

If you want... i did some changes again... test against my instance.

[schlagmichdoch]

Thanks for testing @Robin-Sch !
So the issue with the TURN server is fixed then. Great!

About the problem with persistent storage on Google Chrome:
Google Chrome has an interesting approach to persistent storage. It will only be persistent if one of the following is true:

• The site is bookmarked (and the user has 5 or less bookmarks)
• The site has high site engagement
• The site has been added to home screen
• The site has push notifications enabled

And only after the script has explicitly asked for permissions.

I implemented this on #108 which is live on this test instance: https://pairdrop-persistence.onrender.com

After enabling notifications (via the bell) or adding the page to your bookmarks, device pairing and name changing should work persistently.

Would you please test one more time?

[luckman212]

(moved)

[Robin-Sch]

Would you please test one more time?

WebRTC disabled (firefox): shows that pairdrop requires webrtc enabled to work.
WebRTC enabled (chromium): always shows the error below and doesn't connect, even when bookmarked and/or notifications allowed.
WebRTC enabled (firefox): it asks if the site can save data in the persistent storage. Clicking allow and clicking block both show the same "error" below, and doesn't connect.

Paired device is saved for this session only.

Google Chrome:
Save page as bookmark or enable notifications to enable persistence.

[matiasperaltt]

good day

I am seeing that I have the following problem in relation to all this:

Scenario A is correct, scenario B is wrong:

Scenario A:
1- iphone (Local Wi-fi)
2- PC (local wi-fi)
the connections appear and everything works correctly

ScenarioB:
1-iphone (4g-Data)
2-pc (local Wi-Fi)
In this scenario I perform the pairing using the 6-digits, on the screen there is a message "the pairing is correct".
But the device icons are not displayed on the screen

I attach the screens with the scenarios:

[Robin-Sch]

Please also test on https://pairdrop-persistence.onrender.com/, which is running with the following PR: #108

After enabling notifications (via the bell) or adding the page to your bookmarks, device pairing and name changing should work persistently.

[matiasperaltt]

Hello,
after entering https://pairdrop-persistence.onrender.com/ and I makes the same scenarios A and B and the result is exactly the same scenario A works correctly and scenario B same same error

[Robin-Sch]

That's weird, because on that instance I got a different error: (see #71 (comment) )

[schlagmichdoch]

@matiasperaltt I guess that was an issue with the TURN server. Thanks for reporting! Please test the official instance at pairdrop.net again.

@Robin-Sch Is the pairing process on Google Chrome still not persistent for you using pairdrop.net? I cannot reproduce the issue on my machine anymore.

[Robin-Sch]

It's working right now

[Sschhsd]

Hi all,

i have the same issue. Pairdrop works fine with all devices in same network, but its doesn't work for different networks.

I found the following error message in the developer options:
RTCPeerConnectionIceErrorEvent {isTrusted: true, address: '[0:0:0:x:x:x:x:x]', port: 57846, hostCandidate: '[0:0:0:x:x:x:x:x]:57846', url: 'stun:stun.l.google.com:19302', …}

@Robin-Sch: It works fine with your instance with devices in same and differents networks. I use the default configuration with Docker Compose.

Any ideas?

[Sschhsd]

Hi @schlagmichdoch, thank's for the reply.
I used the default PairDrop Configuration for Docker Compose and didn't deploy a TURN server.
I will wait for the new release and will take a look to the step-by-step guide.

[schlagmichdoch]

The following comments were deleted by GitHub (via hubot) as part of mistakenly marking this account as spam on 17th February 2024. The correct thread order and the creation date is unclear. I decided to manually restore them anyway in order to complete the information this issue holds even though the restored information might be outdated:

Comment by @schlagmichdoch:

thanks for the additional information!

And discovery happens while on different networks

no devices unless they are on the same network

Not sure whether I understand correctly.
When you reopen the PWA are the devices displayed with a green dot below? Then the pairing itself is working and the paired devices are persistent.

When you are on different networks and the devices are not visible to each other, is this unique to the Chrome PWA or is this the case for all browsers of the device? Are the devices becoming visible to each other when you pair them again? If not, that's probably a different issue with your specific network constellation.

As you are self-hosting:

• Are you confident that your PairDrop instance is reachable on the different network constellations?
• Does your issue also occur, when you use the instance on pairdrop.net or is this specific to your instance?

Comment by @schlagmichdoch:

So after writing the following debugging instruction, to check how to find out that something is not working, I tested it myself and found out that the free public TURN server seems to be down for now I'm afraid. I also believe it's likely that it will not come up again. That means that connections between devices behind different NATs are not possible with the default settings right now 😞

Of course you can easily setup your own TURN server and specify to use it via the RTC_CONFIG=/rtc_config.json environment variable. You can also get a free TURN server here and use its credentials on your instance.

As a workaround for pairdrop.net I will use credentials to another free turn server that has a limited quota.

As this quota will not be enough for all PairDrop users by far, I will setup an official TURN server on a VPS soon. Big advantages are that the speed for these relayed connections will increase a lot and that will be under PairDrops control.

Disadvantage is the cost: With the smallest RAM It will create additional costs of 30-40 € per year. Would appreciate any support.

---

Your new info confirms what I thought: It has nothing todo with the PWA or the specific browser but with the network. Your devices must be able to create webrtc connections. You can check here: https://www.webrtc-experiment.com/DetectRTC/#isWebRTCSupported (under Browser Supports WebRTC (Either 1.0 or 1.1)? it must say Yep. Some networks or VPN connections somehow disable all webrtc connections.

Addionally you must be able to connect to the given stun servers. If there is no NAT between you devices googles STUN server (stun:stun.l.google.com:19302) should be enough otherwise you need to able to connect to the given turn server (turn:openrelay.metered.ca:443, username: openrelayproject, password: openrelayproject).
You can test this independendly from PairDrop here: https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/ :
For Google STUN: After clicking on Gather candidates it works if you get a candidate with type srflx.
For OpenRelay TURN: After clicking on Gather candidates it works if you get a candidate with type relay.

Comment by @schlagmichdoch:

@Robin-Sch the websocket fallback is only used if one of the two devices does not support WebRTC as is the case on some vpn connections. In your case WebRTC seems to be supported by both devices but the connection fails as the turn server is not reachable. If you want the fallback to be used additionally if the rtc connection fails i would appreciate if you would create another issue as a feature request

@marcogreiveldinger thanks for the suggestion! I will look into it. I’m not sure though about the bandwidth limit. If many more people are using pairdrop.net in this device constellation 1 TB bandwidth per month is not much. 1 TB is only 1000 users sending 1 GB of files or 100000 users sending 10 MB of files. Even now, without any data relaying, 100GB per month will soon not be enough. So probably I will get some VPS without any bandwidth limit I need to worry about. I have sth in mind but need to check some reviews first :)

Comment by @schlagmichdoch:

@Sschhsd FYI:
I’ll deploy a new version of PairDrop this week which (among a lot of code structure changes) also includes clarifications regarding the documentation and fixes for the docker files.
Either wait until it is merged or take a look at the PR (#209) or better the feature branch here:
https://github.com/schlagmichdoch/PairDrop/tree/next

Have you deployed a TURN server? This is needed for some connections across networks. The new documentation includes a step-by-step guide how to deploy a TURN server alongside PairDrop via docker compose

If you have any questions regarding that, feel free to open a new issue