Newpipe (Youtube) not working in Russia (but official YT app works) #7114

azat11 · 2021-09-15T20:54:56Z

Checklist

I am using the latest version - 0.21.9
I checked, but didn't find any duplicates (open OR closed) of this issue in the repo.
I have read the contribution guidelines given at https://github.com/TeamNewPipe/NewPipe/blob/HEAD/.github/CONTRIBUTING.md.
This issue contains only one bug. I will open one issue for every bug report I want to file.

Steps to reproduce the bug

Open app
Try to search or open anything from history, and it's loading endlessly

Actual behavior

Nothing loads. Tried reinstalling, nothing changes.

Expected behavior

Everytrhing should load.

Screenshots/Screen recordings

https://ibb.co/d29FXVt

Logs

Exception

User Action: searched
Request: lemmino
Content Country: US
Content Language: en-US
App Language: en_US
Service: YouTube
Version: 0.21.9
OS: Linux Android 6.0.1 - 23

Crash log

java.net.ConnectException: Failed to connect to www.youtube.com/2a00:1450:4010:c05::c6:443
	at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:249)
	at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:167)
	at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:258)
	at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
	at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
	at okhttp3.RealCall.execute(RealCall.java:93)
	at org.schabi.newpipe.DownloaderImpl.execute(DownloaderImpl.java:264)
	at org.schabi.newpipe.extractor.downloader.Downloader.post(Downloader.java:131)
	at org.schabi.newpipe.extractor.downloader.Downloader.post(Downloader.java:114)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.areHardcodedClientVersionAndKeyValid(YoutubeParsingHelper.java:342)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.getClientVersion(YoutubeParsingHelper.java:429)
	at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.prepareDesktopJsonBuilder(YoutubeParsingHelper.java:814)
	at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeSearchExtractor.onFetchPage(YoutubeSearchExtractor.java:73)
	at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:54)
	at org.schabi.newpipe.extractor.search.SearchInfo.getInfo(SearchInfo.java:29)
	at org.schabi.newpipe.util.ExtractorHelper.lambda$searchFor$0(ExtractorHelper.java:81)
	at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$qyxpuXgomWa-cbONQns-pd7zxm0.call(lambda)
	at io.reactivex.rxjava3.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:43)
	at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
	at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
	at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:614)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:56)
	at java.util.concurrent.FutureTask.run(FutureTask.java:237)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:269)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
	at java.lang.Thread.run(Thread.java:818)
Caused by: java.net.ConnectException: failed to connect to www.youtube.com/2a00:1450:4010:c05::c6 (port 443) after 10000ms: isConnected failed: ENETUNREACH (Network is unreachable)
	at libcore.io.IoBridge.isConnected(IoBridge.java:234)
	at libcore.io.IoBridge.connectErrno(IoBridge.java:171)
	at libcore.io.IoBridge.connect(IoBridge.java:122)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:183)
	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:452)
	at java.net.Socket.connect(Socket.java:884)
	at okhttp3.internal.platform.AndroidPlatform.connectSocket(AndroidPlatform.java:73)
	at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:247)
	... 39 more
Caused by: android.system.ErrnoException: isConnected failed: ENETUNREACH (Network is unreachable)
	at libcore.io.IoBridge.isConnected(IoBridge.java:223)
	... 46 more

Device info

Android version/Custom ROM version: CyanogenMod 13.1
Device model: OnePlus One

The text was updated successfully, but these errors were encountered:

nleo · 2021-09-15T21:03:55Z

+1 Maybe only Russia affected. It's starts from morning and I surprised where is no reports.

nleo · 2021-09-15T21:10:19Z

There is election soon and there are rumors about full Google services shutdown by goverment.

I checked:

YT opens by browser as usual.
NewPipe loads trends with VPN enabled.

JCU0 · 2021-09-15T22:05:45Z

I thought it was only for me... 3 different devices, same error. Russia. Youtube site is ok, youtube-dl is working fine. The only problem is NewPipe app isn't loading anything.
Getting this exception after a long wait for every video and every subscribed channel.

Exception

User Action: requested stream
Request: https://www.youtube.com/watch?v=0jeT-XhD3j8
Content Country: RU
Content Language: ru
App Language: ru
Service: YouTube
Version: 0.21.9
OS: Linux Android 9 - 28

Crash log

`
java.net.SocketTimeoutException: SSL handshake timed out
at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method)
at com.android.org.conscrypt.NativeSsl.doHandshake(NativeSsl.java:383)
at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:231)
at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:320)
at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:284)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:169)
at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:258)
at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
at okhttp3.RealCall.execute(RealCall.java:93)
at org.schabi.newpipe.DownloaderImpl.execute(DownloaderImpl.java:264)
at org.schabi.newpipe.extractor.downloader.Downloader.post(Downloader.java:131)
at org.schabi.newpipe.extractor.downloader.Downloader.post(Downloader.java:114)
at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.areHardcodedClientVersionAndKeyValid(YoutubeParsingHelper.java:342)
at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.getClientVersion(YoutubeParsingHelper.java:429)
at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.prepareDesktopJsonBuilder(YoutubeParsingHelper.java:814)
at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeStreamExtractor.onFetchPage(YoutubeStreamExtractor.java:701)
at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:54)
at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:68)
at org.schabi.newpipe.extractor.stream.StreamInfo.getInfo(StreamInfo.java:64)
at org.schabi.newpipe.util.ExtractorHelper.lambda$getStreamInfo$3(ExtractorHelper.java:116)
at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$YTHJjScxCJNO1LTCqs3IKy35iyY.call(Unknown Source:4)
at io.reactivex.rxjava3.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:43)
at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
at io.reactivex.rxjava3.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)
at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5330)
at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(MaybeConcatArray.java:153)
at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(MaybeConcatArray.java:78)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(FlowableElementAtMaybe.java:66)
at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray.subscribeActual(MaybeConcatArray.java:42)
at io.reactivex.rxjava3.core.Flowable.subscribe(Flowable.java:15753)
at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(FlowableElementAtMaybe.java:36)
at io.reactivex.rxjava3.core.Maybe.subscribe(Maybe.java:5330)
at io.reactivex.rxjava3.internal.operators.maybe.MaybeToSingle.subscribeActual(MaybeToSingle.java:46)
at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(Scheduler.java:614)
at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:65)
at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:56)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:764)

`

vovanz · 2021-09-16T04:05:48Z

Same here, also in Russia.

novmikvis · 2021-09-16T04:56:06Z

Same here, Russia, but it is not consistent
I can reproduce the issue on one device and partly on another

Phone A
× not working using mobile data (Beeline)
× not working using wifi wired connection (Dom.ru)

Phone B
✔ working using mobile data (Yota)
× not working using wifi wired connection (Dom.ru)

When I switch to mobile data (Yota), request channel list or video, it works like it should.
After that if I switch back to wi-fi (Dom.ru) it keeps working for some time and then stops loading anything again. In one case it worked for about 1 hour, in other for about 30 minutes

Official YouTube app works fine on all devices and all connections.
Youtube-dl and YouTube.com loads all videos and channels through wifi connection (Dom.ru) from PC

Here is the log from phone B, when it is not loading channel videos:

Exception

User Action: requested channel
Request: Start loading: https://www.youtube.com/channel/UCPKT_csvP72boVX0XrMtagQ
Content Country: US
Content Language: en
App Language: ru_RU
Service: YouTube
Version: 0.21.9
OS: Linux Android 8.1.0 - 27

Crash log

`
java.net.SocketTimeoutException: SSL handshake timed out
at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method)
at com.android.org.conscrypt.SslWrapper.doHandshake(SslWrapper.java:374)
at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:217)
at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:320)
at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:284)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:169)
at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:258)
at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:127)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:257)
at okhttp3.RealCall.execute(RealCall.java:93)
at org.schabi.newpipe.DownloaderImpl.execute(DownloaderImpl.java:264)
at org.schabi.newpipe.extractor.downloader.Downloader.post(Downloader.java:131)
at org.schabi.newpipe.extractor.downloader.Downloader.post(Downloader.java:114)
at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.areHardcodedClientVersionAndKeyValid(YoutubeParsingHelper.java:342)
at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.getClientVersion(YoutubeParsingHelper.java:429)
at org.schabi.newpipe.extractor.services.youtube.YoutubeParsingHelper.prepareDesktopJsonBuilder(YoutubeParsingHelper.java:814)
at org.schabi.newpipe.extractor.services.youtube.extractors.YoutubeChannelExtractor.onFetchPage(YoutubeChannelExtractor.java:138)
at org.schabi.newpipe.extractor.Extractor.fetchPage(Extractor.java:54)
at org.schabi.newpipe.extractor.channel.ChannelInfo.getInfo(ChannelInfo.java:47)
at org.schabi.newpipe.util.ExtractorHelper.lambda$getChannelInfo$4(ExtractorHelper.java:124)
at org.schabi.newpipe.util.-$$Lambda$ExtractorHelper$BOLWstv98dC8pFAG_uir5gPXYwY.call(Unknown Source:4)
at io.reactivex.rxjava3.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:43)
at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
at io.reactivex.rxjava3.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)
at io.reactivex.rxjava3.core.Single.subscribe(Single.java:4813)
at io.reactivex.rxjava3.internal.operators.maybe.MaybeFromSingle.subscribeActual(MaybeFromSingle.java:41)

SadmL · 2021-09-16T05:37:20Z

+1 Russian
Starts yesterday, but I thought the problem was on my side.

youtube loads
yewtu.be too
NewPipe works via VPN

Most likely this is due to the fact that our government wants to block some DNS - Google DNS, Cloudflare DNS, Cisco OpenDNS.

Android 10
Ressurection Remix ROM
NewPipe 0.21.9

UPD 1: NewPipe works on my working Wi-Fi without VPN
But still not works on mobile data

andgera · 2021-09-16T05:45:38Z

I am also Russia 🇷🇺
❌ Mobile Operator Tele2 - NewPipe does not work 📵
✔️ Wi-fi operator Starcom - NewPipe works

The official youtube client is working!
NewPipe works via VPN

There is an obvious problem with DNS!
✔️ Installed Rethink DNS and everything worked on mobile operator Tele2!

b1owulf · 2021-09-16T06:02:19Z

im from russia.
newpipe does not work through the Megafon and MTS operator but works via VPN.
official youtube client works fine.

ClaudeLib · 2021-09-16T07:00:11Z

Also doesn't work. Xiaomi Mi 9 running Pixel Experience 11, Yota. Interesting how it doesn't affect anything else, including SkyTube or YouTube itself. Guess we just have to wait until the election is over for now.

SadmL · 2021-09-16T07:07:42Z

I am also Russia ru
x Mobile Operator Tele2 - NewPipe does not work no_mobile_phones
heavy_check_mark Wi-fi operator Starcom - NewPipe works

The official youtube client is working!
NewPipe works via VPN

There is an obvious problem with DNS!
heavy_check_mark Installed Rethink DNS and everything worked on mobile operator Tele2!

Works for me but only in DNS and Firewall mode.
DNS without Firewall - nothing changes.

DmitrysMusic · 2021-09-16T08:28:15Z

Another Russian here.
Checked on both devices on Rostelecom (ISP) and Tele2 (carrier) on my Samsung Galaxy A9 2018 running One UI 2/Android 10 and POCO F3 on Android 12 Beta 4, nothing works directly.
BUT, for some reason, it works fine through the BlueStacks emulator (7.1.2 Nougat 64-bit) on a PC with the same ISP at the same home!
But only on desktop natively.

mai-academ · 2021-09-16T10:37:22Z

Same here, Russia, Tele2 mobile operator. Had to route NewPipe through VPN. YT is working fine.

yuliskov · 2021-09-16T11:24:54Z

Test this fix, guys.

        ConnectionSpec cs = new ConnectionSpec.Builder(ConnectionSpec.COMPATIBLE_TLS)
                .tlsVersions(TlsVersion.TLS_1_2)
                .cipherSuites(
                        CipherSuite.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
                        CipherSuite.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
                        CipherSuite.TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
                )
                .build();
        okBuilder.connectionSpecs(Collections.singletonList(cs));
        return okBuilder.build();

ValdikSS · 2021-09-16T12:14:42Z

@yuliskov do they block uncommon cipher list on TSPU?

Could anyone please collect .pcap capture and upload it here or mail me at [email protected] please?
Use https://play.google.com/store/apps/details?id=com.emanuelef.remote_capture for example.

cheloveg · 2021-09-16T12:20:03Z

Also problem in Russia, Rethink DNS helped. Probably related with public DNS blocks in Russia.

cheloveg · 2021-09-16T12:21:47Z

@yuliskov do they block uncommon cipher list on TSPU?

Could anyone please collect .pcap capture and upload it here or mail me at [email protected] please?
Use https://play.google.com/store/apps/details?id=com.emanuelef.remote_capture for example.

PCAPdroid is also available from F-Droid: https://f-droid.org/packages/com.emanuelef.remote_capture

ValdikSS · 2021-09-16T12:24:32Z

Indeed, youtube does not reply to NewPipe's TLS ClientHello. This is presumably due to some kind of filtering on TSPU.
For more in-depth network details look here: https://ntc.party/t/tls-youtube/1311

WaarisAboobaker · 2021-09-16T13:11:29Z

I'm from Mauritius (African country), and I'm having the same issue.

ValdikSS · 2021-09-16T13:31:49Z

Also problem in Russia, Rethink DNS helped. Probably related with public DNS blocks in Russia.

RethinkDNS contains TCP stack, which behaves a bit unusual (it seems to fragment TCP stream by 512 bytes). This presumably bypasses TSPU. Not because of DNS change, but because of the fragmentation.

vovanz · 2021-09-16T13:37:00Z

When I turn on PCAPdroid for NewPipe the issue disappears and everything works just fine. When i turn it off, the issue returns. Very weird.

ValdikSS · 2021-09-16T13:39:58Z

When I turn on PCAPdroid for NewPipe the issue disappears and everything works just fine. When i turn it off, the issue returns. Very weird.

All these software which use VPN API has to reimplement its own TCP stack, including PCAPdroid, which may behave differently to regular application. If you have root, you can use tcpdump to capture unaltered traffic directly from the interface.

Right now it seems like a TLS Fingerprinting to block an Android application which uses www.youtube.com as a domain fronting (wild guess: Navalny Android/iOS app?). The combination of TLS Cipher Suites + Supported Groups + Signature Algorithms + SNI is filtered, and NewPipe has a pretty unique one according to https://tlsfingerprint.io/

SadmL · 2021-09-16T13:42:12Z

When I turn on PCAPdroid for NewPipe the issue disappears and everything works just fine. When i turn it off, the issue returns. Very weird.

Loads 6 subs per some time, but subs list is not updating in the end for me

ValdikSS · 2021-09-16T14:57:33Z

The exact combination of TLS Cipher Suites + Supported Groups + Signature Algorithms + SNI is being filtered. A simple cipher reordering should help, but unfortunately changing only this line:

NewPipe/app/src/main/java/org/schabi/newpipe/DownloaderImpl.java

Lines 119 to 120 in 793b88a

    
           cipherSuites.add(CipherSuite.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA); 
        
           cipherSuites.add(CipherSuite.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA);

does not work.

ValdikSS · 2021-09-16T15:25:36Z

Try this. This is for test only. Works for me.
NewPipe_v0.21.9-aligned-debugSigned.zip

This version has a modification of APPROVED_CIPHER_SUITES of okhttp3 library, right inside the library (ConnectionSpec.smali file). I've replaced TLS_RSA_WITH_AES_256_CBC_SHA with TLS_RSA_WITH_AES_128_CBC_SHA.

DmitrysMusic · 2021-09-16T16:07:09Z

Good job. Waiting for it to be integrated onto the next hotfix update.

tsilvs · 2021-09-16T16:16:14Z

@ValdikSS am I right to understand that it only works with worsened encryption, which is easier to hack for any MITM (ISPs, governements, etc)?

ValdikSS · 2021-09-16T16:20:08Z

@ValdikSS am I right to understand that it only works with worsened encryption, which is easier to hack for any MITM (ISPs, governements, etc)?

No, the ciphers are still strong ones, I've just made a minor alternation to break the fingerprint chain.

It seems to me that TSPU is blocking traffic from okhttp library (with its standard cryptography cipher suites to be precise) to www.youtube.com domain. Why exactly — no idea. My guess is that they are trying to block some application which uses youtube and okhttp library, but I don't know for sure.

JCU0 · 2021-09-16T16:20:28Z

Android said that app is not installed when I try to install the fix. APK is broken, maybe?

ValdikSS · 2021-09-16T16:21:35Z

Android said that app is not installed when I try to install the fix. APK is broken, maybe?

You need to remove the previous one, since my .app has different signature.

SadmL · 2021-09-16T16:23:45Z

Android said that app is not installed when I try to install the fix. APK is broken, maybe?

Export ur data -> Uninstall NewPipe -> Install debug version -> Import ur data from backup

ClaudeLib · 2021-09-16T16:35:11Z

So for once it's actually not RosComPozor's fault?

ValdikSS · 2021-09-16T16:39:19Z

So for once it's actually not RosComPozor's fault?

It's certainly is, this is a block somewhere inside Russia, on TSPU it seems. Not Google/YouTube or NewPipe fault.

To make sure, I've sent the exact packet which NewPipe sends to YouTube, to my server (to my IP address). As I can see on the server, the packet gets delivered but the server response is not accepted by the client (Tele2 Saint Petersburg network): https://ntc.party/t/tls-youtube/1311/7

More info: https://roskomsvoboda.org/cards/card/tspu-blokrovki-runet/

ClaudeLib · 2021-09-16T16:40:08Z

Well, dang.

litetex · 2021-09-16T17:58:43Z

Wow this issue escalated quickly 😆

@ValdikSS

The exact combination of TLS Cipher Suites + Supported Groups + Signature Algorithms + SNI is being filtered. A simple cipher reordering should help, but unfortunately changing only this line:

NewPipe/app/src/main/java/org/schabi/newpipe/DownloaderImpl.java

Lines 119 to 120 in 793b88a

cipherSuites.add(CipherSuite.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA);

cipherSuites.add(CipherSuite.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA);

does not work.

That's because this is only enabled on KitKat devices 😉

NewPipe/app/src/main/java/org/schabi/newpipe/DownloaderImpl.java

Lines 57 to 58 in 793b88a

    
           if (Build.VERSION.SDK_INT == Build.VERSION_CODES.KITKAT) { 
        
               enableModernTLS(builder);

This issue might also be related to #2898

ValdikSS · 2021-09-16T18:18:22Z

This issue might also be related to #2898

It's not, I've tested TLS combinations as well. This is not due to TLS 1.3, not due to uncommon/unsupported ciphers, etc.
I'm pretty sure it would magically work as usual on 20 September ;)

litetex · 2021-09-16T18:28:01Z

Also note that the used cipher TLS_RSA_WITH_AES_128_CBC_SHA is classified as weak:
https://ciphersuite.info/cs/TLS_RSA_WITH_AES_128_CBC_SHA/

azat11 · 2021-09-16T21:48:44Z

Try this. This is for test only. Works for me.
NewPipe_v0.21.9-aligned-debugSigned.zip

This version has a modification of APPROVED_CIPHER_SUITES of okhttp3 library, right inside the library (ConnectionSpec.smali file). I've replaced TLS_RSA_WITH_AES_256_CBC_SHA with TLS_RSA_WITH_AES_128_CBC_SHA.

Works for me.

Abdel-ilah · 2021-09-16T22:33:53Z

I am also Russia 🇷🇺
❌ Mobile Operator Tele2 - NewPipe does not work 📵
✔️ Wi-fi operator Starcom - NewPipe works

The official youtube client is working!
NewPipe works via VPN

There is an obvious problem with DNS!
✔️ Installed Rethink DNS and everything worked on mobile operator Tele2!

Any good news bro? it work now?

SadmL · 2021-09-17T16:37:01Z

Bad news for me
Rethink DNS is not working anymore with my home Wi-Fi
But it works on mobile data

ValdikSS · 2021-09-17T17:04:24Z

If you don't want to install my build, you can try Intra: https://getintra.org/#!/ Despite the website information only regarding DNS, it also performs TCP-level fragmentation.

…

On 17.09.2021 19:37, Artyom wrote: Bad news for me Rethink DNS is not working anymore with my home Wi-Fi But it works on mobile data — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7114 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXJZCKJTYTA4K6YJ62DD73UCNVDVANCNFSM5EDJIUZA>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

SadmL · 2021-09-17T17:25:17Z

Just wanted to warn others who might have RethinkDNS stop working that they are not alone :)

If you don't want to install my build, you can try Intra:
https://getintra.org/#!/

Im periodically swaps between official and your build.
Pretty annoying that they don't have direct apk download, coz im not a Play Market user at all. Only way how I can download it - from 4pda.
Im ok using mobile data btw, but ty for build.

UPD 1: Same situation with Intra

UPD 2: Wi-Fi works with Tor only

UPD 3: Windscribe VPN works

DmitrysMusic · 2021-09-18T08:13:08Z

Pretty annoying that they don't have direct apk download, coz im not a Play Market user at all.

The original download link to this is this GitHub repository right here.

BTW, somehow I'm able to play videos again just fine on Beeline ISP at other home on a stock build without using DNS things.

ClaudeLib · 2021-09-18T08:18:14Z

The original download link to this is this GitHub repository right here.

Pretty sure he meant Intra which, despite claiming to be open source, is only available via Google Play.

DmitrysMusic · 2021-09-18T08:22:13Z

Pretty sure he meant Intra which, despite claiming to be open source, is only available via Google Play.

Huh, weird.
I did checked their GitHub repository, no releases there indeed.

SadmL · 2021-09-18T09:05:31Z

The original download link to this is this GitHub repository right here.

Pretty sure he meant Intra which, despite claiming to be open source, is only available via Google Play.

Yep, I talked about Intra

SadmL · 2021-09-18T11:23:49Z

Well, looks like NewPipe works now without VPN/Tor (at least for me)

UPD 1: It's broken again (Wi-Fi)

DmitrysMusic · 2021-09-18T18:50:30Z

It slowly becomes available.
Works for me now on stock NewPipe 0.21.9 client on Tele2 mobile carrier connection.
Still going strong on Beeline ISP since about 2 PM GMT/UTC+4.

ValdikSS · 2021-09-18T18:52:44Z

Can confirm, works now.

DmitrysMusic · 2021-09-18T18:53:57Z

I think we can safely close out the issue, as it was in a typical Russian fashion.
Google Documents have also become accessible for me as well.

litetex · 2021-09-18T19:25:09Z

I renamed the issue so that it's clearly visible what this issue is about.

It slowly becomes available.
...
Can confirm, works now.

Nobody says what's working now... I assume NewPipe (and not a VPN as mentioned above)?

Does this mean YT works now again in Russia on "standard" NewPipe?
If yes I'll close the issue 😄

ValdikSS · 2021-09-18T19:30:33Z

@litetex, YouTube in general worked fine all the time, but not NewPipe. As the investigation shows, a government-wide Deep Packet Inspection system for some reason had blocked TLS fingerprint of okhttp library, which is used by NewPipe, when accessing www.youtube.com — NewPipe HTTPS requests, as well as other okhttp software accessing YouTube, left unreplied.
Browser access and official YouTube applications worked fine.

So the original issue title described the issue quite clearly. I can't even guess what that was, why this exact TLS fingerprint become blocked towards www.youtube.com domain.
This is the first time TLS fingerprinting is used in Russia.

SadmL · 2021-09-18T19:52:00Z

Today NewPipe and the Internet worked strangely for me:
About 2 hours ago, NewPipe worked ok, after which it worked unstable during these 2 hours after, and the Internet speed dropped to 100kb/s (should be ~100 mb/s), but with ping it was 1-20ms (wired internet). And NewPipe still works from time to time on Wi-Fi, mobile data is ok.
On PC I tried to connect via Tor, the ping was ~200ms, but the speed was stable - about 20-30 mb/s.
This is some kind of nonsense.

Mb rename issue to "NewPipe not working in Russia"?

litetex · 2021-09-21T20:10:53Z

No more reports since 3 days and the "elections" in Russia are over so I guess we can close the issue.

Feel free to leave a comment if you disagree 😄

ignoramous · 2021-11-08T17:14:46Z

This is a fascinating discussion. Glad I stumbled upon it (:

All these software which use VPN API has to reimplement its own TCP stack, including PCAPdroid, which may behave differently to regular application. If you have root, you can use tcpdump to capture unaltered traffic directly from the interface.

Just a note: A userspace TCP/IP is required only if the apps have to maintain any sort of 'state' for outgoing tcp connections (for ex, splitting tls client-hello) and NAT (and rewrite) udp packets. That's a very specific requirement. For example, NetGuard doesn't embed a TCP/IP stack (but does maintain some form of state for its packet / conn filtering implementation). A stateless firewall wouldn't need to run a userspace TCP/IP stack (which would limit its functionality) and can rely on the underlying Kernel's implementation instead (there are apps that do this, too).

(rethinkdns developer here)

wkrp mentioned this issue Sep 16, 2021

Filtering of TLS with specific features to www.youtube.com in Russia net4people/bbs#85

Open

litetex added the youtube Service, https://www.youtube.com/ label Sep 18, 2021

litetex changed the title ~~NewPipe can't load anything~~ YouTube not working in Russia Sep 18, 2021

opusforlife2 changed the title ~~YouTube not working in Russia~~ Newpipe (Youtube) not working in Russia (but official YT app works) Sep 19, 2021

litetex closed this as completed Sep 21, 2021

Newpipe (Youtube) not working in Russia (but official YT app works) #7114

Newpipe (Youtube) not working in Russia (but official YT app works) #7114

Comments

azat11 commented Sep 15, 2021 • edited Loading

Checklist

Steps to reproduce the bug

Actual behavior

Expected behavior

Screenshots/Screen recordings

Logs

Exception

Device info

nleo commented Sep 15, 2021

nleo commented Sep 15, 2021

JCU0 commented Sep 15, 2021

Exception

vovanz commented Sep 16, 2021

novmikvis commented Sep 16, 2021 • edited Loading

Exception

SadmL commented Sep 16, 2021 • edited Loading

andgera commented Sep 16, 2021 • edited Loading

b1owulf commented Sep 16, 2021

ClaudeLib commented Sep 16, 2021 • edited Loading

SadmL commented Sep 16, 2021

DmitrysMusic commented Sep 16, 2021

mai-academ commented Sep 16, 2021 • edited Loading

yuliskov commented Sep 16, 2021 • edited Loading

ValdikSS commented Sep 16, 2021

cheloveg commented Sep 16, 2021

cheloveg commented Sep 16, 2021

ValdikSS commented Sep 16, 2021

WaarisAboobaker commented Sep 16, 2021

ValdikSS commented Sep 16, 2021

vovanz commented Sep 16, 2021

ValdikSS commented Sep 16, 2021 • edited Loading

SadmL commented Sep 16, 2021 • edited Loading

ValdikSS commented Sep 16, 2021 • edited Loading

ValdikSS commented Sep 16, 2021 • edited Loading

DmitrysMusic commented Sep 16, 2021

tsilvs commented Sep 16, 2021

ValdikSS commented Sep 16, 2021

JCU0 commented Sep 16, 2021

ValdikSS commented Sep 16, 2021

SadmL commented Sep 16, 2021 • edited Loading

ClaudeLib commented Sep 16, 2021

ValdikSS commented Sep 16, 2021

ClaudeLib commented Sep 16, 2021

litetex commented Sep 16, 2021

ValdikSS commented Sep 16, 2021

litetex commented Sep 16, 2021

azat11 commented Sep 16, 2021

Abdel-ilah commented Sep 16, 2021

SadmL commented Sep 17, 2021

ValdikSS commented Sep 17, 2021 via email

SadmL commented Sep 17, 2021 • edited Loading

DmitrysMusic commented Sep 18, 2021

ClaudeLib commented Sep 18, 2021 • edited Loading

DmitrysMusic commented Sep 18, 2021

SadmL commented Sep 18, 2021 • edited Loading

SadmL commented Sep 18, 2021 • edited Loading

DmitrysMusic commented Sep 18, 2021

ValdikSS commented Sep 18, 2021

DmitrysMusic commented Sep 18, 2021

litetex commented Sep 18, 2021

ValdikSS commented Sep 18, 2021 • edited Loading

SadmL commented Sep 18, 2021 • edited Loading

litetex commented Sep 21, 2021

ignoramous commented Nov 8, 2021

azat11 commented Sep 15, 2021 •

edited

Loading

novmikvis commented Sep 16, 2021 •

edited

Loading

SadmL commented Sep 16, 2021 •

edited

Loading

andgera commented Sep 16, 2021 •

edited

Loading

ClaudeLib commented Sep 16, 2021 •

edited

Loading

mai-academ commented Sep 16, 2021 •

edited

Loading

yuliskov commented Sep 16, 2021 •

edited

Loading

ValdikSS commented Sep 16, 2021 •

edited

Loading

SadmL commented Sep 16, 2021 •

edited

Loading

ValdikSS commented Sep 16, 2021 •

edited

Loading

ValdikSS commented Sep 16, 2021 •

edited

Loading

SadmL commented Sep 16, 2021 •

edited

Loading

SadmL commented Sep 17, 2021 •

edited

Loading

ClaudeLib commented Sep 18, 2021 •

edited

Loading

SadmL commented Sep 18, 2021 •

edited

Loading

SadmL commented Sep 18, 2021 •

edited

Loading

ValdikSS commented Sep 18, 2021 •

edited

Loading

SadmL commented Sep 18, 2021 •

edited

Loading