Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[master] Outgoing video stream is no longer working on iOS 13 and Xcode 11 beta #375

Closed
EternallLight opened this issue Aug 30, 2019 · 86 comments
Assignees
Milestone

Comments

@EternallLight
Copy link

Hello,

I've just installed iOS 13.1 Beta on my iPhone and noticed that outgoing video is no longer working. After upgrading plugin to the latest version (previously used version is 4.0.2), the issue is still remaining. This does not seem to be a renderer issue, because I can see incoming video streams; the issues reproduces only for outgoing streams. A black square appears in place where should be my outgoing video view, and the other side cannot see my video, either.

Here is what's going on in logs when I enable the camera for an ongoing call:

2019-08-30 18:26:00.825844+0300 My App Name[1171:317447] iosrtcPlugin#getUserMedia()
2019-08-30 18:26:00.825981+0300 My App Name[1171:317447] PluginGetUserMedia#call()
2019-08-30 18:26:00.829510+0300 My App Name[1171:317447] PluginGetUserMedia#call() | video authorization: authorized
2019-08-30 18:26:00.830827+0300 My App Name[1171:317447] PluginGetUserMedia#call() | video requested (specified device id: 'com.apple.avfoundation.avcapturedevice.built-in_video:1')
2019-08-30 18:26:00.831406+0300 My App Name[1171:317447] PluginGetUserMedia#call() | chosen video device: <AVCaptureFigVideoDevice: 0x147e41bb0 [Front Camera][com.apple.avfoundation.avcapturedevice.built-in_video:1]>
2019-08-30 18:26:00.844505+0300 My App Name[1171:317447] PluginMediaStream#init()
2019-08-30 18:26:00.844641+0300 My App Name[1171:317447] PluginMediaStreamTrack#init()
2019-08-30 18:26:00.844842+0300 My App Name[1171:317447] PluginMediaStreamTrack#run() [kind:video, id:70DD33E1-6235-4D66-B68C-DFD29992BD41]
2019-08-30 18:26:00.844899+0300 My App Name[1171:317447] PluginMediaStream#run()
2019-08-30 18:26:00.845538+0300 My App Name[1171:317447] THREAD WARNING: ['iosrtcPlugin'] took '19.727295' ms. Plugin should use a background thread.
2019-08-30 18:26:00.852460+0300 My App Name[1171:317447] iosrtcPlugin#MediaStreamTrack_setListener()
2019-08-30 18:26:00.852668+0300 My App Name[1171:317467] PluginMediaStreamTrack#setListener() [kind:video, id:70DD33E1-6235-4D66-B68C-DFD29992BD41]
2019-08-30 18:26:00.852905+0300 My App Name[1171:317447] iosrtcPlugin#MediaStream_setListener()
2019-08-30 18:26:00.853019+0300 My App Name[1171:317467] PluginMediaStream#setListener()
2019-08-30 18:26:00.853221+0300 My App Name[1171:317447] iosrtcPlugin#RTCPeerConnection_addStream()
2019-08-30 18:26:00.853409+0300 My App Name[1171:317467] PluginRTCPeerConnection#addStream()
2019-08-30 18:26:00.854124+0300 My App Name[1171:317447] [TCCallKit] onVideoAdded
2019-08-30 18:26:00.855400+0300 My App Name[1171:317497] PluginRTCPeerConnection | onnegotiationeeded
2019-08-30 18:26:00.973638+0300 My App Name[1171:317447] THREAD WARNING: ['NativePageTransitions'] took '114.470947' ms. Plugin should use a background thread.
2019-08-30 18:26:00.975157+0300 My App Name[1171:317447] iosrtcPlugin#RTCPeerConnection_createOffer()
2019-08-30 18:26:00.975654+0300 My App Name[1171:317476] PluginRTCPeerConnection#createOffer()
2019-08-30 18:26:00.976804+0300 My App Name[1171:317476] PluginRTCPeerConnectionConstraints#init()
2019-08-30 18:26:00.976881+0300 My App Name[1171:317476] PluginRTCPeerConnectionConstraints#getConstraints()
2019-08-30 18:26:00.977921+0300 My App Name[1171:317476] PluginRTCPeerConnectionConstraints#deinit()
2019-08-30 18:26:00.978293+0300 My App Name[1171:317497] PluginRTCPeerConnection#createOffer() | success callback
2019-08-30 18:26:00.990259+0300 My App Name[1171:317447] iosrtcPlugin#RTCPeerConnection_setLocalDescription()
2019-08-30 18:26:00.990459+0300 My App Name[1171:317476] PluginRTCPeerConnection#setLocalDescription()
2019-08-30 18:26:00.995002+0300 My App Name[1171:317497] PluginRTCPeerConnection | onsignalingstatechange [signalingState:Optional("have-local-offer")]
2019-08-30 18:26:01.002659+0300 My App Name[1171:317497] PluginRTCPeerConnection#setLocalDescription() | success callback
2019-08-30 18:26:01.066928+0300 My App Name[1171:317447] iosrtcPlugin#new_MediaStreamRenderer()
2019-08-30 18:26:01.067339+0300 My App Name[1171:317447] PluginMediaStreamRenderer#init()
2019-08-30 18:26:01.083873+0300 My App Name[1171:317447] Metal GPU Frame Capture Enabled
2019-08-30 18:26:01.084272+0300 My App Name[1171:317447] Metal API Validation Enabled
2019-08-30 18:26:01.162637+0300 My App Name[1171:317447] PluginMediaStreamRenderer#run()
2019-08-30 18:26:01.162725+0300 My App Name[1171:317447] THREAD WARNING: ['iosrtcPlugin'] took '95.817871' ms. Plugin should use a background thread.
2019-08-30 18:26:01.163036+0300 My App Name[1171:317447] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-08-30 18:26:01.163578+0300 My App Name[1171:317447] PluginMediaStreamRenderer#refresh() [elementLeft:185.0, elementTop:978.0, elementWidth:125.0, elementHeight:85.0, videoViewWidth:125.0, videoViewHeight:85.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:3.0]
2019-08-30 18:26:01.163920+0300 My App Name[1171:317447] iosrtcPlugin#MediaStreamRenderer_render()
2019-08-30 18:26:01.163991+0300 My App Name[1171:317447] PluginMediaStreamRenderer#render()
2019-08-30 18:26:01.193652+0300 My App Name[1171:317447] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-08-30 18:26:01.193826+0300 My App Name[1171:317447] PluginMediaStreamRenderer#refresh() [elementLeft:185.0, elementTop:978.0, elementWidth:125.0, elementHeight:85.0, videoViewWidth:125.0, videoViewHeight:85.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:3.0]
2019-08-30 18:26:01.286278+0300 My App Name[1171:317447] iosrtcPlugin#RTCPeerConnection_setRemoteDescription()
2019-08-30 18:26:01.286527+0300 My App Name[1171:317476] PluginRTCPeerConnection#setRemoteDescription()
2019-08-30 18:26:01.298347+0300 My App Name[1171:317497] PluginRTCPeerConnection | onsignalingstatechange [signalingState:Optional("stable")]
2019-08-30 18:26:01.303718+0300 My App Name[1171:317497] PluginRTCPeerConnection#setRemoteDescription() | success callback
2019-08-30 18:26:01.317835+0300 My App Name[1171:317447] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-08-30 18:26:01.318004+0300 My App Name[1171:317447] PluginMediaStreamRenderer#refresh() [elementLeft:185.0, elementTop:410.0, elementWidth:125.0, elementHeight:85.0, videoViewWidth:125.0, videoViewHeight:85.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:3.0]
2019-08-30 18:26:01.319030+0300 My App Name[1171:317447] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-08-30 18:26:01.319208+0300 My App Name[1171:317447] PluginMediaStreamRenderer#refresh() [elementLeft:185.0, elementTop:410.0, elementWidth:125.0, elementHeight:85.0, videoViewWidth:125.0, videoViewHeight:85.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:3.0]

No obvious error messages, but no video, either.
Could you please take a look?

Thanks,
Andrey.

@EternallLight
Copy link
Author

I just noticed that 13.1 beta was added just yesterday, so I'm going to rollback to 13.0 and check if it works there.

@EternallLight
Copy link
Author

Unfortunately, on iOS 13.0 the problem is remaining :(

@hthetiot hthetiot self-assigned this Sep 2, 2019
@hthetiot hthetiot added the bug label Sep 2, 2019
@hthetiot
Copy link
Contributor

hthetiot commented Sep 2, 2019

@EternallLight Some other user have not the same issue using iOS 13

Would you mind trying to use this simple test script (should with with a basic cordova init app)

I will upgrade to iOS 13 in coming days and see if i can reproduce.

@hthetiot hthetiot added this to the 5.0.x milestone Sep 2, 2019
@EternallLight
Copy link
Author

Hi @hthetiot,

I have tried the test script and had no positive results – both video frames are rendered as black squares.

@hthetiot
Copy link
Contributor

hthetiot commented Sep 2, 2019

@EternallLight Thank you, I will upgrade one of my device to iOS 13.
Unfortunately we would need to know why it does that, any clue is welcome.
cc @banditdevelopment

@hthetiot
Copy link
Contributor

hthetiot commented Sep 2, 2019

@EternallLight Are you using Xcode 11 ? Please provide exact version (currently I'm using XCode Version 10.3 - 10G8)

@EternallLight
Copy link
Author

@hthetiot, yes, it's Version 11.0 beta 6 (11M392r)

@hthetiot
Copy link
Contributor

hthetiot commented Sep 2, 2019

@hthetiot
Copy link
Contributor

hthetiot commented Sep 2, 2019

@EternallLight
Copy link
Author

EternallLight commented Sep 2, 2019

@hthetiot, wow, that must be fresh. I installed beta 6 on Thursday and it was the latest one. Got to upgrade, thanks.
Seems like 11M392r is the latest build at the moment and they just forgot to change the beta version number in the about screen.

@hthetiot
Copy link
Contributor

hthetiot commented Sep 2, 2019

Thx @EternallLight will keep you posted on my discovery on my side.

@hthetiot
Copy link
Contributor

hthetiot commented Sep 2, 2019

Added Xcode 11 on travis and task/ios-13 branch to start debugging.

@hthetiot
Copy link
Contributor

hthetiot commented Sep 4, 2019

@EternallLight Can you see if you have proper hooks triggered on Xcode 11+
See Error in comment #376 (comment)

You should see something like that if it's triggered.

Screen Shot 2019-09-04 at 10 31 46 PM

Make sure to remove and add platform to trigger the hook.

cordova platform remove ios --no-save
cordova platform add ios --no-save

@hthetiot
Copy link
Contributor

hthetiot commented Sep 4, 2019

I dont think cordova 7 or 8 support Xcode 11 I dont even think cordova 9 does support Xcode 11.
@EternallLight Can you tell me the cordova version you use ?

See WIP #376 (comment)

@hthetiot hthetiot changed the title Outgoing video stream is no longer working on iOS 13 Outgoing video stream is no longer working on iOS 13 and Sep 4, 2019
@hthetiot hthetiot changed the title Outgoing video stream is no longer working on iOS 13 and Outgoing video stream is no longer working on iOS 13 and Xcode 11 beta Sep 4, 2019
@hthetiot hthetiot added the build label Sep 4, 2019
@EternallLight
Copy link
Author

@hthetiot
I've just created a brand new cordova/ionic project to test this from scratch. It's running on Cordova 9.0.0 ([email protected]) and cordova-ios 5.0.1 which seem to be the latest. I also installed the latest version of cordova-plugin-iortc from Github that includes your commits made 14 hours ago. ionic cordova run ios ends with ** ARCHIVE SUCCEEDED ** and ** EXPORT SUCCEEDED ** and freezes on native-run ios --app platforms/ios/build/device/MyApp.ipa --device. There's no "Build settings from the configuration file..." in the log, only this:

Build settings from command line:
    CONFIGURATION_BUILD_DIR = /Users/agrunev/Thirdlane/iosrtc-ios13/platforms/ios/build/device
    SHARED_PRECOMPS_DIR = /Users/agrunev/Thirdlane/iosrtc-ios13/platforms/ios/build/sharedpch

After pushing the app to the device, I run your test. It asked Camera and Microphone permission as it is supposed to but still a black square was moving across the screen. Cannot read property 'toLowerCase' of undefined did not show up anywhere, I think I would have noticed it in the first place.
If you want something more from my side, let's arrange a call with screen sharing, so I could make the tests in real time.
Thanks you very much for helping with this!

@EternallLight
Copy link
Author

By the way, not sure if it will be helpful, but there's another plugin in my project that was broken by breaking API changes in iOS 13: j3k0/cordova-plugin-openwith#67
What if there's some changes in the API that is used for camera stream capturing as well?

@hthetiot
Copy link
Contributor

I still think its Xcode 11 issue, would you mind trying Xcode 10+ and run on ios 13.
I will leave that issue open and try to look into Cordova 9 and Xcode 11 support.

@EternallLight
Copy link
Author

@hthetiot I couldn't run it from Xcode 10 – it says that no support files for iOS 13 found. I'm gonna check if there's a way to fix this and push app to an iOS 13 device from 10th Xcode.

@EternallLight
Copy link
Author

EternallLight commented Sep 11, 2019

Today, I've installed our production app from the AppStore on two devices with iOS 12.4 and 13.0, and tested – as expected, the outgoing video looks good on iOS 12.4, but there's a black view instead on iOS 13. I'm going to make a test build with the very latest version of the plugin and try it out on both devices now.
UPD: There's a new Xcode beta released yesterday, got to try it out too!

@EternallLight
Copy link
Author

Just rebuilt the app with the yesterday's Xcode which by the way is no longer beta. Unfortunately, no changes

@myhhx
Copy link

myhhx commented Sep 11, 2019

@EternallLight @hthetiot

Sorry, my English is not good.
This problem does not appear when the screen is horizontal.
The reason may be that the underlying library is too old.
My solution is
Use sfsafariviewcontroller on ios13
It also has a cordova plugin
Maybe you will meet
And iosrtc once it is connected to iosrtc will crash
Can be solved by deleting h264 in sdp

不好意思,我的英文不好
这个问题,横屏的时候不会出现
原因可能是底层的那个库太老了
我的解决方方法是
在ios13上用sfsafariviewcontroller
它也有cordova的插件
可能你们会遇到
和iosrtc一旦建立连接iosrtc会崩溃的问题
可通过在sdp中删除h264解决

@EternallLight
Copy link
Author

@myhhx

This problem does not appear when the screen is horizontal.

I confirm.
Have you managed to fix it on cordova-plugin-iosrtc by switching to SFSafariViewController?

@myhhx
Copy link

myhhx commented Sep 11, 2019

@EternallLight

Using SFSafariViewController is actually using safari's webrtc implementation.
In other words, your js application code can be run on safari without changing it.
At this time, it has nothing to do with cordova-plugin-iosrtc.

用SFSafariViewController实际上用的是safari的webrtc实现
也就是说你的js应用代码不用改就可以在safari上跑
这时已经和cordova-plugin-iosrtc没有关系了

@EternallLight
Copy link
Author

@myhhx

I understand. It sounds like a lot of work. For the current plugin users, fixing the issue within the plugin would be much more acceptable.

@hthetiot hthetiot added this to the 6.0.0 milestone Oct 15, 2019
@hthetiot
Copy link
Contributor

hthetiot commented Oct 15, 2019

@oscarvadillog

@hthetiot are you planning to merge task/libwebrtc-update-m69 into master?

Now ca067c0

@hthetiot hthetiot changed the title [Beta] Outgoing video stream is no longer working on iOS 13 and Xcode 11 beta [master] Outgoing video stream is no longer working on iOS 13 and Xcode 11 beta Oct 15, 2019
@hthetiot
Copy link
Contributor

hthetiot commented Oct 15, 2019

@jesseer9
Copy link

That is fast! Thanks @hthetiot !

My demo is using capacitor (and cocoapods) and having trouble compiling. Trying to figure if I need to bridge headers. Haven't had any luck finding documentation on mix and matching from both.

Jessee

@hthetiot
Copy link
Contributor

hthetiot commented Oct 15, 2019

@jesseer9
See #398 Other Section

Please do not comment on an open or closed issue out of topic problems, you will annoy me and the original reporter and confuse the resolution of the issue, this not the right way to get help. Instead create a new issue and I will help you as I have proven past weeks.

No we have no plan to support capacitor, if capacitor want to support cordova plugin they have to do it the right way, PR are welcome anyway.

See https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/docs/Building.md#building-steps

@jesseer9
Copy link

@hthetiot I'd be willing to take a stab, if I can find some documentation :)

Jessee

@hthetiot
Copy link
Contributor

hthetiot commented Oct 15, 2019

Please do not comment on an open or closed issue out of topic problems,

I will only close that issue aka #375 via master with https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/CHANGELOG.md#version-600

if @EternallLight @myhhx @cah-dunn @pethessa confirm it's fix iOS 13 and Xcode 11, I will close and release master (cd1a545) as 6.0.0

I will make an issue Announcement] cordova-plugin-iosrtc Version 6.0.x notes like for 5.0.x (#398) but we need original reporters confirm and testing this

  • Cordova version: 8.1.2
  • cordova-ios: 4.5.5
  • Plugin version: 6.0.0 (master branch)
  • iOS version: 13.1.2
  • Xcode version: 11.1(11A1027)
  • webrtc-adapter: 7.3.0

@cah-kyle-dunn
Copy link
Contributor

@hthetiot I've been testing for about 3 days on iPad Pro, iPad Mini 4, and iPhone 11 Pro. The only difference is that I've been on [email protected] but I can bump that down for a few tests if needed. I might suggest we remove that 500ms interval call to refreshVideos() unless we still think it's needed for some reason. Otherwise, LGTM.

Thanks again for all of the time and effort!

@hthetiot
Copy link
Contributor

I might suggest we remove that 500ms interval call to refreshVideos() unless we still think it's needed for some reason. Otherwise, LGTM.

@cah-dunn Done

@hthetiot
Copy link
Contributor

hthetiot commented Oct 25, 2019

I plan to release 6.0.0 in next 72 hours but I need help to test iOS 13 on real device, I have only an iPhone 6 and iPad Mini 3 both not eligible to iOS 13, I would need confirmation that master is ok on iOS 13 on real device that would help me. If you need VP8 you can try #416 if you need Unified Plan test #407, I plan to merge this 2 on 6.0.0 potential depending the demand of the community.

@EternallLight
Copy link
Author

I would need confirmation that master is ok on iOS 13 on real device that would help me

I've got iPhone 7 which is a little newer :) Will test the master branch once 6.0.0 is released

@DaveLomber
Copy link

Tested the latest master.
Had a call between iOS and Web Chrome.
At both sides we use 'lib-jitsi-meet'

iOS local video works fine, I can see it at both sides iOS & Chrome

But, have an issue with remote video stream from Chrome - it's simply a black video element is displayed

Our test case:

  1. setup audio call between iOS & Chome
  2. Enable video at Chrome side

In Xcode logs I can see the following after enabled a video stream at Chrome side:

2019-10-28 11:15:16.798468+0200 VideoTalkApp[454:47307] iosrtcPlugin#RTCPeerConnection_setRemoteDescription()
2019-10-28 11:15:16.798615+0200 VideoTalkApp[454:47555] PluginRTCPeerConnection#setRemoteDescription()
2019-10-28 11:15:16.800422+0200 VideoTalkApp[454:47524] PluginRTCPeerConnection | onsignalingstatechange [signalingState:Optional("have-remote-offer")]
2019-10-28 11:15:16.802469+0200 VideoTalkApp[454:47307] onsignalingstatechange have-remote-offer
2019-10-28 11:15:16.802549+0200 VideoTalkApp[454:47524] PluginRTCPeerConnection | onaddtrack
2019-10-28 11:15:16.802622+0200 VideoTalkApp[454:47524] PluginMediaStreamTrack#init()
2019-10-28 11:15:16.802734+0200 VideoTalkApp[454:47524] PluginMediaStreamTrack#deinit()
2019-10-28 11:15:16.802802+0200 VideoTalkApp[454:47524] PluginRTCPeerConnection | onaddstream
2019-10-28 11:15:16.802844+0200 VideoTalkApp[454:47524] PluginMediaStream#init()
2019-10-28 11:15:16.802894+0200 VideoTalkApp[454:47524] PluginMediaStreamTrack#init()
2019-10-28 11:15:16.802965+0200 VideoTalkApp[454:47524] PluginMediaStreamTrack#run() [kind:video, id:bec6b68b-ca43-4e72-a61d-8f8181326cd8-3]
2019-10-28 11:15:16.803033+0200 VideoTalkApp[454:47524] PluginMediaStream#run()
2019-10-28 11:15:16.803369+0200 VideoTalkApp[454:47524] PluginRTCPeerConnection#setRemoteDescription() | success callback
2019-10-28 11:15:16.805217+0200 VideoTalkApp[454:47307] iosrtcPlugin#MediaStream_setListener()
2019-10-28 11:15:16.805357+0200 VideoTalkApp[454:47307] iosrtcPlugin#MediaStreamTrack_setListener()
2019-10-28 11:15:16.805395+0200 VideoTalkApp[454:47555] PluginMediaStream#setListener()
2019-10-28 11:15:16.805564+0200 VideoTalkApp[454:47555] PluginMediaStreamTrack#setListener() [kind:video, id:bec6b68b-ca43-4e72-a61d-8f8181326cd8-3]
2019-10-28 11:15:16.805812+0200 VideoTalkApp[454:47307] setRemoteDescriptionOnSuccess
2019-10-28 11:15:16.806350+0200 VideoTalkApp[454:47307] iosrtcPlugin#RTCPeerConnection_createAnswer()
2019-10-28 11:15:16.806456+0200 VideoTalkApp[454:47883] PluginRTCPeerConnection#createAnswer()
2019-10-28 11:15:16.806528+0200 VideoTalkApp[454:47883] PluginRTCPeerConnectionConstraints#init()
2019-10-28 11:15:16.806627+0200 VideoTalkApp[454:47883] PluginRTCPeerConnectionConstraints#init() | [mandatoryConstraints:{
    OfferToReceiveAudio = true;
    OfferToReceiveVideo = true;
}, optionalConstraints:{
}]
2019-10-28 11:15:16.806684+0200 VideoTalkApp[454:47883] PluginRTCPeerConnectionConstraints#getConstraints()
2019-10-28 11:15:16.807071+0200 VideoTalkApp[454:47883] PluginRTCPeerConnectionConstraints#deinit()
2019-10-28 11:15:16.807150+0200 VideoTalkApp[454:47524] PluginRTCPeerConnection#createAnswer() | success callback
2019-10-28 11:15:16.811744+0200 VideoTalkApp[454:47307] iosrtcPlugin#RTCPeerConnection_setLocalDescription()
2019-10-28 11:15:16.811870+0200 VideoTalkApp[454:47883] PluginRTCPeerConnection#setLocalDescription()
2019-10-28 11:15:16.812355+0200 VideoTalkApp[454:47524] PluginRTCPeerConnection | onsignalingstatechange [signalingState:Optional("stable")]
2019-10-28 11:15:16.813066+0200 VideoTalkApp[454:47307] onsignalingstatechange stable
2019-10-28 11:15:16.813491+0200 VideoTalkApp[454:47524] PluginRTCPeerConnection#setLocalDescription() | success callback
2019-10-28 11:15:17.109846+0200 VideoTalkApp[454:47307] TPC[1,p2p:false] remote track added: 2ca9fc0b-977f-4875-9a4d-8b61c6e5cecc-3 video
2019-10-28 11:15:17.112307+0200 VideoTalkApp[454:47307] TPC[1,p2p:false] associated ssrc bf32cb05 567217713
2019-10-28 11:15:17.705576+0200 VideoTalkApp[454:47307] iosrtcPlugin#new_MediaStreamRenderer()
2019-10-28 11:15:17.705667+0200 VideoTalkApp[454:47307] PluginMediaStreamRenderer#init()
2019-10-28 11:15:17.898807+0200 VideoTalkApp[454:47524] PluginRTCDataChannel | utf8 message received
2019-10-28 11:15:17.906376+0200 VideoTalkApp[454:47307] Metal GPU Frame Capture Enabled
2019-10-28 11:15:17.907015+0200 VideoTalkApp[454:47307] Metal API Validation Enabled
2019-10-28 11:15:18.105387+0200 VideoTalkApp[454:47524] PluginRTCDataChannel | utf8 message received
2019-10-28 11:15:18.328860+0200 VideoTalkApp[454:47307] PluginMediaStreamRenderer#run()
2019-10-28 11:15:18.329245+0200 VideoTalkApp[454:47307] THREAD WARNING: ['iosrtcPlugin'] took '623.668213' ms. Plugin should use a background thread.
2019-10-28 11:15:18.330060+0200 VideoTalkApp[454:47307] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-10-28 11:15:18.330437+0200 VideoTalkApp[454:47307] PluginMediaStreamRenderer#refresh() [elementLeft:125.0, elementTop:652.0, elementWidth:100.0, elementHeight:100.0, videoViewWidth:100.0, videoViewHeight:100.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:0.0]
2019-10-28 11:15:18.331062+0200 VideoTalkApp[454:47307] iosrtcPlugin#MediaStreamRenderer_render()
2019-10-28 11:15:18.331322+0200 VideoTalkApp[454:47307] PluginMediaStreamRenderer#render()
2019-10-28 11:15:18.331844+0200 VideoTalkApp[454:47307] (TIME) Render video:	 51128
2019-10-28 11:15:18.332217+0200 VideoTalkApp[454:47307] (TIME) TTFM video:	 47810
2019-10-28 11:15:18.332572+0200 VideoTalkApp[454:47307] iosrtcPlugin#new_MediaStreamRenderer()
2019-10-28 11:15:18.332844+0200 VideoTalkApp[454:47307] PluginMediaStreamRenderer#init()
2019-10-28 11:15:18.338289+0200 VideoTalkApp[454:47307] PluginMediaStreamRenderer#run()
2019-10-28 11:15:18.338547+0200 VideoTalkApp[454:47307] iosrtcPlugin#MediaStreamRenderer_refresh()
2019-10-28 11:15:18.338655+0200 VideoTalkApp[454:47307] PluginMediaStreamRenderer#refresh() [elementLeft:0.0, elementTop:103.0, elementWidth:414.0, elementHeight:537.0, videoViewWidth:414.0, videoViewHeight:537.0, visible:true, opacity:1.0, zIndex:0.0, mirrored:false, clip:true, borderRadius:0.0]
2019-10-28 11:15:18.338864+0200 VideoTalkApp[454:47307] iosrtcPlugin#MediaStreamRenderer_render()
2019-10-28 11:15:18.338911+0200 VideoTalkApp[454:47307] PluginMediaStreamRenderer#render()
2019-10-28 11:15:18.339636+0200 VideoTalkApp[454:47307] iosrtcPlugin#RTCPeerConnection_RTCDataChannel_sendString()
2019-10-28 11:15:18.339701+0200 VideoTalkApp[454:47307] iosrtcPlugin#RTCPeerConnection_getStats()
2019-10-28 11:15:18.339751+0200 VideoTalkApp[454:47307] iosrtcPlugin#enumerateDevices()
2019-10-28 11:15:18.339754+0200 VideoTalkApp[454:47883] PluginRTCPeerConnection#RTCDataChannel_sendString()
2019-10-28 11:15:18.339930+0200 VideoTalkApp[454:47307] Failed to bind EAGLDrawable: <CAEAGLLayer: 0x2832b31a0> to GL_RENDERBUFFER 1
2019-10-28 11:15:18.339960+0200 VideoTalkApp[454:47307] Failed to make complete framebuffer object 8cd6
2019-10-28 11:15:18.340231+0200 VideoTalkApp[454:47307] Failed to bind EAGLDrawable: <CAEAGLLayer: 0x2832bb900> to GL_RENDERBUFFER 1
2019-10-28 11:15:18.340253+0200 VideoTalkApp[454:47307] Failed to make complete framebuffer object 8cd6
2019-10-28 11:15:18.340490+0200 VideoTalkApp[454:47883] PluginRTCDataChannel#sendString()
2019-10-28 11:15:18.341009+0200 VideoTalkApp[454:47883] PluginRTCPeerConnection#getStats()
2019-10-28 11:15:18.341029+0200 VideoTalkApp[454:47307] [LayoutConstraints] Unable to simultaneously satisfy constraints.
	Probably at least one of the constraints in the following list is one you don't want. 
	Try this: 
		(1) look at each constraint and try to figure out which you don't expect; 
		(2) find the code that added the unwanted constraint or constraints and fix it. 
	(Note: If you're seeing NSAutoresizingMaskLayoutConstraints that you don't understand, refer to the documentation for the UIView property translatesAutoresizingMaskIntoConstraints) 
(
    "<NSAutoresizingMaskLayoutConstraint:0x281198a00 h=--& v=--& UIView:0x107ec7cd0.minX == 125   (active, names: '|':WKWebView:0x108012a00'Unread (3)' )>",
    "<NSAutoresizingMaskLayoutConstraint:0x281198a50 h=--& v=--& UIView:0x107ec7cd0.width == 100   (active)>",
    "<NSAutoresizingMaskLayoutConstraint:0x281198cd0 h=-&- v=-&- WKWebView:0x108012a00'Unread (3)'.minX == 0   (active, names: '|':UIView:0x10b202cc0 )>",
    "<NSAutoresizingMaskLayoutConstraint:0x281198d20 h=-&- v=-&- H:[WKWebView:0x108012a00'Unread (3)']-(0)-|   (active, names: '|':UIView:0x10b202cc0 )>",
    "<NSLayoutConstraint:0x281180f50 UIView:0x107ec7cd0.right == UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide'.right   (active)>",
    "<NSLayoutConstraint:0x281198e10 'UIView-Encapsulated-Layout-Width' UIView:0x10b202cc0.width == 414   (active)>",
    "<NSLayoutConstraint:0x281180b40 'UIViewSafeAreaLayoutGuide-right' H:[UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide']-(0)-|(LTR)   (active, names: '|':WKWebView:0x108012a00'Unread (3)' )>"
)

Will attempt to recover by breaking constraint 
<NSLayoutConstraint:0x281180f50 UIView:0x107ec7cd0.right == UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide'.right   (active)>

Make a symbolic breakpoint at UIViewAlertForUnsatisfiableConstraints to catch this in the debugger.
The methods in the UIConstraintBasedLayoutDebugging category on UIView listed in <UIKitCore/UIView.h> may also be helpful.
2019-10-28 11:15:18.341513+0200 VideoTalkApp[454:47307] [LayoutConstraints] Unable to simultaneously satisfy constraints.
	Probably at least one of the constraints in the following list is one you don't want. 
	Try this: 
		(1) look at each constraint and try to figure out which you don't expect; 
		(2) find the code that added the unwanted constraint or constraints and fix it. 
	(Note: If you're seeing NSAutoresizingMaskLayoutConstraints that you don't understand, refer to the documentation for the UIView property translatesAutoresizingMaskIntoConstraints) 
(
    "<NSAutoresizingMaskLayoutConstraint:0x281198aa0 h=--& v=--& UIView:0x107ec7cd0.minY == 652   (active, names: '|':WKWebView:0x108012a00'Unread (3)' )>",
    "<NSAutoresizingMaskLayoutConstraint:0x281198af0 h=--& v=--& UIView:0x107ec7cd0.height == 100   (active)>",
    "<NSAutoresizingMaskLayoutConstraint:0x281198d70 h=-&- v=-&- WKWebView:0x108012a00'Unread (3)'.minY == 44   (active, names: '|':UIView:0x10b202cc0 )>",
    "<NSAutoresizingMaskLayoutConstraint:0x281198dc0 h=-&- v=-&- V:[WKWebView:0x108012a00'Unread (3)']-(0)-|   (active, names: '|':UIView:0x10b202cc0 )>",
    "<NSLayoutConstraint:0x281180d70 UIView:0x107ec7cd0.bottom == UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide'.bottom   (active)>",
    "<NSLayoutConstraint:0x281198e60 'UIView-Encapsulated-Layout-Height' UIView:0x10b202cc0.height == 896   (active)>",
    "<NSLayoutConstraint:0x281180af0 'UIViewSafeAreaLayoutGuide-bottom' V:[UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide']-(34)-|   (active, names: '|':WKWebView:0x108012a00'Unread (3)' )>"
)

Will attempt to recover by breaking constraint 
<NSLayoutConstraint:0x281180d70 UIView:0x107ec7cd0.bottom == UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide'.bottom   (active)>

Make a symbolic breakpoint at UIViewAlertForUnsatisfiableConstraints to catch this in the debugger.
The methods in the UIConstraintBasedLayoutDebugging category on UIView listed in <UIKitCore/UIView.h> may also be helpful.
2019-10-28 11:15:18.341836+0200 VideoTalkApp[454:47307] [LayoutConstraints] Unable to simultaneously satisfy constraints.
	Probably at least one of the constraints in the following list is one you don't want. 
	Try this: 
		(1) look at each constraint and try to figure out which you don't expect; 
		(2) find the code that added the unwanted constraint or constraints and fix it. 
	(Note: If you're seeing NSAutoresizingMaskLayoutConstraints that you don't understand, refer to the documentation for the UIView property translatesAutoresizingMaskIntoConstraints) 
(
    "<NSAutoresizingMaskLayoutConstraint:0x281198aa0 h=--& v=--& UIView:0x107ec7cd0.minY == 652   (active, names: '|':WKWebView:0x108012a00'Unread (3)' )>",
    "<NSLayoutConstraint:0x281180d20 UIView:0x107ec7cd0.top == UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide'.top   (active)>",
    "<NSLayoutConstraint:0x281180a50 'UIViewSafeAreaLayoutGuide-top' V:|-(0)-[UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide']   (active, names: '|':WKWebView:0x108012a00'Unread (3)' )>"
)

Will attempt to recover by breaking constraint 
<NSLayoutConstraint:0x281180d20 UIView:0x107ec7cd0.top == UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide'.top   (active)>

Make a symbolic breakpoint at UIViewAlertForUnsatisfiableConstraints to catch this in the debugger.
The methods in the UIConstraintBasedLayoutDebugging category on UIView listed in <UIKitCore/UIView.h> may also be helpful.
2019-10-28 11:15:18.342009+0200 VideoTalkApp[454:47883] PluginEnumerateDevices#call()
2019-10-28 11:15:18.342162+0200 VideoTalkApp[454:47307] [LayoutConstraints] Unable to simultaneously satisfy constraints.
	Probably at least one of the constraints in the following list is one you don't want. 
	Try this: 
		(1) look at each constraint and try to figure out which you don't expect; 
		(2) find the code that added the unwanted constraint or constraints and fix it. 
	(Note: If you're seeing NSAutoresizingMaskLayoutConstraints that you don't understand, refer to the documentation for the UIView property translatesAutoresizingMaskIntoConstraints) 
(
    "<NSAutoresizingMaskLayoutConstraint:0x281198a00 h=--& v=--& UIView:0x107ec7cd0.minX == 125   (active, names: '|':WKWebView:0x108012a00'Unread (3)' )>",
    "<NSLayoutConstraint:0x281180eb0 UIView:0x107ec7cd0.left == UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide'.left   (active)>",
    "<NSLayoutConstraint:0x281180aa0 'UIViewSafeAreaLayoutGuide-left' H:|-(0)-[UILayoutGuide:0x280bfc000'UIViewSafeAreaLayoutGuide'](LTR)   (active, names: '|':WKWebView:0x108012a00'Unread (3)' )>"
)

I think the issue relates to

 Failed to bind EAGLDrawable: <CAEAGLLayer: 0x2832b31a0> to GL_RENDERBUFFER 1
 Failed to make complete framebuffer object 8cd6

Is there any advice re this ?

@hthetiot
Copy link
Contributor

hthetiot commented Oct 28, 2019

@DaveLomber Try #416 see "Testing" section in the PR description.
I do not reproduce this issue even without #416
Do you know if you force codec ?
What version of Chrome are you using ?
What version of iOS are you using?
Also Chrome do not support video resultion 1080x960 can you try with GetUserMedia with specific resolution (See #419 for fix)

iOS local video works fine, I can see it at both sides iOS & Chrome
VS
But, have an issue with remote video stream from Chrome - it's simply a black video element is displayed

What do you mean by "iOS local video works fine, I can see it at both sides iOS & Chrome" and "have an issue with remote video stream from Chrome " this is contradictory, do you even have a TURN server stup ? Please create dedicated issue, the this is getting out of topic.

@DaveLomber
Copy link

DaveLomber commented Oct 28, 2019

Thanks, I will try the mentioned branch

Do you know if you force codec ?

we do no force it, use default logic

What version of Chrome are you using ?

OS X, 77 ver

Also Chrome do not support video resultion 1080x960 can you try with GetUserMedia with specific resolution (See #419 for fix)

We simply use:

constraints: {
        video: {
          height: {
            ideal: 240,
            min: 180,
            max: 480
          }
        }
      }

I upgraded to the master branch cause we got an issue with iOS 13.
Previously with iOS 12 our current code worked well

@hthetiot
Copy link
Contributor

@EternallLight did you had time to test iOS 13 ?

@EternallLight
Copy link
Author

@hthetiot Yes, I tested with the test script you provided on a clear Ionic app – it worked ok in both portrait and landscape modes.
Unfortunately, I had issues with integrating the latest version of the plugin with our main app. We have multiple app extensions and have to handle custom changes in cordova build scripts – otherwise it'd add the WebRTC framework to the extension target. Once I get it working, I'll let you know if there are other issues remaining.

@hthetiot
Copy link
Contributor

hthetiot commented Oct 28, 2019

@DaveLomber ideal+min+max is not supported yet, see WIP #419

constraints: {
        video: {
          height: {
            ideal: 240,
            min: 180,
            max: 480
          }
        }
      }

Will result in video: {min: 240, max: 240}, 240 do not match any available resolution on iOS.

Try this first:

constraints: {
        video: {
          width: 640,
          height: 480
        }
      }

@DaveLomber
Copy link

@EternallLight I do not have any issues with local iOS stream, it is displayed well, even with these constrains
The issue we face is with remote video stream from Chrome - it's black at iOS side.
Do you think by changing this constraints at Chrome side can help make it displayable at iOS side?

@hthetiot
Copy link
Contributor

hthetiot commented Oct 28, 2019

@DaveLomber Please create dedicated issue. I do not reproduce this issue.
The main reason this issue exist is for "Rendering local stream on iOS 13", you are getting out of topic.

Also you are using MCU/SFU (jitsi) that can be the reason "Real" peer to peer do not have this problem. Also do you use TURN ?

I do not have any issues with local iOS stream, it is displayed well, even with these constrains

Yes on master but currently fail on #419 prior update

@hthetiot
Copy link
Contributor

We have multiple app extensions and have to handle custom changes in cordova build scripts

@EternallLight Can you fill a separate issue, the hook is supose to handle multiple extensions.
See https://github.com/cordova-rtc/cordova-plugin-iosrtc/blob/master/extra/hooks/iosrtc-swift-support.js#L202

@hthetiot hthetiot reopened this Oct 28, 2019
@hthetiot
Copy link
Contributor

I'm closing for now since @EternallLight said it's ok and he is the original reporter.

PLEASE ONLY COMMENT if pure iOS 13 issue and you do Peer To Peer only (No MCU/SFU here see #416 for MCU/SFU janus/jisti and co.)

@DaveLomber
Copy link

@hthetiot I can confirm as well that a local video stream started working well under iOS 13 with the master branch

Re constraints - I think either jitsi-lib or adapter.js converts it into supported format, so hence no issues with it at my side

Re remote video stream is showing as black square - will file a separated issue

Thank you 👍

@EternallLight
Copy link
Author

EternallLight commented Oct 28, 2019

@hthetiot
The initial issue has been solved. Thank you very much for your help!
Regarding the hook, I'll do some additional debugging and file an issue as you suggested

@DaveLomber
Copy link

@hthetiot btw, not sure what's changed
but this MR fixed my issue #416
So I can finally see a remote Chrome video stream at iOS side 🥇

@hthetiot
Copy link
Contributor

@DaveLomber VP8 and VP9 support if you use jisti you do MCU/SFU make sure to mention it when you give feedback. (This PR with VP8 support was made for Janus support, will be merge soon)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests