Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rendering to SurfaceView sometimes does not work in Android #38

Open
mihai1voicescu opened this issue Jan 27, 2022 · 2 comments
Open

Rendering to SurfaceView sometimes does not work in Android #38

mihai1voicescu opened this issue Jan 27, 2022 · 2 comments

Comments

@mihai1voicescu
Copy link
Contributor

My initial scenario was:

  1. initializeWebRtc in Application (not in Activity, so it survives a tilt)
  2. make a call
  3. hang up
  4. call again
  5. now video is black
  6. tilt the screen in any direction, activity gets rebuilt and everything works OK

According to the logs, webGL is rendering frames but it's all black until the first tilt.

Unfortunately I could not get a minimal replication scenario for this. Should be something similar to this branch https://github.com/mihai1voicescu/WebRTCCompose/tree/with-mainApp

I however managed to replicate the bug (or at least it looks like it) in a different scenario using Activity as the context and rebuilding the webrtc connection each tilt. You can find the minimal replication scenario here https://github.com/mihai1voicescu/WebRTCCompose/tree/master

To replicate, as stated in the readme:

  1. Run App
  2. Connection is established
  3. Tilt screen
  4. Screen is now black but it is rendering frames

I assume it's something regarding some leftover metadata in the webGL context but I am kinda stuck 😅 .

Working render logs:

 01-27 17:40:34.224  I  [10501/15589] org.webrtc.Logging CameraStatistics: Camera fps: 25.
 01-27 17:40:34.307  I  [10501/15678] org.webrtc.Logging EglRenderer: Duration: 4002 ms. Frames received: 100. Dropped: 0. Rendered: 100. Render fps: 25.0. Average render time: 7989 us. Average swapBuffer time: 2912 us.
 01-27 17:40:36.225  I  [10501/15589] org.webrtc.Logging CameraStatistics: Camera fps: 25.
 01-27 17:40:38.226  I  [10501/15589] org.webrtc.Logging CameraStatistics: Camera fps: 25.
 01-27 17:40:38.310  I  [10501/15678] org.webrtc.Logging EglRenderer: Duration: 4003 ms. Frames received: 44. Dropped: 0. Rendered: 44. Render fps: 11.0. Average render time: 7018 us. Average swapBuffer time: 3635 us.
 01-27 17:40:40.227  I  [10501/15589] org.webrtc.Logging CameraStatistics: Camera fps: 25.
 01-27 17:40:42.228  I  [10501/15589] org.webrtc.Logging CameraStatistics: Camera fps: 25.
 01-27 17:40:42.313  I  [10501/15678] org.webrtc.Logging EglRenderer: Duration: 4002 ms. Frames received: 72. Dropped: 0. Rendered: 72. Render fps: 18.0. Average render time: 7423 us. Average swapBuffer time: 3183 us.

Black render logs:

 01-27 17:42:13.401  I  [10501/18707] org.webrtc.Logging EglRenderer: Duration: 4004 ms. Frames received: 120. Dropped: 0. Rendered: 120. Render fps: 30.0. Average render time: 6803 us. Average swapBuffer time: 2567 us.
 01-27 17:42:13.691  I  [10501/18694] org.webrtc.Logging CameraStatistics: Camera fps: 30.
 01-27 17:42:15.691  I  [10501/18694] org.webrtc.Logging CameraStatistics: Camera fps: 30.
 01-27 17:42:17.405  I  [10501/18707] org.webrtc.Logging EglRenderer: Duration: 4002 ms. Frames received: 120. Dropped: 0. Rendered: 120. Render fps: 30.0. Average render time: 4767 us. Average swapBuffer time: 1798 us.
 01-27 17:42:17.692  I  [10501/18694] org.webrtc.Logging CameraStatistics: Camera fps: 30.
 01-27 17:42:18.159  I  [10501/15594] org.webrtc.Logging NetworkMonitorAutoDetect: capabilities changed: [ Transports: WIFI Capabilities: NOT_METERED&INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN&VALIDATED&NOT_ROAMING&FOREGROUND&NOT_CONGESTED&NOT_SUSPENDED LinkUpBandwidth>=430185Kbps LinkDnBandwidth>=677812Kbps SignalStrength: -64 AdministratorUids: [] RequestorUid: -1 RequestorPackageName: null]
 01-27 17:42:19.693  I  [10501/18694] org.webrtc.Logging CameraStatistics: Camera fps: 30.
 01-27 17:42:21.410  I  [10501/18707] org.webrtc.Logging EglRenderer: Duration: 4004 ms. Frames received: 120. Dropped: 0. Rendered: 120. Render fps: 30.0. Average render time: 4716 us. Average swapBuffer time: 1845 us.
 01-27 17:42:21.694  I  [10501/18694] org.webrtc.Logging CameraStatistics: Camera fps: 30.
 01-27 17:42:23.694  I  [10501/18694] org.webrtc.Logging CameraStatistics: Camera fps: 30.
 01-27 17:42:25.420  I  [10501/18707] org.webrtc.Logging EglRenderer: Duration: 4006 ms. Frames received: 120. Dropped: 0. Rendered: 120. Render fps: 30.0. Average render time: 5039 us. Average swapBuffer time: 1979 us.
 01-27 17:42:25.695  I  [10501/18694] org.webrtc.Logging CameraStatistics: Camera fps: 30.
 01-27 17:42:27.698  I  [10501/18694] org.webrtc.Logging CameraStatistics: Camera fps: 29.

Any ideas or pointers?

Note that I am using Jetpack Compose which kills the Surfaces after they exit the tree.
@shepeliev
Copy link
Owner

I've just checked out your sample. Unfortunately, I couldn't reproduce the issue with my environment. However, I can see the potential problem: the local MediaStream is not released after the call ends in your code. It's required to call MediaStream.release() to stop video capturing and release the camera. I guess it might be the cause of the issue.

@mihai1voicescu
Copy link
Contributor Author

The bug seems to not replicate on Android 9. It replicates on Android 11.

For the moment I can't afford to look into it anymore, just made a workaround, but as soon as time allows it I will look more into it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@shepeliev @mihai1voicescu