Rendering is broken when vram is contested

[mishmish66]

System Info

I am using Ubuntu 24.04. I've reproduced this issue on python 3.10, 3.11, and 3.12 and with the current state of the repo as well as the latest version on pypy, and the version from a few commits ago on the main branch.

Information

This problem arises when VRAM is contested and offscreen rendering is used.
I introduced this issue with PR #570

I'll introduce a fix for this momentarily. So sorry to introduce this but hopefully I can get this fix in fast.

Reproduction

Run a large model and robosuite offscreen rendering at the same time

Expected behavior

I would expect robosuite rendering to be agnostic to the amount of VRAM available and in use

[mishmish66]

While this is still an issue I've realized this isn't due to the PR I made (whew) here is a video of what happens.

Warning

Flashing colors
https://github.com/user-attachments/assets/37368a53-4311-4ebe-85c8-05be796fcc1a

I'll work on figuring out what's going on here.

[kevin-thankyou-lin]

Hey @mishmish66, thanks for the PR! Are you able to reproduce this issue on older robosuite versions too?

[mishmish66]

Hey! I confirmed this happens with v1.5.0, but it seems to only happen once the EGL context is reloaded, which I guess people might not do too often?

In case anyone encounters this issue and finds this thread before this has been fixed, a workaround which is to set
renderer="mujoco"
instead of the default which was "mjviewer" in the case I was working with

[abhihjoshi]

Not sure if this would work, but another thing you could try is to set MUJOCO_GL=egl as an env variable prior to running your code (i.e., MUJOCO_GL=egl python foo_bar.py).