Renderer real-time compensation is out of sink

[eugval]

Hello,

I noticed that the render behaves sometimes strangely, with regards to the frequency of rendering and timing.

It is possible to see this by slightly changing the demo.py file by adding sleepand print calls :

Launching the demo then with say Sawyer lift we can see that it only renders every so often, not at every call of render() . Moreover, if we set up a test at a control frequency of 5 hz, and allow for 20 timesteps, we can see that it does not render in 4s which is what would have been expected in real-time.

Upon further investigation, this comes from mujoco-py, that implements a compensation to make the rendering real-time (c.f. lines 194-204 in mjviewer.py):

Now, this compensation relies on sim.nbsubsteps which tells mujoco-py how many simulation steps to take for every control step. This here is set to 1, but the functionality is re-implemented in base.py (note that step in base.py reasons in terms of frequencies and time while sim.nbsubsteps just considers the number of simulation substeps):

The result of this is that every-time that render() is called, mujoco-py compensates time as if only one simulation step was taken between consecutive render calls (as sim.nbsubsteps =1), while there are actually more taken, depending on the control_freq ; which makes the whole thing go out of sink.

I made a quick and dirty hack to compensate for this, but a better ideas are more than welcome:
In base.py:

1. I added self.viewer._render_every_frame = True in _reset_internal to stop the compensation by mujoco-py
2. At every render call I added time.sleep(1/self.control_freq) to get back to real-time.

This does not take into account the actual compute time (as opposed to mujoco-py which does) between render calls so it is actually not showing real-time, but its an approximate fix if you don't care about being very exact.

Thanks!!

[cremebrule]

We've fixed this in the upcoming release!