Incorrect order in foreground objects (wrong object visibility)

[jmp75]

I am far from proficient with 3D viz and not sure this is a mayavi issue as such (apologies if I bark up the wrong tree), but I observe the following, unexpected behavior:

Environment:

conda, anaconda 3, installed mayavi using pip install mayavi vtk pyqt5 rather than conda (after bumping into a similar issue to #652)

Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux
Name: mayavi
Version: 4.6.0
Name: vtk
Version: 8.1.0
Location: /home/xxxyyy/anaconda3/envs/ELA/lib/python3.6/site-packages

Repro

# example from https://stackoverflow.com/a/10740780
from mayavi import mlab
import numpy as np

# Generate some random hypocenters
x, y, z, mag = np.random.random((4, 500))

# Make a curved well bore...
wellx, welly, wellz = 3 * [np.linspace(0, 1.5, 10)]
wellz =  wellz**2

# Plot the hypocenters, colored and scaled by magnitude
mlab.points3d(x, y, z, mag)

# Plot the wellbore
mlab.plot3d(wellx, welly, wellz, tube_radius=0.1)

mlab.show()

The colored balls should surround the grey pipe, hiding portions of it, below.
I do get the expected behaviors with a prior version of mayavi on a different OS- Windows (mayavi version TBC, probably 4.5)

[chiranthsiddappa]

I seem to be having the same issue with plot3d points placed around a 'globe':

    sphere = mlab.points3d(0, 0, 0, scale_mode='none',
                           scale_factor=2,
                           # color=(0.67, 0.77, 0.93),
                           color=ocean_blue,
                           resolution=50,
                           opacity=.85,
                           name='Earth')
    #
    # These parameters, as well as the color, where tweaked through the GUI,
    # with the record mode to produce lines of code usable in a script.
    sphere.actor.property.specular = 0.45
    sphere.actor.property.specular_power = 5
    # Backface culling is necessary for more a beautiful transparent
    # rendering.
    sphere.actor.property.backface_culling = True

[jeasinema]

+1 for pyqt/pyside backend, testing on Debian stretch with [email protected] and [email protected]

[prabhuramachandran]

This is a general issue with VTK it appears and has been seen before in #574, and #491. I haven't had the time to explore this or find a solution.

[lhvalentini]

@jmp75 i had the same problem..could you please confirm which version of mayavi worked?

[jmp75]

@lhvalentini I have this working using an Anaconda 2 (python 2.7.15) on Windows 10, with the packages mayavi 4.5.0 and vtk 6.3.0.

[prabhuramachandran]

Folks, if this is an issue with opacity, that is a different problem.

I tried the OPs code and it works correctly for me. If this is an OS specific issue it is probably an issue with your drivers as it definitely looks right for me on my Mac OS machine with VTK 8.1.1 and Mayavi from master. I doubt that anything has changed in Mayavi to fix this.

[jmp75]

@prabhuramachandran I will see if I can diagnose further to isolate the (a) root cause; indeed mayavi as such is not likely to be where something needs to be fixed if at all but will report in this thread if I can point to something helpful.

[gregoryrd]

I can reproduce the problem (using the colored balls example above) on RHEL 7, PyQt5, and Mayavi 4.6.2. I have been able to make it work correctly by uninstalling PyQt5 and installing wxPython. Unfortunately, I need PyQt5. On one of the two machines I am working with I can run it under VirtualGL (still using RHEL 7, PyQt5, and Mayavi 4.6.2), and the problem goes away. However, on the other machine (the production machine) VirtualGL causes Mayavi to crash.

Any suggestions on how to fix this when using PyQt5?

[gregoryrd]

Does the fact that I reproduced the problem in PyQt5 but not in wxPython provide any clues?

[solarjoe]

I have a similar problem on Linux Mint 18, 64-bit (Anaconda, Python 3.6, Qt 5.6.2, pyqt 5.6.0, VTK 8.1.1, Mayavi 4.6.2, installed from conda-forge).

If I just run the vanilla mlab.test_points3d() the object order is incorrect:

I tried to experiment with the aforementioned settings scene.renderer.use_depth_peeling and f.scene.renderer.maximum_number_of_peels, but without success.
Any ideas what could be the reason?

[prabhuramachandran]

An update. This issue looks like a Qt5 issue on Linux. I am able to see this on my ubuntu machines but not on OS X. Not sure about windows. I do not see it with a Qt4 backend. So it looks like somehow rendering is messed up when we use Qt5 and the QVTKRenderWindowInteractor. I've reported this issue upstream to the VTK folks as well. Am not sure what the issue is.

[ronceray]

Same problem here. I moved back to Qt4 and everything works fine.

[bsxfun]

I have this problem on Linux with Qt4 (activated with ipython's %gui qt4). Although maybe it's because I have Intel Graphics (something related already mentioned in the known issues).

[ronceray]

I had to install Qt4 too; somehow before that just putting "qt4" in the options didn't yield an error message, but used Qt5 instead.

[bsxfun]

ok i didn't realise that qt4 was not installed on my end (indeed, @ronceray, it doesn't yield an error)...

PyQt 4.12.3 didn't run, but I was able to get it to work with version 4.12.1..

Much laggier than PyQt5, but it doesn't have this foreground issue... saves me from setting up another graphics card!

[derekbrokeit]

I upgraded from Fedora 27 -> Fedora 29 and installed the latest NVIDIA graphics drivers. After this, mayavi performs as expected using pyqt5.

[sid5432]

I have been struggling with the same problem (incorrect depth order rendering) using Python3 for the last few weeks, and thought I'd share my experience here as well (as of 2019-04-15).

Just so we're on the same page, here are the versions I am looking at:

• mayavi: 4.6.2
• traits: 5.0.0
• traitsui: 6.0.0
• vtk: 8.1.2
• envisage: 4.7.1
• pyface: 6.0.0
• PyQt5: 5.12.1
• PyQt4: 4.11.4 (from dpkg)
• PySide: 1.2.2 (from dpkg)
• PySide2: 5.12.2

Mayavi/VTK/traits can use either the 'wx' toolkit or the 'qt4' toolkit. Wth the 'qt4' toolkit, there is the further option of specifying the QT API; the choices are 'pyqt', 'pyqt5', 'pyside', and 'pyside2'. These are specified with the environment variables ETS_TOOLKIT (set it to wx or qt4) and QT_API (set it to pyqt, pyqt5, pyside, or pyside2; only applicable if ETS_TOOLKIT is set to qt4).

Under Mint 18.3 (and probably for any Ubuntu 16.04 variant), the official Python3 is 3.5.2. Some modules are available with apt/dpkg, but several require using pip to install. I could not get wxPython to install for Python3. Using Qt, the results vary:

• ETS_TOOLKIT=qt4 and QT_API=pyqt : renders correctly
• ETS_TOOLKIT=qt4 and QT_API=pyqt5 : renders with wrong order
• ETS_TOOLKIT=qt4 and QT_API=pyside : renders correctly
• ETS_TOOLKIT=qt4 and QT_API=pyside2 : renders with wrong order

With Mint 19.1 (and probably all Ubuntu 18.04 variants) the results are similar. The Python3 version is 3.6.7; all the modules are the same as before, except that here I was able to install wxPython (4.0.1) with pip3. In this case setting ETS_TOOLKIT=wx gives the correct depth order. As for the qt4 toolkit, the results were the same as the Mint 18.3 results (i.e., pyqt and pyside rendered correctly; pyqt5 and pyside2 did not).

I tried compiling PySide2 from source and was able to get it to compile; the rendering is still wrong however.

I also tried using pyenv to test different versions of Python3 (in particular 3.5.2 and 3.7.3). In both cases, I could install the necessary modules through pip3, except the official PySide code does not support Python versions higher than 3.4. PySide2 is available, as is PyQt5; neither of these work correctly. I could not get wxPython to compile.

At this point, I am wondering if it is not a VTK problem, but rather a QT problem (I have given up on wxPython now). But curiously, on my work laptop (with Mint 18.3), with pyenv I had Python 3.7.3 running, and in this case, the depth order was rendered correctly -- with either PyQt5 or PySide2 (whereas on my home laptop both of the QT_API gave the wrong depth order). So what was the difference? As far as I can tell, the only difference was the OpenGL/GLX driver: my home laptops runs the SGI driver (as reported by glxinfo), while my work laptop uses the NVIDIA driver.

Finally, I also tried running a couple of tests with VirtualBox (Mint 19.1 with xfce). I am still trying different combinations, but so far the results are the same. (The glxinfo reports the OpenGL/GLX driver as SGI).

[nlgranger]

This thread on VTK ML seems to mention the same problem. Unfortunately, it was left unanswered...

[jmp75]

On a hunch I tried something on my Debian laptop worth reporting. This is an NVIDIA Optimus laptop as described in the Debian wiki. Basically there are two graphic cards on these systems: NVIDIA and Intel.

Short story: all other things being equal the original test script I posted renders incorrectly on the Intel card but correctly on the NVIDIA hardware.

environment (relevant packages):

envisage                  4.7.2                    pypi_0    pypi
mayavi                    4.6.2                    pypi_0    pypi
pyface                    6.0.0                    pypi_0    pypi
pygments                  2.3.1                    pypi_0    pypi
pyqt5                     5.12.2                   pypi_0    pypi
pyqt5-sip                 4.19.17                  pypi_0    pypi
python                    3.6.8                h0371630_0  
traits                    5.1.1                    pypi_0    pypi
traitsui                  6.0.0                    pypi_0    pypi
vtk                       8.1.2                    pypi_0    pypi

See the Debian wiki for how to use optirun. optirun python ~/src/tmp/qtbug.py worked correctly. To illustrate the difference with/without optirun, using the sample app:

glxgears -info returns

GL_RENDERER   = Mesa DRI Intel(R) HD Graphics 630 (Kaby Lake GT2) 
GL_VERSION    = 3.0 Mesa 18.3.4
GL_VENDOR     = Intel Open Source Technology Center

whereas optirun glxgears -info

GL_RENDERER   = Quadro M1200/PCIe/SSE2
GL_VERSION    = 4.6.0 NVIDIA 418.56
GL_VENDOR     = NVIDIA Corporation

[chrisjbillington]

Just to add another confirmation, here on Arch Linux on a Dell Precision 5520, I see the issue running on the intel chip, but not with the nvidia card via optirun. This is with Python 3.7, Qt5 and mayavi all installed via Arch's official repos.

[agarret7]

Same issue, running mayavi with PyQt5 backend results in incorrect rendering order using Mesa/SGI drivers on Debian Stretch, but it works with ETS_TOOLKIT=wx environment variable set.

[gstfrr]

I solved this by using wxPython instead of PyQt.

[jolaem]

This thread was just invaluable in solving the issue, many thanks for the comments.

Same problem, could not make it work with wx, so installed python-pyqt4, set ETS_TOOLKIT=qt4 and QT_API=pyqt and this has solved it.

[rahulgaur104]

I would like to add another confirmation for one of the configurations mentioned above. I am working on a Linux machine with Ubuntu 18.04 as its OS. My system has an Intel graphics card. I am using anaconda3 to animate the motion of a charged particle. The end result should be a helix with a line going through its centre. For my Python 3.6.8 environment with mayavi 4.6.2, vtk 8.1.2 and pyqt 5.9.2, I get something like this:

Note that the red line seems to be outside of the helix which is wrong.

To fix this problem I created a new Python 3.5 environment in anaconda3 and installed packages in the following order:

pyqt 4.11.4 from conda-forge
vtk 8.1.2 from pypi
mayavi 4.6.2 from conda-forge

and set the relevant variables in my shell using
export ETS_TOOLKIT=qt4
export QT_API=pyqt

and ran my code to get the correct diagram as shown below
.
Thanks to all the smart people above who figured it out. Cheers!

[gregoryrd]

After reproducing the problem (using the colored balls example above) earlier today, I was able to resolve the problem in a conda environment that contained:

1. Python 3.8.5
2. VTK 9.0.1 (installed using a whl file)
3. pyqt5
4. Latest 'Bleeding edge' version of Mayavi (4.7.3.dev0).

So perhaps VTK 9 fixes this issue?

[bburan]

I can confirm that VTK 9.0.1 with pyqt5 and Mayavi 4.7.2 (from conda-forge) fixes this issue.

[prabhuramachandran]

Yes indeed, I can confirm that this issue is now resolved with VTK-9.0.1 and the latest Mayavi, although it looks like VTK is where the fix has originated. I tested on a Linux machine where this used to fail earlier. This is great. Thanks for reporting it.