sddm-greeter coredump due to GLX on Laptop with Optimus sync

[johnalotoski]

Describe the bug
On a system76 Gazelle laptop with HDMI and mini-Displayport outputs, and Optimus capable graphics hardware (Intel, Nvidia, details below), the only way I've found so far to get a video signal to both the HDMI and mini-DisplayPort outputs at the same time is to use Optimus mode, like with the following nixos configuration:

  boot.kernelPackages = pkgs.linuxPackages_latest;
  services.xserver.videoDrivers = [ "intel" "nvidia" ];
  hardware.nvidia.optimus_prime = {
    enable = true;
    nvidiaBusId = "PCI:1:0:0";
    intelBusId = "PCI:0:2:0";
  };

Xserver configuration is:

  services.xserver.desktopManager.plasma5.enable = true;
  services.xserver.displayManager.sddm.enable = true;
  services.xserver.enable = true;
  services.xserver.exportConfiguration = true;
  services.xserver.layout = "us";
  services.xserver.libinput.enable = true;

The graphics hardware on the system is:

$ lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation UHD Graphics 630 (Mobile)
01:00.0 VGA compatible controller: NVIDIA Corporation TU117M [GeForce GTX 1650 Mobile / Max-Q] (rev a1)

When booting using this setup, I can see Xwindows start and a non-blinking underline-cursor appears on all 3 monitors but nothing further happens and other than the cursor being visible, the screens (the primary laptop display, an external monitor hooked up to the HDMI output and another external monitor hooked up to the mini-DisplayPort output) are black.

Switching to a console with CTRL-ALT-F1, and investigating the boot logs, it seems the GLX driver can no longer be loaded with this configuration:

Aug 05 22:49:23 nixos-g76 xserver-wrapper[3228]: (EE) AIGLX error: dlopen of /run/opengl-driver/lib/dri/i965_dri.so failed (/run/opengl-driver/lib/dri/i965_dri.so: cannot open shared object file: No such file or directory)
Aug 05 22:49:23 nixos-g76 xserver-wrapper[3228]: (EE) AIGLX error: unable to load driver i965
Aug 05 22:49:23 nixos-g76 xserver-wrapper[3228]: (EE) AIGLX error: dlopen of /run/opengl-driver/lib/dri/swrast_dri.so failed (/run/opengl-driver/lib/dri/swrast_dri.so: cannot open shared object file: No such file or directory)
Aug 05 22:49:23 nixos-g76 xserver-wrapper[3228]: (EE) AIGLX error: unable to load driver swrast
Aug 05 22:49:23 nixos-g76 xserver-wrapper[3228]: (EE) GLX: could not load software renderer

sddm-greeter then proceeds to throw a qglx_findConfig error and coredump:

Aug 05 22:49:24 nixos-g76 sddm-greeter[3444]: qglx_findConfig: Failed to finding matching FBConfig for QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SingleBuffer, swapInterval 1, colorSpace QSurfaceFormat::DefaultColorSpace, profile  QSurfaceFormat::NoProfile)
Aug 05 22:49:24 nixos-g76 sddm-greeter[3444]: qglx_findConfig: Failed to finding matching FBConfig for QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SingleBuffer, swapInterval 1, colorSpace QSurfaceFormat::DefaultColorSpace, profile  QSurfaceFormat::NoProfile)
Aug 05 22:49:24 nixos-g76 sddm-greeter[3444]: qglx_findConfig: Failed to finding matching FBConfig for QSurfaceFormat(version 2.0, options QFlags<QSurfaceFormat::FormatOption>(), depthBufferSize -1, redBufferSize 1, greenBufferSize 1, blueBufferSize 1, alphaBufferSize -1, stencilBufferSize -1, samples -1, swapBehavior QSurfaceFormat::SingleBuffer, swapInterval 1, colorSpace QSurfaceFormat::DefaultColorSpace, profile  QSurfaceFormat::NoProfile)
Aug 05 22:49:24 nixos-g76 sddm-greeter[3444]: Could not initialize GLX
Aug 05 22:49:24 nixos-g76 systemd[1]: Created slice system-systemd\x2dcoredump.slice.
Aug 05 22:49:24 nixos-g76 systemd[1]: Started Process Core Dump (PID 3447/UID 0).
Aug 05 22:49:24 nixos-g76 sddm-helper[3436]: [PAM] Closing session
Aug 05 22:49:24 nixos-g76 sddm-helper[3436]: [PAM] Ended.
Aug 05 22:49:24 nixos-g76 systemd[1]: session-c1.scope: Succeeded.
Aug 05 22:49:24 nixos-g76 sddm[3152]: Auth: sddm-helper exited with 6
Aug 05 22:49:24 nixos-g76 sddm[3152]: Greeter stopped.
Aug 05 22:49:24 nixos-g76 systemd-logind[2652]: Session c1 logged out. Waiting for processes to exit.
Aug 05 22:49:24 nixos-g76 systemd-logind[2652]: Removed session c1.
Aug 05 22:49:24 nixos-g76 systemd-coredump[3449]: Cannot resolve systemd-coredump user. Proceeding to dump core as root: No such process
Aug 05 22:49:24 nixos-g76 systemd-coredump[3449]: Process 3444 (.sddm-greeter-w) of user 175 dumped core.

Additional context
If the video driver configuration is either intel alone, or nvidia with modesetting, this GLX error and coredump is not an issue:

  # This works but only outputs to the HDMI output, not the mini-Displayport
  #services.xserver.videoDrivers = [ "intel" ];

  # This also works but similarly only outputs to the HDMI output, not the mini-Displayport
  #services.xserver.videoDrivers = [ "modesetting" "nvidia" ];

This issue appears similar to the following other issues (where the same type of qglx_findConfig error is reported):
#65399
#85921
#82959

Notify maintainers
(Adding these names here as users having participated in the above issues with similar errors as a starting point)
@ttuegel
@matthewbauer

Metadata

• system: "x86_64-linux"
• host os: Linux 5.7.11, NixOS, 20.03.2685.977000f149b (Markhor)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.3.6
• channels(root): "nixos-20.03.2685.977000f149b, nixosunstable-20.09pre236721.840c782d507"
• channels(user): "nixosunstable-20.09pre222244.22a3bf9fb9e, nixosuser-20.03.1577.74a80c5a9ab"
• nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

[johnalotoski]

Issue resolved. I do need Optimus mode to get both HDMI and miniDisplay output to external monitors at the same time. My confusion here was the number of tuning parameters available and the different ways it fails if the selected options aren't compatible. Also, I didn't realize that for Optimus mode, only Nvidia driver is required. To summarize:

The following combinations failed to achieve video output to the laptop LCD and all external output ports:

1. Intel only, optimus off (doesn't output to mDP)

• services.xserver.videoDrivers = [ "intel" ];

2. Nvidia only, optimus off (doesn't output to mDP)

• services.xserver.videoDrivers = [ "modesetting" "nvidia" ];
• This options requires the "modesetting" driver or there is no video output at all

3. Intel + Nvidia, optimus on or off, modesetting on or off (no video output)

• services.xserver.videoDrivers = [ "intel" "nvidia" ]; # or [ "modesetting" "intel" "nvidia" ]
• With or without optimus_prime attrs shown above

4. Nvidia, modesetting on, optimus on (no video output)

• services.xserver.videoDrivers = [ "modesetting" "nvidia" ];
• With optimus_prime attrs shown above

I think when initially trying to resolve this, I tested all the above combinations, believing I had exhausted the test possibilities, but in fact, I just needed to remove the "modesetting" driver from option (4) and my issue is resolved. I had left that "modesetting" driver option enabled when initially testing (4) because I found it was required to work with just the nvidia driver in (2) when Optimus was off, so it stood to reason it was required when Optimus was on too. Now I get full output to all external display ports now and no obvious video driver problems. Working config is simply:

  services.xserver.videoDrivers = [ "nvidia" ];
  hardware.nvidia.optimus_prime = {
    enable = true;
    nvidiaBusId = "PCI:1:0:0";
    intelBusId = "PCI:0:2:0";
  };