[🚫 Bug] ExTester CI fails with ubuntu-24.04 runner image

[TamiTakamiya]

Describe the bug

We are trying to use the new ubuntu-24.04 GitHub Action runner image in our CI and found UI test was failing (reference). As far I looked at the failure, it seemed to occur when the VSBrowser.openResources() API was called.
I looked at the ExTester's CI setup and found it is using ubuntu-latest (=22.04) image. So I attempted to replace it to ubuntu-24.04 (commit) and run the Main CI (results) and saw the CI failed with ubuntu-24.04. The first failure occurred with Open resource test:

  1) Open resource test
       Single folder is open from CLI:
     TimeoutError: Error: Could not get title from TitleBar.
Wait timed out after 28208ms
      at /home/runner/work/vscode-extension-tester/vscode-extension-tester/node_modules/selenium-webdriver/lib/webdriver.js:913:22
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

These results suggest some compatibility issues in ExTester's opening resource function on ubuntu-24.04.

Steps to reproduce

1. Create a bew branch from the latest main branch of vscode-extension-tester
2. Replace the string `ubuntu-latest` that appears in files with `ubuntu-24.04`
3. Run the Main CI workflow

Logs

See the description above.

Operating System

Ubuntu 24.04 LTS

Visual Studio Code

1.92.2, 1.91.1, 1.90.2

vscode-extension-tester

latest (8.6.0)

NodeJS

20.16.0

npm

10.8.1

[djelinek]

I was able to reproduced on GH Actions but when I tried Ubuntu 24.04 in VM outside of github runners it was working fine..
on the screenshot from a failed CI run the vscode workbench seems to be shown correctly..

also I don't believe that there would be some VS Code DOM change depending just on OS

the true is that xvfb version between ubuntu 22.04 and 24.04 was changed so maybe the problem can be there.. unfortunately I was not able to install previous version of xvfb into ubuntu-2404 runner 😞

[odockal]

It seems that you are facing the same problem as we did: podman-desktop/podman-desktop#7256.

To fix the problem, make sure to run this code on ubuntu 24.04 before running the tests: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
PR: #1599.