Configure auto logon for macos to support non-headless tests

[sxa]

Extracted from #184
@smlambert I'm splitting this out because the solution is different

[smlambert]

Thanks @sxa555, appreciated!

[jdekonin]

Try XQuartz
https://www.xquartz.org/

[sxa]

@gdams and I initially tried using xquartz but didn't manage to succeed within the jenkins environment for the JCK but we haven't got back to working it further again - if you have knowledge/experience of it @jdekonin that would be great :-)

[jdekonin]

@sxa555 I don't have experience running via jenkins on osx, so I can't help there. But that comment leads me to believe that if you ran manually and not through a jenkins agent/environment that it was working??

[sxa]

I think so yeah but it was a while ago :-) The xquartz server is probably still on the box I was using for testing

[geraintwjones]

I'm not convinced it helps, but I found this conversation in another community that talks about this issue. I'll keep digging.

[geraintwjones]

So I just typed which Xvfb on my macbook. It returned /opt/X11/bin/Xvfb. On most Unix systems it's in /usr/bin/Xvfb. I'm probably being naïve here, but is it not as simple as doing something with the path, or setting up a symbolic link?

[sxa]

@geraintwjones No idea -you tell us - does it work? ;-) I don't mind whether it's called Xvfb or xvfb

[sxa]

@smlambert A thought - do we require remote access to your mac GUI display to be able to run GUI tests? (Ref: #267 which I've raised for JCK purposes)

[sxa]

@Haroon-Khel We never managed to get this to a solution - since you looked at the Xvfb stuff on other platforms and are familiar with running the tests can you see if you can make an equivalent work on macos?

[Haroon-Khel]

I've tried to use the Xvfb jenkins plugin, like we do with Linux, but I couldn't quite get it to work. I tested this in this grinder on test-macincloud-macos1014-x64-1, which does have Xvfb/Xquartz installed, using the jdk test awt/BasicStroke/DashOffset.java which fails if Xvfb isn't correctly setup.

According to https://stackoverflow.com/questions/31857203/jenkins-cannot-run-program-xvfb-java-io-ioexception-no-such-file-or-director, the error in that grinder suggests that Xvfb isn't added to the PATH variable, but it certainly is on the machine; running which Xvfb returns /opt/X11/bin/Xvfb, so I'm not sure what is causing the error

[Haroon-Khel]

My plan now is to use the test scripts to use bash commands to setup an Xvfb server and export DISPLAY accordingly, during a test run, and then shut down the server after the run sort of how we do with aix. However to implement this in the test scripts, I first need to achieve this locally on the machine, using test-macincloud-macos1014-x64-1 again.

I'm using Xvfb :0 -ac -screen 0 1024x768x8 to run an Xvfb process, which seems to run fine, and exporting DISPLAY=:0. However when I run the test awt/BasicStroke/DashOffset.java locally I get the following error:

java.awt.AWTError: WindowServer is not available
        at java.desktop/sun.lwawt.macosx.LWCToolkit.<clinit>(LWCToolkit.java:166)
        at java.base/java.lang.Class.forNameImpl(Native Method)
        at java.base/java.lang.Class.forName(Class.java:342)

which sort of suggests that more than just an Xvfb process needs to be running. Has anyone here managed to get an X11 based test to run locally on a mac?

[gdams]

the error in that grinder suggests that Xvfb isn't added to the PATH variable, but it certainly is on the machine; running which Xvfb returns /opt/X11/bin/Xvfb

@Haroon-Khel looking at that grinder run it appears that /opt/X11/bin isn't in the PATH:

17:58:13  ITERATIONS=1
17:58:13  PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin/
17:58:13  MAIL=/var/mail/jenkins

I've kicked off this grinder with the PATH updated - let's see how it goes:

18:56:49  ITERATIONS=1
18:56:49  PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin/:/opt/X11/bin
18:56:49  MAIL=/var/mail/jenkins

[smlambert]

From grinder launched above:
Caused: java.io.IOException: Cannot run program "Xvfb": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at hudson.Proc$LocalProc.(Proc.java:252)
at hudson.Proc$LocalProc.(Proc.java:221)

[smlambert]

Is xquartz installed on the mac machine that Grinder is being run on, I am presuming not as I do not think that is in any ansible playbook or regular config setup at present. As I mentioned in #1450 (closed as dup of this issue), there were some attempts internally to enable, and eventual success on a machine (referenced in internal backlog/280 issue).

[gdams]

It's there:

test-macincloud-macos1014-x64-1:~ admin$ which xvfb
/opt/X11/bin/xvfb

[Haroon-Khel]

I created a link between /opt/X11/bin/Xvfb and /usr/bin/Xvfb. In this grinder, it seems jenkins is now using the plugin, but is now getting the same error that I would get by running the test locally (which i've posted in one of my comments above):

14:18:21  java.awt.AWTError: WindowServer is not available
14:18:21  	at java.desktop/sun.lwawt.macosx.LWCToolkit.<clinit>(LWCToolkit.java:166)
14:18:21  	at java.base/java.lang.Class.forNameImpl(Native Method)
14:18:21  	at java.base/java.lang.Class.forName(Class.java:342)
14:18:21  	at java.desktop/java.awt.Toolkit$2.run(Toolkit.java:588)
14:18:21  	at java.desktop/java.awt.Toolkit$2.run(Toolkit.java:583)

This suggests that more than simply an Xvfb session needs to be running for these tests to work

[gdams]

just tried running this on the mac in question as the Jenkins user:

bash-3.2$ Xvfb
_XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
_XSERVTransMakeAllCOTSServerListeners: server already running
(EE) 
Fatal server error:
(EE) Cannot establish any listening sockets - Make sure an X server isn't already running(EE) 

this could well be the issue, just investigating

[sxa]

Suggest not using the first display i.e. try Xvfb :1 or a higher number in case it's defaulting to zero which will be the physical display on the machine (Not sure if I have access to the machine to look myself)

[sxa]

That doesn't stop the reviewer from decrypting it locally and making sure it hasn't corrupted the content, which has happened in the past

[jdekonin]

I don't see DISPLAY in machine configuration environment variables or added in a playbook. Is DISPLAY set somewhere I am missing?

[Haroon-Khel]

@jdekonin
I believe its in the test scripts

https://github.com/adoptium/aqa-tests/blob/master/buildenv/jenkins/JenkinsfileBase#L582

[llxia]

@Haroon-Khel did you change the script for testing Mac? Currently, the test scripts turn on Xvfb for AIX and Linux, not Mac.

[smlambert]

I believe Andreas's findings were that that xvfb is not used for Mac (just as its not needed for other 'windowed' systems like Windows).

Main part that is required for Mac is that the machines do need their configuration to be updated (as per #2062) to disable screen lock).

[lumpfish]

I've got here following a path of closed issues trying to find out if there is still an open issue for what is causing the openjdk test failures described here, which are still occurring: adoptium/aqa-tests#135
I'd like to know if this is the issue I should reference to exclude the tests via ProblemList.txt

[smlambert]

re: #211 (comment) - yes, we should always attempt to identify the root cause of test failures, create the issue in the appropriate repository, and then exclude the tests under the root cause issue.

In this case, jdk_awt and jdk_swing and several other test cases from other test targets fail on mac with errors
like could not find editor can be resolved when this infrastructure issue is addressed.

There are a couple of jck test targets on mac that are also affected. Without this issue being fixed, the few jck targets will have to be added to the list of manual tests, as automated runs will fail with DISPLAY issues.

[benty-amzn]

We've been working to solve a similar issue for Corretto, and the information on this thread has been very helpful.

The exceptionjava.awt.AWTError: WindowServer is not available mentioned earlier in the thread is thrown by this code (jdk11 is used as an example, but the code is also present in 8 and 16).
The relevant check here is isInAquaSession() which can be overridden by an environment variable, AWT_FORCE_HEADFUL.

By setting AWT_FORCE_HEADFUL=true in our test environment we have been able to successfully execute awt/swing tests over VNC on macos. Note that with this environment variable set, we have not found a need to set DISPLAY (which appears to be ignored on macos) or to use other dependencies such as xvfb or XQuartz.

[karianna]

We've been working to solve a similar issue for Corretto, and the information on this thread has been very helpful.

The exceptionjava.awt.AWTError: WindowServer is not available mentioned earlier in the thread is thrown by this code (jdk11 is used as an example, but the code is also present in 8 and 16).
The relevant check here is isInAquaSession() which can be overridden by an environment variable, AWT_FORCE_HEADFUL.

By setting AWT_FORCE_HEADFUL=true in our test environment we have been able to successfully execute awt/swing tests over VNC on macos. Note that with this environment variable set, we have not found a need to set DISPLAY (which appears to be ignored on macos) or to use other dependencies such as xvfb or XQuartz.

Thanks, Ben, really good of you to chime in and help us out!

[sophia-guo]

Feels this can be closed @sxa or @Haroon-Khel ?

[smlambert]

I am not sure the mac machines attached to the public Jenkins servers have been configured (or that the openjdk tests that require it have been re-enabled), differs from the temurin-compliance and jck test targets.

jdk_awt/L428, jdk_swing/L966