Default to a 4K x 4K browser window

[Rotonen]

The largest categories of Robot test flakiness we have are:

1. Things being unclickable by being outside the browser viewport
2. Interactive elements, like form buttons, collapsing into themselves in a way where they overlap in a mobile view

Both are caused by the default window size being a rather arcane 800 x 600. (Or in the case of Xvfb, also limited by the default desktop size and color depth of 640x480x8). In some cases, with Xvfb, the browser defaults, for unknown reasons, to a window size of 1024x768. None of these represent contemporary actual use very well.

The usability issues, especially the mobile ones, can be real usability issues and should thus be eventually tackled. I deem stable tests more valuable at this point in time. Currently these issues manifest mostly for people whom are not tackling them and they simply demotivate contributors.

Tackling those has also been a very annoying heisenbug for developers, as the browser window sizes on their local runs are different than on the CI as on desktop environments the default browser size is larger than in a naked X11 session. And someone with a smaller screen, for example on a smaller laptop, cannot even set their browser window size very large and is currently stuck with flakier Robot tests.

Thus we also need to use a headless browser without Xvfb for both the CI runs and local development. The support for this is already present in geckodriver, chromedriver, the version of Selenium we use, the version of Robot framework we use and thus also in our testing stack. Using a headless browser for local development is also a more pleasant experience as it does not spawn windows (some of which pop onto the foreground and gain focus). This allows the developer to do something else while waiting for the tests to run.

If there is just a single place from where the viewport size can be controlled, anyone interested in tackling these issues can do a local checkout of plone.app.robotframework and set the viewport size to whatever they're interested in working on.

I've figured out the path to get to that point and it starts here.

Attending to this one will happen in multiple parts:

1. Set the window size in plone.app.robotframework to a suitably unrealistic 4K x 4K
2. Remove window size setting from all Robot test layers and tests
3. Use the same shared test setup in all Robot tests
4. Use headless browsers on the CI instead of xvfb-run wrapped windowed browsers
5. Default to headless browsers for all Robot testing in Plone

For anyone interested in giving the headless browsers a spin locally:
ROBOT_BROWSER=headlesschrome
ROBOT_BROWSER=headlessfirefox

[Rotonen]

@jenkins-plone-org please run jobs

[Rotonen]

@jenkins-plone-org please run jobs

[Rotonen]

Seems I'll also need to convert the tests and documentation of this package itself. Apparently Products.CMFPlone is just about the only package which uses the shared setups from here.

[Rotonen]

I'm deliberately not yet running the tests on the Plone CI now as I'm hunting for what broke in plone/plone.dexterity#102 for Python 2. After the 5.2 branch of buildout.coredev is green again, I'll resume hunting for all things Robot.

[Rotonen]

@jenkins-plone-org please run jobs

[jensens]

LGTM, merge yourself if you think its ready!

[Rotonen]

I'll wait for the Zope 4 dust to settle first at least. @pbauer is currently on that one.

I want to be increasing the certainty and reliability of the test runs.

I'm not in a particular hurry to merge, this will take many pull requests to get everything done and as our CI infra elegantly allows the testing of combinations of PRs, I can work towards the end goal with or without some parts of the puzzle having been merged.

I'll merge this when we have a green 5.2 again and I have a few subsequent PRs tested against this one.

[Rotonen]

As the layer PRs are now in, I'll merge this and start requesting reviews for the ones which are only tested against 5.2 on the master branch.

I'll later need to port this to at least 5.1 as well.

[mauritsvanrees]

I have released plone.app.robotframework 1.5.2 with this change.

Tiny note: please do not add dashes in front of the news snippet files. They get included literally when towncrier runs, and towncrier adds its own dashes so you end up with double dashes.
And please only use one snippet per file. If you want more items in the resulting changelog, you can put them in separate files, like I did here before the release.

[Rotonen]

@mauritsvanrees thank you. First time driving towncrier. I'll amend all the other PRs.

[thet]

At the Buschenschanksprint we encountered high performance requirements for running the Jenkins Robot tests on a node we own.
The 4k change might make the browsers require up more CPU/RAM resources than before.

Just as a side note.

[Rotonen]

@thet using a headless browser without Xvfb has negated both extra CPU and GPU use in testing.

See and track also plone/jenkins.plone.org#259