Do not download updates by default

[paulb-elastic]

Requirement
As a user of Elastic Synthetics in an environment without internet access
I want an option to prevent Synthetics from updating from the internet
So that I can use Synthetics based on the version I have installed

In reality, this will mostly be used in the docker version of the Synthetics Agent, as this already contains all of the components needed to run a test. However, this should be a Synthetics Agent option (which can be passed in to a docker run as well), which in theory could still be used if all the components and NPM packages had been installed manually somehow in the environment without internet access.

This specifically relates to inline monitors (those defined in the heartbeat.yml) as suites are handled differently. In the case of suites, an npm install has to run to ensure any dependencies in the package.json have been met (e.g. a user could add a new requirement for an additional node module for something they are doing in the script). The user is in full control of the package.json, which includes any versions of any packages (as well as Synthetics).

Acceptance Criteria

• A configuration/parameter option to prevent the Synthetics Agent from attempting to update any component (Playwright, Chromium, Synthetics library etc.)
• By default, updates will be disabled
• When updates are disabled (the default), the Agent will attempt to run with the components available and not download or update any from the internet
• When updates are enabled, the Synthetics Agent will do what it does now, looking for updates when run

@jahtalab has pointed out this area of the code to modify behaviour: https://github.com/elastic/beats/blob/heartbeat-synthetics/x-pack/heartbeat/monitors/browser/synthexec/synthexec.go#L57

[paulb-elastic]

There has been some discussion on this, but it warrants a wider discussion that we should have here.

Having an option to stop automatically (and forcefully) looking for updates, and downloading Playwright and Chromium, solves one particular use case, however it’s worth considering wider options.

Firstly, making the forced update fail silently would be an alternative to an option to prevent it from downloading (so at least it would continue to run with what’s in the docker container, even if it can’t update from external locations).

However, a wider reaching alternative worth discussing, is about not updating at all, but controlling the updates through the docker image itself.

One of the reasons for the auto-update is to ensure that Synthetics is always using an up to date Chromium browser. Not only is this typically more secure, it also more accurately reflects the browser that internet users will be using to access websites (as browsers typically auto-update now).

The downside of this is that the Heartbeat machine needs access to the internet to download updates to Playwright (from the NPM repository) and Chromium (from upstream).

An alternative would be to build and release updates to the docker image, with each update to Playwright (which will also include the defined version of Chromium).

An advantage of this is that we could run our tests with each update, to ensure there are no breaking changes in the Playwright/Chromium update with Synthetics. We would release the docker image as a ‘stack’ with a working set of software versions.

The disadvantage of this is that there is a higher effort for users to keep their Synthetic Agents up to date, the most notable thing being the version of Chromium running. If users do not update their docker images regularly, they will be running with older and older versions of the Chromium browser that less reflect the browsers used by users of their website.

This is a quite specific discussion about updates, that has a bearing on the wider packaging discussion in #140.

/cc @elastic/uptime @vigneshshanmugam @jahtalab

[vigneshshanmugam]

This is an interesting issue and there is an alternative solution to this problem to the one proposed above.

We control the installation through environment variable for air gapped and other for non air gapped environments.

Solution to air gapped environments

For air gapped env, an easier solution would to use the browsers from local node_modules directory and use the same for all packages that synthetics library depend on. The node_modules directory would be available already in the docker image (This would mean heartbeat machine does not need access to internet to download playwright/browser updates and other packages).

With each update of the Node.js synthetics library, we release a new docker image with modified versions of playwright if required and we could still keep the compatibility going on as usual. The disadvantage of not updating the docker image regularly, still exists but if users needs new features, then we ask them to update the images regularly like every other Elastic stack update.

Solution to non air gapped env

For non air gapped env, we always rely on the latest version of the Node.js synthetic library which would install the pinned playwright versions. This would work like how its working right now in master branch.

[vigneshshanmugam]

Did some work on this branch to make it work with locally installed node_modules. https://github.com/vigneshshanmugam/synthetics/tree/local-packages/examples

[paulb-elastic]

I have updated the title and requirement definition slightly, after some discussions about this with @andrewvc.

This requirement specifically relates to inline monitors, as suites are handled separately, via the package.json, for which an npm install will need to happen (as the user may have added additional dependencies). This will include the version of Synthetics (either any, or a specific version), and as such is user the user's control.

[paulb-elastic]

For reference, elastic/observability-docs#284 will document more detail about how the package.json works and should be used.

[paulb-elastic]

Closing this in favor of elastic/beats#22928 as this and #160 cover the auto-update and offline requirement (with consideration to the differences between inline and suite monitors).