Export the DEVELOPER_DIR environment variable instead of invoking xcode-select to set the active Xcode version.

Setting the active Xcode version with sudo xcode-select changes the setting system wide.
Github actions can also be run with a self-hosted runner: https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners. When using a self-hosted github runner, the run isn't necessarily containerized. Changing the system-wide Xcode version then creates conflicts between concurrent runs.

From man xcode-select:

ENVIRONMENT
DEVELOPER_DIR
Overrides the active developer directory. When DEVELOPER_DIR is set, its value will be used instead of the system-wide active developer directory.