[ci] fix locale-setting in jobs running in ubuntu container

[jameslamb]

Today, the Linux CI jobs on Azure DevOps run in a container using the ubuntu:22.04 image. Since that image is just a minimal operating system distribution, running LightGBM's CI in it requires a lot of extra configuration.

Here's where much of that happens:

LightGBM/.ci/setup.sh

Line 26 in a174893

if [[ $IN_UBUNTU_LATEST_CONTAINER == "true" ]]; then

I noticed while working on #5638 that one piece of that has been silently failing for a while (unsure how long).

There are a few problems with this bit of configuration that tries to change the locale inside the container to en_US.UTF-8:

LightGBM/.ci/setup.sh

Lines 58 to 61 in a174893

	export LANG="en_US.UTF-8"
	export LC_ALL="${LANG}"
	sudo locale-gen ${LANG}
	sudo update-locale

Setting LC_ALL environment variable to a locale that doesn't exist yet (i.e. hasn't been downloaded or generated with locale-gen) isn't possible. That's why this warning shows up in the Linux_latest logs

/__w/1/s/.ci/setup.sh: line 59: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

(recent build on master)

Setting environment variables inside a process only affects that process and it's sub-processes...so those variables won't be used by test.sh, since it runs in a separate process not as a sub-process of setup.sh

LightGBM/.vsts-ci.yml

Lines 142 to 145 in a174893

	- bash: $(Build.SourcesDirectory)/.ci/setup.sh
	displayName: Setup
	- bash: $(Build.SourcesDirectory)/.ci/test.sh
	displayName: Test

I noticed this because once I started trying to get .ci/test_r_package.sh to run in an ubuntu:22.04 container, the R unit test about handling of non-ASCII feature names was failing, and R was reporting that it had fallen back to the default character set for the image (ASCII, the "POSIX" locale).

This PR proposes the following changes:

• just install locales-all instead of generating a specific locale with locale-gen
• set environment variable LC_ALL again in test.sh
• switch environment variable IN_UBUNTU_LATEST_CONTAINER to IN_UBUNTU_BASE_CONTAINER
  • In [ci] run r-package Linux jobs in containers #5638, I'd like to run the R 3.6 (older version of R) linux jobs in an ubuntu:18.04 instead of ubuntu:22.04 container, but it can re-use all of the setup stuff.
  • So everything these scripts guard with that environment variable as being relevant in an ubuntu "latest" container are actually just useful whenever running the scripts in an ubuntu:{something} container

[github-actions]

This pull request has been automatically locked since there has not been any recent activity since it was closed.
To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues
including a reference to this.