[CI] Java Packaging test05InstallFailsWhenJavaMissing failure

[davidkyle]

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+packaging-tests/303/console

There was 1 failure:
1) test05InstallFailsWhenJavaMissing(org.elasticsearch.packaging.test.DefaultDebBasicTests)
java.lang.RuntimeException: dpkg failed and the lockfile still exists. Failure:
<org.elasticsearch.packaging.util.Shell$Result exitCode = [2] stdout = [] stderr = [dpkg: error: dpkg frontend is locked by another process
]>
Lockfile:
<org.elasticsearch.packaging.util.Shell$Result exitCode = [0] stdout = [COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
apt-get 16941 root    5uW  REG    8,3        0 392693 /var/lib/dpkg/lock
] stderr = []>
	at __randomizedtesting.SeedInfo.seed([EBFFF57A9D273FA3:EAC50CAAA80B5B66]:0)
	at org.elasticsearch.packaging.util.Packages.runInstallCommand(Packages.java:119)
	at org.elasticsearch.packaging.util.Packages.runInstallCommand(Packages.java:105)
	at org.elasticsearch.packaging.test.PackageTestCase.test05InstallFailsWhenJavaMissing(PackageTestCase.java:71)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:404)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:708)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:629)

Reproduce:
./gradlew :qa:vagrant:vagrantUbuntu1604#javaPackagingTest -Dtests.seed=20AC14AFD47FDD08

Could be related to #33762

[elasticmachine]

Pinging @elastic/es-core-infra

[dliappis]

dpkg failed and the lockfile still exists.

This is most likely due to apt-daily-upgrade.timer and unattended-upgrades.service enabled (which is the default) in our the Elastic Vagrant boxes used by Elasticsearch packaging tests.

New versions of these boxes (Ubuntu 16.04/18.04 + Debian 9) have been pushed (see below) with the aforementioned services disabled (note: unattended-upgrades isn't installed on Debian 9 boxes).
This hopefully will prevent such failures happening in the future.

Closing the issue now, will monitor stats and re-open if it resurfaces.

for i in ubuntu1604vbox ubuntu1804vbox; do vagrant ssh $i -c 'sudo systemctl status apt-daily.timer apt-daily-upgrade.timer unattended-upgrades.service'; done
● apt-daily.timer - Daily apt download activities
   Loaded: loaded (/lib/systemd/system/apt-daily.timer; disabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/apt-daily.timer.d
           └─apt-daily.timer.conf
   Active: inactive (dead)

● apt-daily-upgrade.timer - Daily apt upgrade and clean activities
   Loaded: loaded (/lib/systemd/system/apt-daily-upgrade.timer; disabled; vendor preset: enabled)
   Active: inactive (dead)

● unattended-upgrades.service - Unattended Upgrades Shutdown
   Loaded: loaded (/lib/systemd/system/unattended-upgrades.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:unattended-upgrade(8)
Connection to 127.0.0.1 closed.
● apt-daily.timer - Daily apt download activities
   Loaded: loaded (/lib/systemd/system/apt-daily.timer; disabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/apt-daily.timer.d
           └─apt-daily.timer.conf
   Active: inactive (dead)
  Trigger: n/a

● apt-daily-upgrade.timer - Daily apt upgrade and clean activities
   Loaded: loaded (/lib/systemd/system/apt-daily-upgrade.timer; disabled; vendor preset: enabled)
   Active: inactive (dead)
  Trigger: n/a

● unattended-upgrades.service - Unattended Upgrades Shutdown
   Loaded: loaded (/lib/systemd/system/unattended-upgrades.service; disabled; vendor preset: enabled)
   Active: inactive (dead)
     Docs: man:unattended-upgrade(8)
Connection to 127.0.0.1 closed.
$ vagrant ssh debian9vbox -c 'sudo systemctl status apt-daily.timer apt-daily-upgrade.timer'
● apt-daily.timer - Daily apt download activities
   Loaded: loaded (/lib/systemd/system/apt-daily.timer; disabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/apt-daily.timer.d
           └─apt-daily.timer.conf
   Active: inactive (dead)

● apt-daily-upgrade.timer - Daily apt upgrade and clean activities
   Loaded: loaded (/lib/systemd/system/apt-daily-upgrade.timer; disabled; vendor preset: enabled)
   Active: inactive (dead)
Connection to 127.0.0.1 closed.

[harishgrb]

Still facing this issue.

hguruba@hguruba-in-ll01:$ sudo rm /var/lib/dpkg/lock
hguruba@hguruba-in-ll01:$ sudo dpkg --configure -a
dpkg: error: dpkg frontend is locked by another process
hguruba@hguruba-in-ll01:~$