[Che Issue] Maven terminal reset during build

[ScrewTSW]

For some strange reason the terminal output got reset during application build.

This issue appeared three times during the weekend on production 1b cluster.

[Katka92]

Seen again on 2a https://ci.centos.org/job/devtools-rh-che-periodic-prod-2aProd/1320/console

[Katka92]

Seen again on 1a https://ci.centos.org/job/devtools-rh-che-periodic-prod-1a/1329/console

[ScrewTSW]

happened again in the Vert.x stack test today on 1a

[ibuziuk]

@ScrewTSW @Katka92 do we have an issue for upsrtream (this does not look Hosted Che specific)?
I saw there were some terminal related issue opened already (not sure those are related those) e.g eclipse-che/che#14887

[Katka92]

Seen again on 1b https://ci.centos.org/job/devtools-rh-che-periodic-prod-1b/1421/console

[Katka92]

Seen again on 1b.

[Katka92]

Not reproduced for 22 days. Closing.

[Katka92]

Seen again on production cluster 2a.

[Katka92]

Seen on production 1b cluster 1x https://ci.centos.org/job/devtools-rh-che-periodic-prod-1b/2173/console

[ScrewTSW]

seen again on production cluster 2a x2:
https://ci.centos.org/view/Devtools/job/devtools-rh-che-periodic-prod-2aProd/2179/console
https://ci.centos.org/view/Devtools/job/devtools-rh-che-periodic-prod-2aProd/2183/console
on production cluster 1b x1:
https://ci.centos.org/view/Devtools/job/devtools-rh-che-periodic-prod-1b/2194/console

[ScrewTSW]

Seen again on production cluster 2a x1:
https://ci.centos.org/view/Devtools/job/devtools-rh-che-periodic-prod-2aProd/2204/console

[Katka92]

Seen again on cluster 2 1x
https://ci.centos.org/job/devtools-rh-che-periodic-prod-2/2600/console

[Katka92]

Now we have a video and the reason for that failure seems to be clear. The workspace switches to offline and then the terminal output is removed and the command is kind of stopped.
See the video [1] from 1:12.
It's not clear to me why the workspace turned to offline. There are some 403 in browser logs [2] but I'm not sure if those are related to the problem. There is no evicted event in events log [3].

[1] http://artifacts.ci.centos.org/devtools/rhche/devtools-rh-che-periodic-prod-2/2774/e2e_report/report/ffmpeg_report/output.mp4
[2] http://artifacts.ci.centos.org/devtools/rhche/devtools-rh-che-periodic-prod-2/2774/e2e_report/report/Java_Maven_test_Validation_of_workspace_build_and_run_Run_command_'maven_build'/browserlogs-Run_command_'maven_build'.txt
[3] http://artifacts.ci.centos.org/devtools/rhche/devtools-rh-che-periodic-prod-2/2774/events_report.txt

Seen again on cluster 2 1x
Seen on prod-preview cluster 2a 2x
https://ci.centos.org/job/devtools-rh-che-periodic-prod-preview-2a/2778/console
https://ci.centos.org/job/devtools-rh-che-periodic-prod-preview-2a/2779/console

[Katka92]

Closing in favor of #1848 as this issue describes the root cause.