[JetBrains] Consider reducing the Gateway integration tests to a single IDE

[felladrin]

Description

JetBrains Integration Tests usually fail as they take more than 1h to complete (here, for example, search for "test timed out after 1h0m0s"). When it happens, we're required to run the tests manually [1].

As we're testing only the Gateway connection with the backend IDE at this moment, we could reduce it to 1 backend IDE (IDEA or Rider), as their codebase (IntelliJ) works the same for all. This would reduce in 87% the time to run Werft Build during PRs.

Related Issues

• [JetBrains] Auto-remove the hold label from PRs that should be auto-merged #15427

[andreafalzetti]

I have been investigating how to improve integration tests for JetBrains in the past days, and one thing I've noticed is that we wait 3 minutes at least, just for the workspace to start, which means 8 IDEs * 3 min = 24 minutes of wait, plus prev env build time etc.. it quickly reaches 1h.

It would be interesting if we could parallelise the tests, however 1 prev env won't be enough (unless we create a bigger VM), or we use 1 prev env for each IDE, but it feels like a waste.

Alternatively, we could test random IDEs each day, e.g. (2 out of 8)

[felladrin]

Agreed.
I like this idea of rotating the IDEs for tests!

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.