[jb] configure vmoptions for intellij backend server #10175

yaohui-wyh · 2022-05-22T04:15:20Z

Description

Specify JVM max heap usage with -XX:MaxRAMPercentage (follows jb: java perf monitoring #9796 ) for better resource usage in a containerized env
Move all vmoptions customizations into /ide-desktop/status and make it easier for future vmoptions optimizations

Related Issue(s)

Related #8704

How to test

Release Notes

- Allow customize VMOptions for JetBrains backend server, by setting `INTELLIJ_VMOPTIONS` (also GoLand/PyCharm/PhpStorm) environment variable

Documentation

akosyakov · 2022-05-22T19:33:35Z

Maybe we should look into more generic solution to allow customisations for different JVMs than introducing an env var for everything? Maybe we could use one env var instead like [option]_VM_OPTIONS? [option]_XMX makes sense to have separate only because it treated specially by launcher script. Could you have look at launcher script maybe there is already such env var and rather we are missing better docs?

components/ide/jetbrains/image/status/main.go

yaohui-wyh · 2022-05-23T17:32:33Z

Maybe we should look into more generic solution to allow customisations for different JVMs than introducing an env var for everything? Maybe we could use one env var instead like [option]_VM_OPTIONS? [option]_XMX makes sense to have separate only because it treated specially by launcher script. Could you have look at launcher script maybe there is already such env var and rather we are missing better docs?

I looked into remote-dev-server.sh, launcher.sh, idea.sh, and extract the logic involving how VMOptions is passed to the Java process (intellij server) across all the wrapper scripts:

# 1. remote-dev-server.sh
launcher.sh "idea.sh" "IU" "IDEA" "idea" "-Xmx750m" "$@"

# 2. launcher.sh
# note that "idea" (idea64.vmoptions) and "-Xmx750m" below are passed by remote-dev-server.sh
TEMP_VM_OPTIONS=$IJ_HOST_SYSTEM_DIR/pid.$$.temp.vmoptions
cp idea64.vmoptions $TEMP_VM_OPTIONS
sed "s/-Xmx750m/-Xmx2048m/g" ... > $TEMP_VM_OPTIONS

# export for idea.sh
IDEA_VM_OPTIONS=$TEMP_VM_OPTIONS 

# 3. idea.sh
VM_OPTIONS_FILE=$IDEA_VM_OPTIONS                       
VM_OPTIONS=`cat $VM_OPTIONS_FILE`
java ${VM_OPTIONS} com.intellij.idea.Main ...

And here are some thoughts:

User customization to JVM arguments should be written to idea64.vmoptions (for desktop IntelliJ IDEs, customization also happens to idea64.vmoptions or similar vmoptions files such as from Toolbox).
Besides the -Xmx, all other JVM arguments are treated equal for remote server. And if user has modified the -Xmx, launcher.sh does nothing.

Suggestions:

If we want to modify the default JVM args in case of performance tuning (or to allow Gitpod users overwriting the default value), better modify the idea64.vmoptions than launcher.sh (thus configureXmx needs update).
Provide a single [alias]_VMOPTIONS env var which allows a list of arguments separated by space (which is nature to Java Options), and no separated [alias]_XMX

jetbrains:
  // default for all
  vmoptions: "-XX:MaxRAMPercentage=50.0 -Dgtw.disable.exit.dialog=true" 
  intellij:
    // override or prepend -Xmx for intellij
    vmoptions: "-Xmx6g" 
  goland:
    // override or prepend -Xmx for goland
    vmoptions: "-Xmx4096m"

akosyakov · 2022-05-24T02:30:50Z

It sounds good

yaohui-wyh · 2022-05-24T12:51:10Z

I've updated the changes. PTAL again @akosyakov 🙏

During the VMOptions investigation I wrote a post about the bootstrap process of a IntelliJ IDE workspace:
https://spurious-amount-763.notion.site/Gitpod-workspace-with-JetBrains-Gateway-a78f71c39c364ab8ba90ed3ac5ec9643

akosyakov · 2022-05-24T13:32:49Z

@iQQBot do you think you can help @yaohui-wyh how to setup self-hosted solution to test PRs?

iQQBot · 2022-05-24T13:40:53Z

@iQQBot do you think you can help @yaohui-wyh how to setup self-hosted solution to test PRs?

I can help if you need @yaohui-wyh

…mization

yaohui-wyh · 2022-06-10T10:56:33Z

I test the PR on Gitpod SaaS. PTAL @akosyakov 🙏

The test plan is to use a Gitpod workspace (with intellij backend server) to build the status binary and replace the /ide-desktop/status, and kill the status process from the workspace (supervisor would restart the status process), then the main logic can be tested in-place

Choose IntelliJ as the default editor https://gitpod.io/preferences
Add envvar INTELLIJ_VMOPTIONS, such as -XX:MinRAMPercentage=25 -Xmx3g
Start Gitpod workspace from this PR: https://gitpod.io/#https://github.com/gitpod-io/gitpod/pull/10175
Launch a new Terminal and build the status binary: cd /workspace/gitpod/components/ide/jetbrains/image/status && go build (maybe better with leeway build ... <some args>)
Replace the status binary: rm /ide-desktop/status; cp status /ide-desktop/status
Check the IDE server Java Process VMOptions: jps -ml | grep intellij; jinfo <pid> > ~/jinfo1.txt
Kill the status process: ps aux | grep status; kill -9 <pid>
(Supervisor restart status process), and check step7 again: jps -ml | grep intellij; jinfo <pid> > ~/jinfo2.txt

Expected behavior:

jinfo2.txt (Step7) should contain VMOptions set by envvar (Step2), and no other VMOptions are touched
Connect to Gitpod workspace via Gateway, check the "backend control center" from JBThinClient, and the Maximum heap size should be 3GiB

akosyakov · 2022-06-13T07:34:43Z

components/ide/jetbrains/image/status/main.go

+func updateVMOptions(alias string, content string) string {
+	// inspired by how intellij platform merge the VMOptions
+	// https://github.com/JetBrains/intellij-community/blob/master/platform/platform-impl/src/com/intellij/openapi/application/ConfigImportHelper.java#L1115
+	filterFunc := func(l, r string) bool {


why it is not part of deduplicateVMOption?

To make it easier to have finer control over the type of VMOptions to dedup. Currently, only memory-related VMOptions (-Xmx / -Xms, etc.) are handled, however, if we need to deal with VMOptions such as GC-related by another feature toggle (e.g. a separate env var). Putting all filter logic into deduplicateVMOption is a bit messy.

It's similar to updateVMOptions (https://github.com/JetBrains/intellij-community/blob/master/platform/platform-impl/src/com/intellij/openapi/application/ConfigImportHelper.java#L1068) from the Intellij codebase

akosyakov · 2022-06-13T07:37:16Z

/werft run

👍 started the job as gitpod-build-local-fork.1
(with .werft/ from main)

akosyakov

/lgtm

akosyakov · 2022-06-13T08:17:04Z

@yaohui-wyh Do you want to add it to .gitpod.yml?

yaohui-wyh · 2022-06-13T08:31:54Z

@yaohui-wyh Do you want to add it to .gitpod.yml?

Hi Anton! Do you mean adding to .gitpod.yml user document how to customize VMOptions for Gateway? Or modify the .gitpod.yml of some JB Gateway demo projects such as gitpod-io/spring-petclinic?

akosyakov · 2022-06-13T09:11:45Z

Hi Anton! Do you mean adding to .gitpod.yml user document how to customize VMOptions for Gateway? Or modify the .gitpod.yml of some JB Gateway demo projects such as gitpod-io/spring-petclinic?

I mean first adding it to gitpod.yml schema and actually making use of it. Then everything else.

yaohui-wyh · 2022-06-13T10:20:17Z

I mean first adding it to gitpod.yml schema and actually making use of it. Then everything else.

Thanks, I get it. Yeah I would add the vmoptions config to gitpod-config-type (according to this reply #10175 (comment))

One thing to mention, I think users can already make use of it now by specifying variables for a project. However, add the option to project-wide .gitpod.yml config does give users more flexibility.

akosyakov · 2022-06-13T10:58:58Z

One thing to mention, I think users can already make use of it now by specifying variables for a project. However, add the option to project-wide .gitpod.yml config does give users more flexibility.

.gitpod.yml is to share across all users of the project, it can be overridden by user env vars

yaohui-wyh requested a review from a team May 22, 2022 04:15

roboquat added release-note size/M labels May 22, 2022

iQQBot added community-contribution cla: accepted ✔️ labels May 22, 2022

akosyakov added the team: IDE label May 22, 2022

akosyakov reviewed May 22, 2022

View reviewed changes

components/ide/jetbrains/image/status/main.go Show resolved Hide resolved

yaohui-wyh force-pushed the local branch from 4998ed7 to 568fe7f Compare May 24, 2022 08:39

roboquat added size/L and removed size/M labels May 24, 2022

[jb] configure vmoptions for intellij backend server performance opti…

898a3f1

…mization

yaohui-wyh force-pushed the local branch from ef634a5 to 898a3f1 Compare June 10, 2022 09:42

akosyakov reviewed Jun 13, 2022

View reviewed changes

akosyakov approved these changes Jun 13, 2022

View reviewed changes

roboquat merged commit 49224d1 into gitpod-io:main Jun 13, 2022

roboquat added deployed: IDE IDE change is running in production deployed Change is completely running in production labels Jun 14, 2022

yaohui-wyh mentioned this pull request Jun 20, 2022

[jb] enable vmoptions config in .gitpod.yml #10768

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[jb] configure vmoptions for intellij backend server #10175

[jb] configure vmoptions for intellij backend server #10175

yaohui-wyh commented May 22, 2022 •

edited

Loading

akosyakov commented May 22, 2022 •

edited

Loading

yaohui-wyh commented May 23, 2022

akosyakov commented May 24, 2022

yaohui-wyh commented May 24, 2022 •

edited by werft-gitpod-dev-com bot

Loading

akosyakov commented May 24, 2022

iQQBot commented May 24, 2022

yaohui-wyh commented Jun 10, 2022 •

edited by werft-gitpod-dev-com bot

Loading

akosyakov Jun 13, 2022

yaohui-wyh Jun 13, 2022

akosyakov commented Jun 13, 2022 •

edited by werft-gitpod-dev-com bot

Loading

akosyakov left a comment

akosyakov commented Jun 13, 2022

yaohui-wyh commented Jun 13, 2022

akosyakov commented Jun 13, 2022

yaohui-wyh commented Jun 13, 2022

akosyakov commented Jun 13, 2022

[jb] configure vmoptions for intellij backend server #10175

[jb] configure vmoptions for intellij backend server #10175

Conversation

yaohui-wyh commented May 22, 2022 • edited Loading

Description

Related Issue(s)

How to test

Release Notes

Documentation

akosyakov commented May 22, 2022 • edited Loading

yaohui-wyh commented May 23, 2022

akosyakov commented May 24, 2022

yaohui-wyh commented May 24, 2022 • edited by werft-gitpod-dev-com bot Loading

akosyakov commented May 24, 2022

iQQBot commented May 24, 2022

yaohui-wyh commented Jun 10, 2022 • edited by werft-gitpod-dev-com bot Loading

akosyakov Jun 13, 2022

Choose a reason for hiding this comment

yaohui-wyh Jun 13, 2022

Choose a reason for hiding this comment

akosyakov commented Jun 13, 2022 • edited by werft-gitpod-dev-com bot Loading

akosyakov left a comment

Choose a reason for hiding this comment

akosyakov commented Jun 13, 2022

yaohui-wyh commented Jun 13, 2022

akosyakov commented Jun 13, 2022

yaohui-wyh commented Jun 13, 2022

akosyakov commented Jun 13, 2022

yaohui-wyh commented May 22, 2022 •

edited

Loading

akosyakov commented May 22, 2022 •

edited

Loading

yaohui-wyh commented May 24, 2022 •

edited by werft-gitpod-dev-com bot

Loading

yaohui-wyh commented Jun 10, 2022 •

edited by werft-gitpod-dev-com bot

Loading

akosyakov commented Jun 13, 2022 •

edited by werft-gitpod-dev-com bot

Loading