When vscode opens a Java project for about 2 hours, the suggestion function will fail

[nullgit]

When I enable the Spring Boot Tools extension and opens a Java project for about 2 hours, the suggestion function will fail. It will always show loading.... There is no such problem in the first two hours. And when I disable this extension or use v1.31.0, it will not have this bug.

• Spring Boot Tools: v1.32.0
• VSCode: 1.66
• Java Runtime: jdk-17.0.2.8-hotspot
• OS: Windows11

[martinlippert]

Sorry for the late reaction here. We will take a look at this, of course.

Can you configure the Spring Tools via the preferences in VSCode to produce a log file somewhere, so that you could attach it here? Would be interesting to see what is going on on the language server side while this "hangs" happens.

[nullgit]

I configured spring-boot.ls.logfile and opened a spring boot program in A:/computer/spring/spring/
The code suggestion worked will at 09:05, and it failed before 10:12:58. I did not do anything else during this period
log.txt
.

[martinlippert]

@nullgit Thanks for attaching the log file. I took a deeper look as well as tried to reproduce this on my end without success so far. I am working on adding additional log output here to investigate this deeper. Once I have a CI build of our extension that includes the additional log output, it would be great if you could install that version of the extension in your VSCode and try to reproduce the behavior - and attach the log output again here. It doesn't solve the issue yet, but I hope to - at least - find a few more useful hints in the enhanced log output then.

I will let you know once the new CI version is around. Thanks for helping us to debug this issue.

[martinlippert]

The CI build of the VSCode extension is available from here: https://dist.springsource.com/snapshot/STS4/nightly-distributions.html

Please download the vscode-spring-boot-1.33.0-<timestamp>.vsix file from the bottom section of that page, go to VSCode, open the Extensions section, click on the three dots on the top right corner of the extensions view to show more menu options, and select "Install from VSIX...", then select the downloaded VSIX file, and restart VSCode.

Would be great to get some additional log output data from this slightly enhanced version. Many many many thanks upfront!!!

[martinlippert]

One more thing: Can you capture one or a few thread dumps of the Spring Boot Language Server process (using jps and jstack) while the content-assist is not working anymore? Using jps you should see a process called BootLanguageServerBootApp - which is the language server process for which the stack trace would be interesting to see.

[nullgit]

It failed before 08:36.
log.txt
jstack.txt
You said you failed to reproduce the issue, could it be because of a problem with my VSCode and not the plugin?

[martinlippert]

@nullgit Thanks for the additional logs and stack traces, that is extremely interesting. Can you go to the preferences and disable the Live Information - Automated Process Tracking Enabled, and then restart your VSCode to see if that makes a difference?

In case I am thinking into the right direction, this should workaround the issue - and would indicate how to fix the underlying problem.

[martinlippert]

@nullgit Please ignore my previous comment, looks like this setting doesn't really workaround the issue as I expected it to do. So let me work on this and update the issue once I have a new CI build of the VSIX file for you to try.

[martinlippert]

@nullgit There is a new CI build available from: https://dist.springsource.com/snapshot/STS4/nightly-distributions.html

Would be awesome if you could give that a try as well and report back how things are going (and submit log files and stack traces again, if things go wrong).

Many many thanks for helping us investigating this, much appreciated!!!

[xyzCh]

My problem was solved, many thanks for @martinlippert @nullgit

[nullgit]

@martinlippert Hello, after installing the new plug-in, it seems that this problem has been solved. Do you still need log? Now I can happily write Spring Boot program on VSCode again. Thank you very much!!

[martinlippert]

@xyzCh @nullgit Thanks for the feedback and for letting us know that this is working now with the latest fixes. Great news!!!

@nullgit It would indeed by interesting to get a stack trace (logs are not necessary, I think) from the language server process on your end after you had it running for a longer while (several hours probably). So if possible, that would be much appreciated.

[nullgit]

@martinlippert This is a new stack trace.
jstack.txt

[martinlippert]

@nullgit Thanks for attaching the log, much appreciated.

[martinlippert]

Some more details from investigating what is going on here:

It looks like the attempt to update the internal status of tracked processes hangs. The stack trace for those update threads all look like this:

"pool-6-thread-4" #31 prio=5 os_prio=0 cpu=62.50ms elapsed=49826.91s tid=0x0000021fe7e94ce0 nid=0x6300 runnable  [0x000000c97ccfe000]
   java.lang.Thread.State: RUNNABLE
	at sun.tools.attach.VirtualMachineImpl.connectPipe(jdk.attach@17/Native Method)
	at sun.tools.attach.VirtualMachineImpl.execute(jdk.attach@17/VirtualMachineImpl.java:110)
	at sun.tools.attach.HotSpotVirtualMachine.executeCommand(jdk.attach@17/HotSpotVirtualMachine.java:310)
	at sun.tools.attach.HotSpotVirtualMachine.getSystemProperties(jdk.attach@17/HotSpotVirtualMachine.java:191)
	at org.springframework.ide.vscode.boot.java.livehover.v2.SpringProcessDescriptor.shouldIgnore(SpringProcessDescriptor.java:156)
	at org.springframework.ide.vscode.boot.java.livehover.v2.SpringProcessDescriptor.checkStatus(SpringProcessDescriptor.java:124)
	at org.springframework.ide.vscode.boot.java.livehover.v2.SpringProcessDescriptor.lambda$updateStatus$0(SpringProcessDescriptor.java:115)
	at org.springframework.ide.vscode.boot.java.livehover.v2.SpringProcessDescriptor$$Lambda$443/0x0000000800f4b500.get(Unknown Source)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(java.base@17/CompletableFuture.java:1768)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17/ThreadPoolExecutor.java:1136)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17/ThreadPoolExecutor.java:635)
	at java.lang.Thread.run(java.base@17/Thread.java:833)

In the past, the thread pool that was responsible for executing this was the common fork/join thread pool that is used by all the CompletableFutures if you don't pass your on executor service to it. After a while, all the available threads on that thread pool were running (actually hanging with the stack trace above). As a result, every other place on the language server that wanted to execute something using that thread pool was waiting forever. The language server was using that thread pool for many things, among them the computation of the content-assist proposals. Therefore the content-assist newer really showed anything anymore.

The first step here was to differentiate the different thread pools used by the different parts of the language server. Now the process tracker uses its own thread pool for updating the status of running spring processes. In the case above, this thread pool is still filling up with hanging threads (which is bad and probably breaks the live hover feature), but it doesn't affect all the other parts of the language server anymore.

Next step here is to improve the communication with the external processes that are being tracked in order to avoid this handing threads altogether. But that is probably something that will not be done for the upcoming 4.14.1.RELEASE, but the one after that.

[nullgit]

@martinlippert Thanks for letting me know why there is this bug. For now, this problem is solved on the surface.

[martinlippert]

Thanks @nullgit for verifying and closing the issue. I created an additional one to continue the work on the underlying problem (#771).