Performance of Reconciling Spring Java

[fenjen]

I have several projects in my Eclipse of which some are quite large. For one of the projects, Spring Boot will do "Reconciling Spring Java" for almost two minutes (1:52). This happens on each Eclipse startup, and when I change source code (like adding a comment to the end of a code line). Not only is it super slow, it also takes a lot of system resources.

1. Spring Boot should ignore all non Spring Boot projects.
2. Reconciling Spring Java should cache the results.

I haven't found any option to exclude my big project for Reconciling Spring Java - I tried adding the project name to Settings → Spring → Boot Dashboard → Exclude Projects (I don't think it's the correct place, but I haven't found anything else).

[BoykoAlex]

Is this large project a spring-boot project?

You can also disbale spring-java reconciling but for all projects as a workaround...

[fenjen]

No, it's not a Spring Boot project. I disabled spring-java reconciling globally, thanks!

[martinlippert]

Hey @fenjen, I would be interested in some additional details. Can you try something for me? I would like to see how much time and resources the symbol scanning consumes initially in your case if there are no symbols in the cache.

Can you:

• disable reconciling
• enable the console logging for the Spring Boot language server (Preferences -> Language Servers -> Spring Language Server) - that way we can extract some information from the log output
• shutdown the IDE
• delete the symbol cache files (~/.sts4/.symbolCache)
• restart the IDE
• keep an eye on resource consumption

In the language server log output, there should be messages showing up talking about scanning symbols for Java files and how much time it took. Would be interesting to see those lines (if possible).

Since we don't have progress bar support for this symbol scanning in place, this stuff will not show up in the progress bar on the UI, but you will see the messages showing up in the log view and you should see some CPU and memory consumption on the system (from the language server process, not the IDE process). Would be interesting to hear your subjective impressions on this period, compared to the reconciling observations.

Many many thanks upfront!!!

[fenjen]

I disabled reconciling, enabled looging, ended Eclipse, removed all files in symbolCache and started Eclipse. But starting Eclipse always takes 100% of CPU and takes some time, so it's generally hard to tell how much slower it became with Spring Tools.

The relevant part in Eclipse log:

!ENTRY org.springframework.tooling.ls.eclipse.commons 1 0 2023-07-10 14:31:28.065
!MESSAGE Command list starting LS: org.springframework.tooling.boot.ls

[...]

!ENTRY org.springframework.tooling.boot.ls 1 0 2023-07-10 14:32:04.131
!MESSAGE Boot project ADDED: eu...

!ENTRY org.springframework.tooling.boot.ls 1 0 2023-07-10 14:32:04.136
!MESSAGE Starting Boot LS...

!ENTRY org.springframework.tooling.boot.ls 1 0 2023-07-10 14:32:04.144
!MESSAGE Started Boot LS process

!ENTRY org.springframework.tooling.boot.ls 1 0 2023-07-10 14:32:04.149
!MESSAGE Boot project ADDED: ...

!ENTRY org.springframework.tooling.boot.ls 1 0 2023-07-10 14:32:04.153
!MESSAGE Boot project ADDED: ...

The symbolCache file of the big project is 680K. What I know is that reconciling made everything extremely slow.

[martinlippert]

@fenjen Thanks for giving this a try. I am looking at lines in the log that looks like this:

08:33:29.358 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files for symbols for project: spring-petclinic took ms: 2044

Would be interesting to see the numbers for your big project there (after deleting the files in symbols cache, of course)

[fenjen]

Running find . -name '*log*' -exec zgrep SpringIndexerJava {} \; in my Eclipse workspace doesn't yield anything.

[martinlippert]

Hey @fenjen, I am looking for the console log output of the language server process, which you can enable in the preferences under Preferences -> Language Servers -> Spring Language Server, there is a checkbox to stream the logs from the language server process to a console tab. Quite annoying for everyday use, only meant for debugging situations like this. Once you enabled this, you need to restart the IDE in order to take effect, so best to switch that on, then stop the IDE, then clear the symbols cache, then start the IDE again. You should see the new tab in the console view appearing automatically.

[fenjen]

Oh, I'm sorry. I thought [x] Logging to Console would log to Eclipse Error Log. I renamed the project names here to project1, project2 and project3:

13:21:13.546 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files for symbols for project: project1 - no. of files: 29
13:21:13.551 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files, AST parse, pass 1 for files: 29
13:21:16.785 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files done, number of symbols created: 23
13:21:16.839 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files for symbols for project: project1 took ms: 3293
13:21:16.882 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files for symbols for project: project2 - no. of files: 6
13:21:16.888 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files, AST parse, pass 1 for files: 6
13:21:17.170 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files done, number of symbols created: 2
13:21:17.182 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files for symbols for project: project2 took ms: 300
13:21:17.495 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files for symbols for project: project3 - no. of files: 2409
13:21:17.512 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files, AST parse, pass 1 for files: 2409
13:21:41.836 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files done, number of symbols created: 532
13:21:42.099 [pool-4-thread-1] INFO  o.s.i.v.b.j.utils.SpringIndexerJava - scan java files for symbols for project: project3 took ms: 24604

[martinlippert]

The majority of the work has been done in the https://github.com/spring-projects/sts4/commits/jdt-diagnostics-via-symbols branch and is scheduled to ship as part of the 4.20.0 release, scheduled for end of September 2023. Therefore closing this item here for now.

[martinlippert]

Early versions of this will be available once the 4.19.1 release is out and the work is being merged into main. Once that is done, this will be included in the CI builds for Eclipse and VSCode.

[iloveuaa]

i want change to idea

[martinlippert]

i want change to idea

If you mean IntelliJ IDEA, it provides awesome support for working with Spring (Boot) applications in the IntelliJ Ultimate Edition, so feel free to jump to that IDE to work on your Spring applications, if you would prefer that environment.

If you are using the Spring Tools 4 for Eclipse or VSCode and you are observing issues, please provide more details what exactly the issues are, so that we can work on them and ship an improved version to you as soon as possible.

In case you are observing issues, especially performance-related trouble while having the Java source reconciling activated, please de-activate this (as described above) for the moment. A vastly improved version of the source reconciling is in the works and will ship soon.