-
Notifications
You must be signed in to change notification settings - Fork 38.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory leak with CglibAopProxy$ProxyCallbackFilter #26266
Comments
Thanks for the sample. I've reproduced the problem. As far as I can tell, it's caused by your configuration of the test context cache size. Limiting it to one means that many different application contexts are created, and for each context a new CGLib proxy of You said above that configuring the cache size was to "make sure that the grow of memory does not come from cache". It sounds like you'd identified a leak before making this configuration change but, as far as I can tell, this configuration is the cause of the leak. Perhaps you can share a sample that reproduces the problem without tuning the test framework's context cache, or perhaps even reproduces it when running your application's main method rather than its tests? |
Docs says that For sample project I set When I run tests in sample project with I didn't try to reproduce the problem with application's main method because the problem rather is connected somehow with Spring that fails to clean up old context when tests are running. It's only my guess so I'm not sure what causes it. Maybe it is somehow connected with |
This would have been useful information in the original description.
Yes, I believe this is the case. The use of I think we should get someone from the Framework team to look at this. It may be that something isn't behaving as intended or that could be improved in the proxy creation that would enable some reuse. |
Any updates here? Some cglib generated proxy classes keep reference to BeanFactory, so BeanFactory can't be garbage collected, and memory footprint grows and grows. It is the most sensitive in SpringBootTests, where a lot of contexts could be created. |
If the cause of the problem is due to the fact that Spring Security's |
Thanks for bringing this to my attention @sbrannen. It does appear that the class generated by CGLIB is (indirectly) holding a reference to For reference here is a list of objects retained by Here is the shortest GC Root of I don't think Spring Security is the correct place to solve this. Any cleanup that the I think it would be ideal if Spring Framework could remove the reference to the Removing the refererence to Obviously this still means the generated classes are still unnecessarily around, but I'm not sure we can drop the references to those given they are held by the jdk. |
Would it be possible to generate cglib classes using a classloader that is tied to the ApplicationContext? So when the context gets dropped, the loader is no longer referenced from the GC root and the classes are unloaded. If I'm not mistaken, the proxy factory already uses the bean factory's classloader, so it would only be a matter of creating a new classloader inside AbstractContextLoader#prepareContext(ConfigurableApplicationContext, MergedContextConfiguration) with |
@grubeninspekteur Switching the |
Closing this one based on the assumption that #27375 covers all occurrences of this leak (as a regression in the 5.3.x line). Please give the 5.3.10 release a try, it's available in snapshots already and will be officially released tomorrow! |
@jhoeller Unfortunately, it does not appear to have fixed this specific issue. Updating the original sample to use Spring 5.3.10 does not resolve the issue. I've put together a branch named spring-5.3.10 that demonstrates it is not fixed. If you run Can we please reopen the issue? |
with org.springframework:spring-core:5.3.14 a little class like this:
seems to cause a nasty leak like this: when running integration tests, that leads to OutOfMemoryError |
We are having the same issue. We start and close some contexts during the lifetime of our app. If that happens a lot, after a while we get OOM. After manually nulling all CALLBACK_FILTER references in all proxy classes, the bean factory is finally garbage collected.
This would be great. I tried to find a way to do it myself, but didn't find any hook that would allow me that. One dirty way was listing all classes from ClassLoader, filtering those containing This however won't work if there is more than one context and I don't know which proxy classes belong to context that I'm closing. I also tried to collect proxy classes in a custom class loader which I set before the context startup but that didn't really work either. The classes were coming from different class loaders in the hierarchy (some are loaded from parent, cglib proxy classes are magically set to the child loader by reflection.. then they don't match or are not assignable from one to another, and he'll is lose, so this is not a way to go I guess). |
I tried to write a class loader to collect created proxy classes and clean the static field after the context is closed. However, our setup of contexts is not trivial and I was running into many class-to-class problems (classes not assignable to each other, etc.). So I went for the BeanPostProcessor instead, which might not work at all times maybe, but works for us so far.
It collects the bean classes if they are Spring proxy classes and clears the static field when the bean post processor is disposed (context is closed). |
Thanks for sharing the workaround @redhead! |
I have made a pull request for this. |
Will this issue also affect the new Spring Releases, e.g. Spring Boot 3.0.x and the then used Spring Framework 6.0.x ? |
Same here. with spring-boot 3.0.4 and spring-framework 6.0.6 Running a few integration test with @DirtiesContext and then I get an OOM after a few tests. In my case it seems to be the @validated annotation
After running three simple tests with @DirtiesContext I have3 applicationContexts on the heap. After adding this workaround to my test context, everything works again: ((thanks to @redhead, code slightly modified)
|
I've got a promising solution in the works where we reduce the While the |
Can someone please let me know if the issue that is being discussed here is similar to the one I am observing here? |
It seems that there is a memory leak with Spring Boot 2.4.1 on my Debian 10.5 with Java 11.0.1 when using
@PreAuthorize("hasRole('ROLE')")
annotation on controller's endpoint. As you can see in the first image, there is constant grow in old generation memory of G1 garbage collector when running tests. The grow stops at max heap size which is 512 MB. I prepared a sample project where you can reproduce the problem.Memory usage 1
Memory usage 2
After importing the project just run this command:
./gradlew clean test
Tests will fail because they assume that HTTP response will be 200, but it is 401, but it shouldn't matter.
If you remove
@PreAuthorize
from DummyController orprePostEnabled = true
from SecurityConfig then memory usage seems to be ok.Memory usage 3
Memory usage 4
The project uses dummy controller with one endpoint which has
@PreAuthorize("hasRole('ROLE')")
annotation. It has also basicSecurityConfig
to enable@PreAuthorize
annotation. Controller uses some dummy services to make sure that for each test Spring will recreate the context. I also setspring.test.context.cache.maxSize=1
in build.gradle to make sure that the grow of memory does not come from cache.If you have any more questions about my configuration or anything else, let me know.
The text was updated successfully, but these errors were encountered: