python: circular reference in python objects causes future leak #2566
Comments
SteVwonder
added a commit
to SteVwonder/flux-core
that referenced
this issue
Dec 10, 2019
When the user of Python objects inheriting from `Wrapper` attempt to access an unkonwn method (i.e., a method that is not manually written and included in the bindings)will automatically search the cffi auto-generated bindings for functions with a matching name. Once a match is found, the function is bound as a method to the object to effectively cache the search and make future accesses quicker. This bound method creates a circular reference between the `Wrapper` and it's method. In the case of `Future`s, this resulted in a leak and the piling up of open futures, as documented in flux-framework#2566. To avoid this circular reference, use the `weakref` library to create a weak reference to the object when constructing the bound method. When the object leaves scope, it is now immediately destructed. Remove the no longer necessary calls to `future._clear()` Closes flux-framework#2566.
SteVwonder
added a commit
to SteVwonder/flux-core
that referenced
this issue
Dec 10, 2019
When the user of Python objects inheriting from `Wrapper` attempt to access an unkonwn method (i.e., a method that is not manually written and included in the bindings)will automatically search the cffi auto-generated bindings for functions with a matching name. Once a match is found, the function is bound as a method to the object to effectively cache the search and make future accesses quicker. This bound method creates a circular reference between the `Wrapper` and it's method. In the case of `Future`s, this resulted in a leak and the piling up of open futures, as documented in flux-framework#2566. To avoid this circular reference, use the `weakref` library to create a weak reference to the object when constructing the bound method. When the object leaves scope, it is now immediately destructed. Remove the no longer necessary calls to `future._clear()` Closes flux-framework#2566.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Just creating an issue as a reminder that we still need to fix the circular reference problem found in #2549 which causes python futures and rpc objects to fail to get automatically garbage collected.
I ran across this in some code that called an rpc in a loop. In order to avoid the assertion failure added in #2563, an explicit call to
f.pimpl._clear()needed to be added.The text was updated successfully, but these errors were encountered: