It's NOT memory leak

[jiedo]

hi，
Golang plugin can not be reload/unload right now. when building plugins in a random path. golang load them all as different plugins.
So I think it's not memory leak, just loading more and more files .

[tmaiaroto]

Good to know! Your comment saved me some time. I'm trying to think of a system that would allow for a circuit breaker functionality (taking it a step farther and pulling the files from S3 down to local disk first - that way updates and application composition is done via .so file delivery outside the container). So having multiple versions of a plugin all loaded at the same time isn't necessarily bad...Except for when you have enough that you're confident enough to unload some old ones. Definitely would complete the hot swap picture.

ie.
myplugin.1.so
myplugin.2.so
myplugin.3.so <-- failed, go back to using 2

...but we know 2 works so let's unload 1. That would be nice. Super super super powerful here for microservices and such. For me, probably one of the biggest wins in cloud computing development (developing apps for) that I've ever seen. Can't wait for this to mature!

[mikelnrd]

From my own exploration into plugins I've found that each .so plugin binary contains a copy of the go runtime and a copy each library used by the plugin. So each plugin, however small the api it adds, actually takes up a lot of memory when loaded. Having n plugins loaded makes this n times worse of course.

For anyone interested in this 'plugin memory issue' I've found that:

• There's currently an issue tracking removing the redundant copy of the runtime: cmd/go: drop runtime from plugins golang/go#17150

• And also this issue suggests that using the -linkshared buildmode flag could remove the libraries from the .so file.
  See here: cmd/go: -buildmode=plugin -linkshared -> build failure golang/go#18671
  (I assume this means that only one copy of the library would need to be loaded into memory instead of one per plugin using it as is currently the case. This would drastically reduce the memory problem with loading many plugins at once.)

[l0k18]

An interim measure would add restarting using a second shell launcher and a trigger to restart the plugin running app. Like reload on a browser really. With an input logger and state persistence it would not be that big a deal, but one would have to think about the lack of direct channels between processes versus goroutines. Plugins themselves for this, I guess...