Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plugin seems to be slowly filling up process table with <defunct> #9

Open
eleaner opened this issue Dec 31, 2019 · 3 comments
Open
Labels
bug Something isn't working

Comments

@eleaner
Copy link

eleaner commented Dec 31, 2019

Hi,

my VPSes are dying because there are not enough processes.
I traced it back to thousands of processes like that
root 31304 1149 0 12:30 ? 00:00:00 [mfsmount]
the father 1149 is
root 1149 1096 0 12:22 ? 00:00:01 node index.js

I am pretty sure it is the lizardfs plugin

I know too little to understand what's going on. Could you please help?
I could periodically reset my VPSes but it does not sound like the best option.

@zicklag zicklag added the bug Something isn't working label Jan 2, 2020
@zicklag
Copy link
Member

zicklag commented Jan 2, 2020

Oh, that isn't good.

It must be either failing to unmount the volumes or, for some reason, the mfsmount process isn't exiting when it does the unmount ( right here in the code ).

Is there any kind of "abnormal" behavior that you might know of in your particular use-case? Like creating and removing containers with mounts frequently or something like that, or is it mostly long-running containers that don't change much?

Unfortunately I'm not sure exactly when I'll have time to look into this, but I'll try to look at it if I get the chance.

@eleaner
Copy link
Author

eleaner commented Jan 2, 2020

it feels like any time a container/swarm service dies because of whatever reason, the mfsmount process of the used volume stays. Possibly because it is not a child of the dying service but of the plugin that is still alive.
due to some problems, one of my services kept dying every few seconds
all seems to be good when I simply bind container/service to mounted lizard

@zicklag
Copy link
Member

zicklag commented Jan 2, 2020

Yeah, I'll have to add some log messages in there and test killing containers to see what is happening in the plugin when a container dies. It sounds like it should be fixable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants