-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Detached threads don't get disposed of #8201
Comments
Is this a browser OOM? We don't destroy workers, I believe, so that would make sense. If we destroyed them, we'd have issues with their creation (it's async, that's why we have the compile option to spin them up beforehand). So not sure we can do much better here. At minimum we should document this better though. |
Not browser OOM, the program just reaches the TOTAL_MEMORY limit. I don't think pthreads work with ALLOW_MEMORY_GROWTH, but each worker takes a decent amount of memory, so if this is ever supported there will be issues, I'd imagine. I would have thought that if a detached thread gets disposed, either the webworker would get disposed, or it would just go back into the thread pool. As it is worker just keep getting created until For what I was currently working on, I was registering a click callback on the canvas element, and on click create a thread that performs some lengthy synchronous operation (related to the fun I was having with #8117). Obviously joining the thread would block the main thread (or in my case create a deadlock). However if every click creates a web worker that never gets disposed the application is going to quickly run out of memory.
Can you explain further what the issue is here? |
I see. Yeah, we intend to have memory growth working with pthreads, but this is not yet ready.
I meant the same issues at startup - when you do pthread_create, if we need to spin up a worker, that's an async operation, so it must return to the main browser event loop for the thread to actually be created. To work around that, we can preload some workers at startup, and then pthread_create just activates an existing worker, which we can do synchronously. |
Would there be a way to return detached threads to the thread pool when they finish?
and both functions could run on the same thread? |
I believe that's how things currently work, yes, there is a pool that is reused. (Assuming |
Hm, that doesn't seem to be the case in my testing. I'm guessing the example I created above creates threads too fast, but in another project I'm creating a thread per mouse click (using a callback attached to the canvas) and it seems like that created additional threads. I may try making another example to better show this. |
For debugging this, I think the key location is Looks like the |
Thanks for the pointer. I looks like the The initial example I posted still creates threads too fast, and threads aren't removed from the pool (which seems reasonable). However an example like this behaves as expected, with detached threads being returned to the pool.
|
I'm seeing the same issue using std::async - an event being triggered launches a background thread (if one isn't already being run) - what I see in emscripten compiled code is that a new thread appears each time std::async is called, which, as above rapidly causes memory to run out - there will only ever be one async-launched thread running at a time. std::async documentation suggests that the implementation likely uses a threadpool under the hood, but there is no user facing configuration for this in the specification |
Does the fix I mentioned above also fix your std::async issue? |
@VirtualTim - yeah, I'd suggest opening a PR. I'm not familiar enough with that code to say offhand. Can discuss in a PR and hopefully other people can chime in there too. |
Once detached threads are finished their execution they emit the 'exit' command. Instead of a noop they should rejoin the pool. Resolves #8201.
Once detached threads are finished their execution they emit the 'exit' command. Instead of a noop they should rejoin the pool. Resolves emscripten-core#8201.
Once detached threads are finished their execution they emit the 'exit' command. Instead of a noop they should rejoin the pool. Resolves emscripten-core#8201.
According to this "Any allocated resources will be freed once the thread exits." However when a detached thread exits it doesn't get disposed. So not only does that leak memory, it makes maintaining a thread pool not really work.
Example code (swap 100 with something like 10 it you don't want to go out of memory):
Compile like:
emcc test.cpp -std=c++11 -s USE_PTHREADS=1 -s TOTAL_MEMORY=52428800 -o test.html
With the hardocded 100 threads I go out of memory at about 20. Changing the hardcoded value to 10 doesn't go OOM, but the workers still never get freed.
The text was updated successfully, but these errors were encountered: