Webassembly worker thread will hang if printf is used in threads. #8325

overocean · 2019-03-22T01:01:19Z

In my c program, if printf is used in a thread, the thread won't print anything and won't finish either.

The following is my emcc info.
emcc -v
emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 1.38.27
clang version 6.0.1 (https://github.com/kripken/emscripten-fastcomp-clang.git 5f5d737f1cfd84e1fcdeca85889b9f85ce7f51fb) (https://github.com/kripken/emscripten-fastcomp.git 9a451c3a09af0fb887413bba5e3e7c340d695ad2) (emscripten 1.38.27 : 1.38.27)
Target: x86_64-pc-windows-msvc
Thread model: posix

VirtualTim · 2019-03-22T05:20:48Z

Yeah unfortunately this is a problem/limitation I'm all too familiar with.
So DOM operations can only be done on the main thread. That includes things like printing to the console, and even things like XMLHttpRequests.
So when you print from a thread what actually happens is that this call gets proxied to the main thread.
The issue is that thread.join blocks until the thread operation is complete. But since the thread is waiting on a proxied call to the main thread, and the main thread is waiting on the thread the code deadlocks.

Solutions:

Build with PROXY_TO_PTHREAD. Personally I don't like this one since I have less control on what runs on which threads.
Use thread.detach() instead of thread.join(). Probably fine for your eaxmple, but may not actually work in real situations.
Reactor your code so that the waiting happens on another thread instead of the main thread.
So change this:

void PrintMessage() {
	printf("message from thread\n");
}
	
EMSCRIPTEN_KEEPALIVE void PrintWithThread() {
	printf("start a thread to print a message\n");
	std::thread thr(PrintMessage);
	printf("thread is running\n");
	thr.join();
	printf("thread is done\n");
}

to something like this:

void PrintMessage() {
	printf("message from thread\n");
}

void PrintMessageManager() {
	printf("thread is running\n");
	std::thread thr(PrintMessage);
	printf("thread is done\n");
	thr.join();
}

EMSCRIPTEN_KEEPALIVE void PrintWithThread() {
	printf("start a thread to print a message\n");
	std::thread thr(PrintMessageManager);
	thr.detach();
}

This is the solution I'm using myself.
Note: detached threads currently don't get returned to the thread pool. There is a PR open, but no one has reviewed it yet #8286 :(

TLDR: Don't use thread.join() on the main thread.

sbc100 · 2019-03-22T17:34:38Z

@VirtualTim is right. I would add that problem is nothing to do with printf. The problem is that the thread itself is not even starting.

You can't start a new thread without yielding the event loop since starting a new threads involves, among other things, postMessage() between the main thread and the new worker.

You might also find -s PTHREAD_POOL_SIZE=1 solves your problem by creating the worker up front.

This issue is common enough and confusing enough that perhaps its worth detecting it and erroring in join.

overocean · 2019-03-22T19:45:42Z

Just tried with @VirtualTim's third idea. The PrintMessageManager thread still stuck at the first printf. However, when I run the webassembly module directly in the html, everything works. Even when I changed last line of PrintWithThread to thr.join();.

Here is the new html

<html lang="en" moznomarginboxes mozdisallowselectionprint>
    <head>
        <meta charset="utf-8" />
    </head>
    <body>
        <script>
		self.Module = {
			onRuntimeInitialized: function() {
				Module._PrintWithThread();
			}
		};

		</script>
		<script src="PrintLibrary.js"></script>
    </body>
</html>

VirtualTim · 2019-03-25T03:29:52Z

I think the issue must be some other interaction then. I can run the following with no problems:

#include <thread>
#include <stdio.h>

void PrintMessage() {
	printf("message from thread\n");
}

void PrintMessageManager() {
	printf("thread is running\n");
	std::thread thr(PrintMessage);
	printf("thread is done\n");
	thr.join();
}

void PrintWithThread() {
	printf("start a thread to print a message\n");
	std::thread thr(PrintMessageManager);
	thr.detach();
}

int main(int argc, char** argv) {
	PrintWithThread();
}

Compile with: emcc PrintWithThread.cpp -s USE_PTHREADS=1 -s PTHREAD_POOL_SIZE=2 -o test.html

Maybe there's something going on with module workers and threads? Or maybe lto is doing something weird?
If you can create a more minimal example maybe I could take another look at it.

overocean · 2019-03-25T23:45:03Z

This example works because there is no web worker involved. When we run PrintWithThread with web worker, the issue will surface.

overocean · 2019-03-26T00:43:07Z

Just want to clarify that, in my original example, the web worker was created in a separate JavaScript file.

sbc100 · 2019-03-26T01:43:43Z

Can you explain what you mean by "the web worker was created in a separate JavaScript file." Are you saying that you created the worker yourself and loaded the emscripten generated JS into it? If so, I doubt this will work. Normally emscripten takes care of creating any workers it needs. Also I believe that and a lot of code in emscripten relies on the emscripten-generated JS running in the browser main thread, not in a worker.

overocean · 2019-03-26T17:54:31Z

Sorry about the confusion, let me try to clarify it again :) In the original example, I first compiled the c files to expose the c function PrintWithThread. Second I created a JavaScript file, named PrintWorker, where I call the PrintWithThread function with Emscripten API. Third I created a html file, in its body I have

<script>
    var myWorker = new Worker('PrintWorker.js');
</script>

I found that if I don't use the Worker('PrintWorker.js') in the html, but call the function directly in the html body, the printfs will all work.

<script>
	self.Module = {
		onRuntimeInitialized: function() {
			Module._PrintWithThread();
		}
	};
</script>
<script src="PrintLibrary.js"></script>

" I believe that and a lot of code in emscripten relies on the emscripten-generated JS running in the browser main thread, not in a worker."
Is this specified in any documentation? If so, why does this restriction exist? Without this support, the main thread could become unresponsive (e.g. proxied filesystem operations), so this seems like something Emscripten should support.

…s, see emscripten-core/emscripten#8325

stale · 2020-03-25T22:32:06Z

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.

overocean changed the title ~~Webassebly worker thread will hang if printf is used in threads.~~ Webassembly worker thread will hang if printf is used in threads. Mar 25, 2019

blockspacer pushed a commit to blockspacer/skia-opengl-emscripten that referenced this issue Jul 9, 2019

TODO: Webassembly worker thread will hang if printf is used in thread…

aca7594

…s, see emscripten-core/emscripten#8325

blockspacer mentioned this issue Jul 24, 2019

std::async will hang if no threads are available #8988

Closed

blockspacer mentioned this issue Aug 15, 2019

Thread is not working in WebAssembly export godotengine/godot#12699

Closed

stale bot added the wontfix label Mar 25, 2020

stale bot closed this as completed Apr 1, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Webassembly worker thread will hang if printf is used in threads. #8325

Webassembly worker thread will hang if printf is used in threads. #8325

overocean commented Mar 22, 2019

VirtualTim commented Mar 22, 2019

sbc100 commented Mar 22, 2019

overocean commented Mar 22, 2019

VirtualTim commented Mar 25, 2019

overocean commented Mar 25, 2019

overocean commented Mar 26, 2019

sbc100 commented Mar 26, 2019

overocean commented Mar 26, 2019

stale bot commented Mar 25, 2020

Webassembly worker thread will hang if printf is used in threads. #8325

Webassembly worker thread will hang if printf is used in threads. #8325

Comments

overocean commented Mar 22, 2019

VirtualTim commented Mar 22, 2019

sbc100 commented Mar 22, 2019

overocean commented Mar 22, 2019

VirtualTim commented Mar 25, 2019

overocean commented Mar 25, 2019

overocean commented Mar 26, 2019

sbc100 commented Mar 26, 2019

overocean commented Mar 26, 2019

stale bot commented Mar 25, 2020