[WIP] Add WebWorker API

[bartlomieju]

Ref #1222

ATM I am getting this error during build:

Undefined symbols for architecture x86_64:
  "v8::Shell::WorkerTerminate(v8::FunctionCallbackInfo<v8::Value> const&)", referenced from:
      deno::CreateGlobalTemplate(v8::Isolate*) in libdeno.a(binding.o)
  "v8::Shell::WorkerGetMessage(v8::FunctionCallbackInfo<v8::Value> const&)", referenced from:
      deno::CreateGlobalTemplate(v8::Isolate*) in libdeno.a(binding.o)
  "v8::Shell::WorkerPostMessage(v8::FunctionCallbackInfo<v8::Value> const&)", referenced from:
      deno::CreateGlobalTemplate(v8::Isolate*) in libdeno.a(binding.o)
  "v8::Shell::WorkerNew(v8::FunctionCallbackInfo<v8::Value> const&)", referenced from:
      deno::CreateGlobalTemplate(v8::Isolate*) in libdeno.a(binding.o)

[ry]

Interesting!

v8::Shell functions have to be copied over. Start a new file called libdeno/worker.cc and add them. You can grep for the locations in third_party/v8

> grep -R WorkerTerminate third_party/v8/
third_party/v8//src/d8.cc:void Shell::WorkerTerminate(const v8::FunctionCallbackInfo<v8::Value>& args) {
third_party/v8//src/d8.cc:      FunctionTemplate::New(isolate, WorkerTerminate, Local<Value>(),
third_party/v8//src/d8.h:  static void WorkerTerminate(const v8::FunctionCallbackInfo<v8::Value>& args);

[bartlomieju]

I started with WorkerNew. It required copying a lot of code from v8/src/d8.cc and I can't get rid of those two errors:

In file included from ../../libdeno/binding.cc:13:
../../libdeno/worker.cc:108:42: error: incomplete type 'v8::internal::String' named in nested name specifier
  if (i::FLAG_use_external_strings && i::String::IsAscii(chars, size)) {
                                      ~~~^~~~~~~~
../../third_party/v8/src/globals.h:528:7: note: forward declaration of 'v8::internal::String'
class String;
      ^
In file included from ../../libdeno/binding.cc:13:
../../libdeno/worker.cc:187:14: error: no matching member function for call to 'push_back'
    workers_.push_back(worker);
    ~~~~~~~~~^~~~~~~~~
../../third_party/llvm-build/Release+Asserts/include/c++/v1/vector:707:36: note: candidate function not viable: no known conversion from 'v8::Worker *' to 'const std::__1::__vector_base<deno::Worker *, std::__1::allocator<deno::Worker *> >::value_type' (aka 'deno::Worker *const') for 1st argument
    _LIBCPP_INLINE_VISIBILITY void push_back(const_reference __x);
                                   ^
../../third_party/llvm-build/Release+Asserts/include/c++/v1/vector:710:36: note: candidate function not viable: no known conversion from 'v8::Worker *' to 'std::__1::vector<deno::Worker *, std::__1::allocator<deno::Worker *> >::value_type' (aka 'deno::Worker *') for 1st argument
    _LIBCPP_INLINE_VISIBILITY void push_back(value_type&& __x);

Is that the right direction @ry?

[ry]

Ya - you will have to make some heavy modifications to get it to work. I will give some more detailed comments later

Be sure to update build.gn with the new source file

[bartlomieju]

Awesome!

[kevinkassimo]

Just curious (sorry for not really following the discussion about workers recently in Node), are we going to have separate event loops for each worker thread, or are we going to share a single event loop across all worker threads?

Also I believe we also need to implement messaging together with Workers

[kitsonk]

It should match the specification IMO, which is a seperate event loop.

[ry]

This is obviously a larger project - and I'm glad you're starting it - so far so good. Maybe add tests/worker.js with an extremely simple worker test - and see how far you can get towards running it. Feel free to ping me on gitter or email for help.

[ry]

This works. I would consider using our existing deno::InitializeContext for setting up the global variables... However using a ObjectTemplate is probably a better option - and how D8 is doing it - so don't worry about this nit for now.

[bartlomieju]

@ry I've started with your feedback.

I put some placeholder methods (like Worker::ExecuteInThread) to get compiler happy, but that resulted in:

$ ./tools/test.py
[...]
[----------] 17 tests from LibDenoTest
[ RUN      ] LibDenoTest.InitializesCorrectly

#
# Fatal error in v8::HandleScope::CreateHandle()
# Cannot create a handle without a HandleScope
#

Digging further!

[ry]

@bartlomieju In V8's binding API, basically every function needs to declare a HandleScope for storing local variables. I suggest reading https://v8.dev/docs/embed
For debugging such errors, if a line number isn't printed, you should use a debugger like lldb or gdb. I often debug deno like this:

lldb -- target/debug/deno tests/worker.js
> run
> bt
> up
> up
> l

[ry]

@kevinkassimo Sorry didn't see your message until now. I'd like the different workers to share the same Tokio runtime.

[bartlomieju]

Thanks for tips @ry great read.

There is some progress now, actual worker instantiation is called (run deno tests/test_worker.ts) but some error when reading worker file emerges. I'll get to that tomorrow and keep going.

[ry]

I've started an alternative implementation in rust here #1331

[bartlomieju]

Closing in favor of #1331