Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP V8 API usage in Node.js #26929

Closed
wants to merge 2 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
241 changes: 241 additions & 0 deletions doc/guides/V8-api-for-node.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
## V8 API usage for Node.js

v8 docs, not particularly useful:
- https://v8.dev/docs

v8 docs, generated from master:
- https://v8.paulfryzel.com/docs/master/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these are out of date? Also Firefox warns that it's using an expired certificate.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's documenting V8 6.9 so yeah, pretty stale.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we fork this https://github.com/paulfryzel/v8-doxygen, and redeploy it to GH pages or Nodejs.org

Copy link
Member

@gengjiawen gengjiawen May 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@gengjiawen gengjiawen May 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Online preview by gitlab: https://gengjiawen.gitlab.io/v8-docs/.


v8 code search:
- https://cs.chromium.org/chromium/src/v8/include/v8.h

Some API info at https://v8.dev/docs/embed

- An isolate is a VM instance with its own heap.
Node has one isolate per Worker.
Can get from `Local<Object>->GetIsolate()` (but this is slow and discouraged
by the V8 people), `Local<Context>->GetIsolate()`, or more typically from
`(node::Environment*env)->isolate()` or `FunctionCallbackInfo<Value>
args.GetIsolate()`.

- `Local<...>`: A local handle is a pointer to an object. All V8 objects are
accessed using handles, they are necessary because of the way the V8 garbage
collector works. Local handles can only be allocated on the stack, in a
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extra space before Local.

scope, not with new, and will be deleted as stack unwinds, usually with a
scope.
- We don't explicitly create HandleScopes very often, because all functions
that take `FunctionCallbackInfo` (or similar) already come with one
built-in.
- Allocating a `Local<>` outside of a scope can lead to difficult to track
down memory leaks, so the top-level event loop instantiates a
`SealHandleScope`. Basically, the idea is to make sure that the code inside
the SealHandleScope always needs to explicitly open a HandleScope if it does
use handles, so that they will be deleted when the stack unwinds.
- Persistent handles last past C++ functions.
- PersistentBase::SetWeak trigger a callback from the garbage collector when
the only references to an object are from weak persistent handles.
- A `v8::Global<>` (alias of a `UniquePersistent<>`) handle relies on C++
constructors and destructors to manage the lifetime of the underlying
object. We don’t use `UniquePersistent` in Node.js.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd drop the notes about UniquePersistent, that's its old name and will probably go away entirely someday.

- A Persistent<SomeType> can be constructed with its constructor, but must be
explicitly cleared with Persistent::Reset. This is a good example of how
usage is shifting – we introduced `node::Persistent<>`, which automatically
resets in the destructor, but which becomes unnecessary through
`v8::Global<>`, which also does that and additionally supports move
semantics.
- Eternal is a persistent handle for JavaScript objects that are expected to
never be deleted for the lifetime of the isolate. It is cheaper to use because
it relieves the garbage collector from determining the liveness of that
object.
- A handle scope can be thought of as a container for any number of handles.
When you've finished with your handles, instead of deleting each one
individually you can simply delete their scope.
- EscapableHandleScope: Locals can't be returned, their scope will delete them,
so you need an escable scope, and to `return scope.Escape(...the local)`. It
scopes the locals into the enclosing scope and returns a local for that scope.

Local, MaybeLocal, Maybe, Value, oh my...

https://groups.google.com/forum/#!topic/v8-users/gQVpp1HmbqM

- MaybeLocal may be "empty", basically not contain a pointer of its type. See
`class MaybeLocal`, has some useful notes on why, but basically its returned
when there is an exception pending in V8.
- If you know that the MaybeLocal has a value, then call ToLocalChecked() and
Node.js will abort in `node_errors.cc:OnFatalError()`.
- Otherwise, you have to call `bool ToLocal(Local<S>* out)`, and check the
return value to see if there was a value. Or call IsEmpty() to check.

- Maybe is similar, but doesn't hold a Local, just a value of T. "Just" means it
"just has a value", a bizarrely named Haskellism :-(. It has a To() and
ToChecked() similar to MaybeLocal.
- A common Node.js idiom is to make a seemingly side-effect free call to
`.FromJust()` after `->Set()`, which will crash Node.js if the Set failed.
FromJust is also() commonly called after getting a Maybe<> of a concrete data
type from a Local<Value>. It will crash if the conversion fails!

int32_t v = (Local<Value>)->Int32Value(env->context()).FromJust()

- `Maybe::Check()` is is an equivalent short-hand to FromJust() which V8
describes (in header comments) as to be used where the actual value of the
Maybe is not needed like Object::Set. It returns void and the name makes it
clearer that it can fail.

target->Set(env->context(), class_name, function).Check();

- As<T> always returns a value, though it does not perform typechecking on its
own, so the type should be checked. It is basically an unchecked
reinterpret_cast in release builds (it typechecks and aborts in debug builds).
- Boolean becomes 1/0 as int, "true"/"false" as strings, etc.
- Numbers become false as Boolean (for any value), -3 casts to String "-3"
- Functions become numerically zero, and "function () { const hello=0; }" as a
String
Copy link
Member

@addaleax addaleax Mar 26, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using .As<> to convert e.g. functions to strings isn’t a valid thing to do, and I don’t think we should document it – that should crash in debug mode?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, this I didn't understand. A Function becoming the source code of it's definition doesn't seem to be at all similar to a C-style untype-checked cast, unless a Function object is somehow derived from String. It appeared to me, while experimenting, to be doing the equivalent of String(a_function) -- so, not a cast.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sam-github Could you maybe give an example of how you came to that conclusion? It might help to know that. (If you used String::Utf8Value: That just “happens” to work out because the constructor of that class takes a Local<Value>, and explicitly converts the argument to string if necessary)

Copy link
Member

@devsnek devsnek Apr 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to clarify, As should only be known when you are clarifying the handle type, not to perform a conversion. like Anna said, using As to perform an invalid cast will even abort in debug mode.

Boolean to number would be b->ToNumber(), number to string would be b->ToString(), etc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sam, can you remove these three bullet points?

- Examples:
CHECK(args[0]->IsInt32()); // Optional, but typical
Local<Int32> l = args[0].As<Int32>(); int32_t v = li->Value();

CHECK(args[0]->IsString()); // Optional, but typical
const node::Utf8Value v(env->isolate(), args[0]); const char* s = *s;

- To<T> will convert values in fairly typical js way:
... never seems to be used by Node.js?
AFAICT, is identical to the As<> route, except for Boolean, which is always
false with As<T>(), but is "expected" with ToT().
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m not sure which To you’re referring to here, but if you’re talking about ToObject(), ToBoolean(), etc., then that’s different from .As<>().As<> performs a pointer cast without checking, but the To... methods perform the JS conversion operations, e.g. foo->ToNumber() is the same as +foo in JS.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That text describes what I saw when I wrote some scratch code to explore the difference: sam-github@3efb492 , so the behaviour I saw doesn't really agree with the As() is just a cast. I saw conversion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sam-github That should crash hard in debug mode as invalid casts – it’s just luck that V8 doesn’t crash in release mode, because the implementations of the methods you’re using there don’t actually require an object of the specified type. E.g. Boolean::Value() is really just checking whether the object strict-equals true – that operation can work on any type. Number::Value() uses a V8 internal that’s implemented for all object types, so it also just happens to work out by accident (possibly incorrectly – I don’t think it actually handles conversions).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove this paragraph? There's no templated To<T>.

(There's Maybe<T>::To() but that's different.)


- FunctionCallbackInfo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Document args.Holder() ?

- can GetIsolate()
- can get Environment: Environment* env = Environment::GetCurrent(args);
- can get args using 0-based index, which returns Local<Value>
- has a .Length(), access past length returns a Local<Value> where value is
Undefined (as in JavaScript).
- The argument values have a number of Is*() predicates which check exact type
of Value, and (mostly) do NOT consider possible conversions:
- {then: ()=>{}} is not considered a Promise,
- `1` is not considered `true`,
- `null` is not an `Object` (!),
- new String() is a StringObject (not a String),
- 3 is a Int32 and also a Uint32, -3 is only a Int32, both are a Number
- etc.

Conventions on arg checking: two patterns are common:
1. C++ functions directly exposed to the user
2. C++ functions wrapped in a js function, only js is exposed to the user

The first option requires careful checking of argument types.

The second option is becoming more common. In this case the js function can
check all the argument types are as expected (throwing an error if they are
not), and destructure options properties to pass them as positional args (so the
C++ doesn't have to do Object property access and presence/type checks). C++
can use its args fairly directly, aborting if the js layer failed to pass the
expected types:

CHECK(args[0]->IsInt32());
int32_t arg0 = args[0].As<Int32>()->Value();


- A context is an execution environment that allows separate, unrelated,
JavaScript code to run in a single instance of V8. The motivation for using
contexts in V8 was so that each window and iframe in a browser can have its
own fresh JavaScript environment.
XXX Node uses one context, mostly, does vm. create new ones? anything else?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that’s about it.

One kind-of-open question is whether we want Node’s own APIs to eventually supports multiple contexts, but that would be a major change to some of our internals…

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does vm (or anything else) create new Contexts?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the vm module does that. It’s currently the only public API for that.


Can get from `isolate->GetCurrentContext()`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

… which is why this should be the same as env->context() in almost all cases.



XXX Function vs FunctionTemplate ... wth?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The distinction isn’t that important to us because we don’t expose many APIs for more than one context (MessagePort being the only exception I can think of right now) – essentially, a FunctionTemplate can be used to create functionally identical function instances for multiple contexts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FunctionTemplates are also useful because they give you an ObjectTemplate (through InstanceTemplate()) that you can use to define internal fields. You can't do that with regular Functions.



- node::Environment contains an Isolate and a Context, various other
information global to Node, and many convenient methods.

It is possible to get an Environment from many v8 objects by calling
Environment::GetCurrent() on a v8::Isolate*, v8::Local<v8::Context>,
v8::FunctionCallbackInfo<v8::Value>, etc...

An Environment can be used to get an Isolate, Context, uv_loop_t, etc.

Commonly used convenience methods:
- ThrowError/TypeError/RangeError/ErrnoException/...
XXX why are some called Error and others called Exception?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Legacy :-) ThrowErrnoException() and ThrowUVException() are old, the others are newer. I'd be okay with renaming them if that clears up the confusion / cognitive dissonance.

- SetMethod/SetMethodNoSideEffect/SetProtoMethod
NoSideEffect means it's safe for the debugger to eagerly evaluate,
SetProtoMethod() sets on obj.__proto__ rather than obj
- SetImmediate
- EnvironmentOptions* options(): "some" options...
XXX how to reach PerIsolateOptions, PerProcessOptions
- etc.

Contains many global strings and symbols.
XXX ...


## `NODE_MODULE_CONTEXT_AWARE_INTERNAL`

See: https://nodejs.org/api/addons.html#addons_context_aware_addons

Called with:
- `Local<Object> exports`: where to put exported properties, conventionally
called `target` in Node.js
- `Local<Value> module`: conventionally unused in Node.js
XXX what is this for? addon docs don't mention or use it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be the same value that module has in CJS scripts – I think this is mostly there so that it matches the addon API more closely? We can remove it if we want, I’d say.

- `Local<Context> context`:
- void* priv: not commonly used
XXX where is it ever used? for what?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think anybody uses this in practice, neither Node.js itself nor addons. I think the idea was for it to function as sort of a opaque pointer, but that never became useful due to the way that we load addons?


Initialize is generally used to set methods and contants:

Environment* env = Environment::GetCurrent(context);
env->SetMethod(target, "name", Name);
// ... see Environment for method creation convenience functions
NODE_DEFINE_CONSTANT(target, MACRO_NAME);
// read-only, not deletable, not enumerable:
NODE_DEFINE_HIDDEN_CONSTANT(target, MACRO_NAME);

For functions that wrap a C++ object in a js object, manually do what SetMethod
does, see Hmac::Initialize as an example:
1. Create a FunctionTemplate, used to setup function properties. The various
wrappers all call v8::FunctionTemplate::New() with various arguments, but
env->NewFunctionTemplate() is most commonly used. Signature,
ConstructorBehavior, and SideEffectType can be customized, but aren't
documented, and usually are left as default.
2. Call env->setProtoMethod() to setup instance methods
3. Get a function from the template
4. Set a string in the target to the function

Mysterious boilerplate:
- ToLocalChecked() see MaybeLocal vs Local
- FromJust(): XXX



Local<FunctionTemplate> t = env->NewFunctionTemplate(New);

Fields are used to store pointers to C++ objects:
XXX I think
t->InstanceTemplate()->SetInternalFieldCount(1);
Set methods:


https://v8.dev/docs/embed#more-example-code
- XXX read through, it has examples of calling the API



- <http://izs.me/v8-docs/classv8_1_1Object.html>
- https://code.google.com/p/v8/
- Building: <https://code.google.com/p/v8/wiki/BuildingWithGYP>
- how to compile js to see what it looks like?
- [Breaking V8 Changes](https://docs.google.com/a/strongloop.com/document/d/1g8JFi8T_oAE_7uAri7Njtig7fKaPDfotU6huOa1alds/edit)
- <https://developers.google.com/v8/get_started>
- <https://chromium.googlesource.com/v8/v8/+/master/docs/using_git.md>

- <https://developers.google.com/v8/embed>

Handle is base, from that are Local (go in HandleScope), and Persistent
(manually managed scope). Constructors (String::New) seem to return Locals.

`return Local<Array>();` ... seems to do exactly what you are not supposed
to do, but it's because the handle is empty. Its OK to return empty
handles, just not handles that point to something without going through
`EscapableHandleScope::Escape()`.