Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Substitute for Inherit #229

Closed
MichaReiser opened this issue Feb 19, 2018 · 48 comments
Closed

Substitute for Inherit #229

MichaReiser opened this issue Feb 19, 2018 · 48 comments
Labels

Comments

@MichaReiser
Copy link

Hy

First of all, great work! I'm migrating a Nan based native addon to node-addon-api, and so far, the code is much cleaner than before. I do no longer need to copy past fragments as I now can remember the signatures and requires far less boilerplate code.

Right now I'm migrating my LLVM Wrapper for Node.js to napi and face the issue that I don't see a way to implement a class hierarchy using Napi::ObjectWrap. I need class hierarchies to correctly model the LLVM Value API.

Inheritance with the v8 API can be achieved by using the Inherit Function of the FunctionTemplate.

Is this use case yet supported or are there ideas how to implement it in node-addon-api?

Cheers,
Micha

@addaleax
Copy link
Member

Is this use case yet supported or are there ideas how to implement it in node-addon-api?

I don’t think it’s explicitly part of the API right now, not even the C one. :/ What exactly do you need from such an API? Exact 1:1 correspondence to V8’s Inherit?

How far do you think you could get with classical JS inheritance, that is, extending the prototype chain and calling the superclass constructor from the subclasses?

I need class hierarchies to correctly model the LLVM Value API.

Which project are you talking about? I’d be curious to look at it if it’s open source. :)

@MichaReiser
Copy link
Author

Hy Anna,

Thanks for your super fast response.

I don’t think it’s explicitly part of the API right now, not even the C one. :/ What exactly do you need from such an API? Exact 1:1 correspondence to V8’s Inherit?

No, I do not need a 1:1 correspondence to the V8 Inherit function.
From the consumer perspective, I would like that instanceof operator is working as expected. E.g. ConstantInt instanceof Value should return true.

As the maintainer, I would prefer if I can use ObjectWrap and that I do not have to reimplement the functions of the superclass, e.g., that Constant does not need to reimplement the methods of Value.

How far do you think you could get with classical JS inheritance, that is, extending the prototype chain and calling the superclass constructor from the subclasses?

I was also thinking about creating a prototype chain but am unsure how to achieve this with the new API (that's the reason I ask). Could you help me out with a small example or some ideas?

Which project are you talking about? I’d be curious to look at it if it’s open source. :)

It is open source. You can find it on GitHub.

@addaleax
Copy link
Member

From the consumer perspective, I would like that instanceof operator is working as expected. E.g. ConstantInt instanceof Value should return true.

Prototype manipulation should be able to do that. :)

As the maintainer, I would prefer if I can use ObjectWrap and that I do not have to reimplement the functions of the superclass, e.g., that Constant does not need to reimplement the methods of Value.

I think that’s not quite working yet. Making the necessary adjustments to the C++ wrapper should be pretty doable, but I think there’s at least one thing we can’t emulate: How V8 does signature checking for the invoked methods.

When invoking a subclass method, I don’t think how we could easily tell that the this object actually is an instance of that subclass… we could either ignore that and let the user deal with invalid casts, or add something like an instanceof check (that has runtime overhead + can be fooled by userland too).

I’m not sure what to do. :/

@MichaReiser
Copy link
Author

Prototype manipulation should be able to do that. :)

How would you do that? By setting the Prototype property on the constructor function and call the parents constructor in the subclasses constructor? If I know how to achieve this I could try it shortly if it is working.

I see the pros and cons we are facing with adjusting the ObjectWrap class and I think paying the overhead in all other cases where inheritance isn't needed is not really an option. The question is, might we expose an ObjectWrapper like class especially for the inheritance use case? I'm sorry that I'm not a great help here. What are the specifics I could study to get a better overview of the topic so that I can be of more help?

@gabrielschulhof
Copy link
Contributor

From nodejs/node#4179 it seems that full inheritance can be accomplished with

Object.setPrototypeOf(SubClass.prototype, SuperClass.prototype);
Object.setPrototypeOf(SubClass, SuperClass);

This can be done with existing N-API calls (untested, and ignoring return status):

void napi_inherits(napi_env, napi_value ctor, napi_value super_ctor) {
  napi_value global, global_object, set_proto, ctor_proto_prop, super_ctor_proto_prop;
  napi_value args[2];

  napi_get_global(env, &global);
  napi_get_named_property(env, global, "Object", &global_object);
  napi_get_named_property(env, global_object, "setPrototypeOf", &set_proto);
  napi_get_named_property(env, ctor, "prototype", &ctor_proto_prop);
  napi_get_named_property(env, super_ctor, "prototype", &super_ctor_proto_prop);

  argv[0] = ctor_proto_prop;
  argv[1] = super_ctor_proto_prop;
  napi_call_function(env, global, set_proto, 2, argv, NULL);

  argv[0] = ctor;
  argv[1] = super_ctor;
  napi_call_function(env, global, set_proto, 2, argv, NULL);
}

@iclosure
Copy link

Thank you very much!

@MichaReiser
Copy link
Author

Thanks. This helps to set up the prototype chain correctly.

The issue that remains from my point of view is that I do not know how to call the super constructor from the child class. This is needed in my case to initialize some private fields that are then accessed in the methods.

@gabrielschulhof
Copy link
Contributor

@MichaReiser In my example I simply call one binding from the other.

@MichaReiser
Copy link
Author

@gabrielschulhof thanks for your example. However, I'm using the node-addon api (C++ object wrappers) and I don't believe I can call the super constructor in any way (since I cannot inherit).

@gabrielschulhof
Copy link
Contributor

gabrielschulhof commented Apr 25, 2018 via email

@MichaReiser
Copy link
Author

I believe this is not possible when the constructor initializes private fields (these are not static). Except if you store these as JavaScript values (but then you lose the benefits of using ObjectWrap in the first place).

@gabrielschulhof
Copy link
Contributor

IIRC if the static function is part of the same class then it has access to the private fields:

#include <stdio.h>

class Something {
 public:
  Something(int x, double y) {
    InitializeSomething(this, x, y);
  }
  void print() {
    fprintf(stderr, "%d, %lf\n", x, y);
  }
  static void InitializeSomething(Something* self, int x, double y) {
    self->x = x;
    self->y = y;
  }
 private:
  int x;
  double y;
};

class SomethingSubclass: public Something {
 public:
  SomethingSubclass(int x, double y): Something(x, y) {}
};

int main() {
  SomethingSubclass z(-1, 42.0);
  z.print();
  return 0;
}

@mhdawson
Copy link
Member

Any update on the discussion here?

@MichaReiser
Copy link
Author

I tried to follow the instructions from @gabrielschulhof but I still believe that the proposed solution does not work with ObjectWrap and, therefore, no real substitution for the functionality of v8:Inherit and nan::ObjectWrap exist.

I still try to migrate some llvm bindings for node from nan to napi and node-addons-api. I struggle to migrate the PointerType class of llvm that inherits from type.

I implemented the Type (header) and PointerType (header) classes using node-addons-api without inheritance.

If I use the proposed napi_inherits function the prototype chain is set up correctly. But since the PointerTypeWrapper class does not inherit from the TypeWrapper class all calls to methods defined in the Type class on instances of PointerType fail.

When I used nan, the PointerTypeWrapper class inherited from the TypeWrapper class. However, with node-addon-api this seems no longer possible.

So my question is: Is there a way to implement class inheritance using node-addon-api and ObjectWrap or can I not make use of ObjectWrap at all in cases where inheritance is needed?

Thanks,
Micha

@gabrielschulhof
Copy link
Contributor

@MichaReiser

But since the PointerTypeWrapper class does not inherit from the TypeWrapper class all calls to methods defined in the Type class on instances of PointerType fail.

How do you mean "fail"?

@MichaReiser
Copy link
Author

The call throws a type error. E.g. when invoking toString on a PointerType

    TypeError: Illegal invocation

      181 |       llvm.Type.getDoubleTy(context)
      182 |         .getPointerTo()
    > 183 |         .toString()
          |          ^
      184 |     ).toBe("double*");
      185 |   });
      186 | });

Which makes sense since the TypeWrapper has a private field type which PointerTypeWrapper does not initialize.

@gabrielschulhof
Copy link
Contributor

gabrielschulhof commented Jul 10, 2018

@MichaReiser actually, the "Illegal invocation" is thrown internally by V8. The cause is indeed the fact that PointerTypeWrapper is not a subclass of TypeWrapper, but it has nothing to do with whether you're using node-addon-api, NAN, or, indeed, plain V8. It has to do with the fact that the way Node.js defines classes (both when using NODE_SET_PROTOTYPE_METHOD and when using N-API's napi_define_class()) is that it defines prototype methods with a signature check. That is, it instructs V8 to ensure that the receiver for the method is an instance of the class or one of its parent classes.

Unfortunately, V8 decides whether the receiver is an instance of a parent class solely based upon whether the child class was created using v8::FunctionTemplate::Inherit on the native side, or the extends keyword on the JavaScript side. Merely lining up the prototype chain will not cause V8 to consider an instance of PointerTypeWrapper to be an instance of TypeWrapper for the purposes of the signature check, even though the instanceof operator will work correctly.

If we were to implement napi_inherits() in N-API, we could only do so if we could retrieve the FunctionTemplate for a v8::Function so as to be able to pass it to v8::FunctionTemplate::Inherit. V8 currently does not allow us to do this.

In N-API's napi_define_class() we could store the v8::FunctionTemplate in a private field of the v8::Function we create, and retrieve it when the addon calls napi_inherit. However, this would only work for functions created with N-API. If we wanted to extends a class defined in JavaScript, we'd only have access to the v8::Function of its constructor, and not its v8::FunctionTemplate. Thus, napi_inherit would only work some of the time.

The other alternative would be to drop the signature check in Node.js – both in node::ObjectWrap for V8 addons, and in napi_define_class() for N-API or node-addon-api addons. If we did that, lining up the prototype chain would be sufficient. Yet Node.js has had the signature check for as long as it's been around, so doing so would definitely require wider discussion.

@hashseed @verwaest is it possible to have V8 check the prototype chain as well as the FunctionTemplate chain in GetCompatibleReceiver()? I have found that, unless the subclass derives from the parent class via v8::FunctionTemplate::Inherit or in JavaScript via the extends keyword, the prototype chain is never consulted, although there is code in GetCompatibleReceiver() for checking the prototype chain.

@gabrielschulhof
Copy link
Contributor

@hashseed @verwaest I made a patch that adds a V8 cctest to illustrate the difference in the behaviour of GetCompatibleReceiver(). It seems that GetCompatibleReceiver() only does the prototype chain iteration if the candidate receiver has a hidden prototype – which in the case of a plain setPrototypeOf()-based inheritance it does not.

@gabrielschulhof
Copy link
Contributor

Returning nullptr if the prototype is not hidden was added in v8/v8@9bfd7b9 as part of an optimization pass.

@hashseed
Copy link
Member

hashseed commented Jul 31, 2018

Relaying message from @verwaest since he doesn't have access to his github account right now:

The signature check only allows exact receivers to go through since they need to be instances of the template to have a known layout. This has nothing to do with prototype chain checks
and the hidden thing isn't an "optimization". That's just the only place where "prototypes" are "part of" the receiver.

Unfortunately, V8 decides whether the receiver is an instance of a parent class solely based upon whether the child class was created using v8::FunctionTemplate::Inherit on the native side, or the extendskeyword on the JavaScript side. Merely lining up the prototype chain will not cause V8 to consider an instance of PointerTypeWrapper to be an instance of TypeWrapper for the purposes of the signature check, even though the instanceof operator will work correctly.

You misunderstand the purpose of this check. The signature only allows receivers created from an exact template to go through.

TL;DR: the signature check has nothing to do with prototypal inheritance. It checks whether an object has been stamped out from a certain template, expecting a certain object layout. Changing the prototype chain does not affect this.

Alternatively, you could post this question to [email protected] too.

@gabrielschulhof
Copy link
Contributor

@hashseed @verwaest in that case I would say that this is a bug, because this check prevents one from implementing prototype-based inheritance. That is, given a native-backed superclass created with signature-checked prototype methods it is not possible to create a subclass of that superclass using only prototype assignment on the JavaScript side such that superclass prototype methods can be called on instances of the subclass.

@hashseed
Copy link
Member

Its purpose is not checking prototypal inheritance though. It's purpose is to make sure the object has the expected object layout, e.g. in-object fields.

@gabrielschulhof
Copy link
Contributor

@hashseed I understand that, but as a result it's preventing the correct functioning of prototypal inheritance.

@hashseed
Copy link
Member

Talked to @verwaest offline.

{__proto__:Array.prototype} doesn't make it an array. Same goes for RegExp objects. These objects have per spec internal fields that their methods require. Just overriding the prototype doesn't magically e.g. Array.length work correctly. The signature check exposes this logic to the embedded.

@gabrielschulhof
Copy link
Contributor

gabrielschulhof commented Jul 31, 2018

@hashseed OK, I think I understand.

var Sub = function RegExpSubclass(string, flags) {
  RegExp.call(this, string, flags);
};

Object.setPrototypeOf(Sub.prototype, RegExp.prototype);
Object.setPrototypeOf(Sub, RegExp);

var superInst = new RegExp('\S+', 'g');
var subInst = new Sub('\S+', 'g');

console.log(JSON.stringify(superInst.exec('abc def'), null, 4));
console.log(JSON.stringify(subInst.exec('abc def'), null, 4));

produces an error in the second case on both V8 and SpiderMonkey.

@nodejs/n-api I guess if we want addon authors to allow subclassing of the classes they expose (by allowing them to specify that N-API forego the signature check) we need to give them a way to create prototype methods without signatures – i.e. a new variation of napi_define_class() which does not add signature checks to prototype methods.

@verwaest-zz
Copy link

verwaest-zz commented Jul 31, 2018 via email

@BotellaA
Copy link
Contributor

Any news on this issue?

@gabrielschulhof
Copy link
Contributor

@verwaest currently V8 allows for the following two scenarios:

  1. Both superclass and subclass are written in JavaScript, and the subclass is defined using the extends keyword.
  2. Both superclass and subclass are written in C++ and the code performing the FunctionTemplate::Inherit has access to both FunctionTemplates.

Neither of these scenarios works from N-API code, because the FunctionTemplate is not available at the point of defining the subclass and cannot be retrieved using the Local<Function> that was created from it.

If the parent class and the child class are part of the same code base, such as in a native addon, it's easy to keep the parent and child FunctionTemplates around to subsequently connect them with a call to FunctionTemplate::Inherit, but in N-API it cannot be done reliably, because we cannot assume that the two values passed into a call to a potential future napi_inherit() were both created with N-API. Users of N-API might expect to be able to subclass a class they created by means other than N-API's napi_define_class(). For example:

const bindings = require('./build/Release/native_bindings');
class Superclass {
}
bindings.subclassMe(Superclass);

The fundamental problem is that V8 is internally able to retrieve the parent function template when defining the subclass, but embedders have no way of retrieving the FunctionTemplate from a Function.

In N-API's napi_define_class() we could store a persistent reference to the function template on the resulting function so, when we are then asked to subclass it, we can, but that would only work for pairs of classes where both were created with N-API and it would not work for the above example.

So, is it possible to have a V8 API that requires only two Local<Function>s to perform an Inherit()?

@gabrielschulhof
Copy link
Contributor

gabrielschulhof commented Sep 19, 2018

@verwaest scenario 0 does work for template-backed functions.

What doesn't work is my example below that (copied and corrected here):

const bindings = require('./build/Release/native_bindings');
class Superclass {
}
const Subclass = bindings.subclassMe(Superclass);

because in the native implementation of subclassMe there is no way to call FunctionTemplate::Inherit(), because we do not have the FunctionTemplate of the superclass.

@verwaest-zz
Copy link

verwaest-zz commented Sep 19, 2018 via email

@gabrielschulhof
Copy link
Contributor

@verwaest I would like to write the following function in C++:

function subclassMe(superclass) {
  class SubClass extends superclass {}
  return SubClass;
}

in such a way that, if I use it, I will not subsequently get Illegal Invocation exceptions when I call a superclass method on an instance of the subclass.

However, this would mean that the C++ implementation of subclassMe would have to have access to the FunctionTemplate of both the superclass that was passed into it and the subclass it creates, so that it can connect the two via FunctionTemplate::Inherit.

The JavaScript implementation of subclassMe is able to receive a JavaScript function and correctly perform the extends. The C++ implementation can only do prototype assignment to try to mimic the extends, but that results in illegal invocation exceptions if the superclass has a prototype method, and that method gets called on an instance of a subclass created by prototype assignment - which is the only way to create subclasses in C++.

Here's a gist of what I'm trying to do:
https://gist.github.com/gabrielschulhof/377d1cf557794efc324d3c9a13c834e9

I realize that I could implement subclassMe in C++ by removing the v8::Signature on the method, but I would like to be able to keep the signature – after all, I'm able to keep the signature if I'm using the JS implementation of subclassMe.

@verwaest-zz
Copy link

verwaest-zz commented Sep 19, 2018 via email

@gabrielschulhof
Copy link
Contributor

@verwaest you mean, like, with RunScript? But then wouldn't you have to write the subclass in JS and save it in a string? Like,

const char* subclassMeJs =
"function subclassMe(superclass) {"
"  class subclass extends superclass {"
"    /* subclass methods would be defined in JS here */"
"  };"
"  return subclass;"
"}";

... and ultimately ...

Local<Value> subclass =
  subclassMeFn->Call(context, recv, 1, &superclass).ToLocalChecked();

@devsnek
Copy link
Member

devsnek commented Sep 20, 2018

Perhaps the solution here lies in using another pattern for your classes. What if you create all your classes in JS land and then supplement them with native methods?

@verwaest-zz
Copy link

verwaest-zz commented Sep 20, 2018 via email

@gabrielschulhof
Copy link
Contributor

@verwaest I can certainly implement that, but then I have to write the methods of the subclass in JavaScript.

So, a V8 API on Function that would be super awesome would be

v8::Local<v8::FunctionTemplate> v8::Function::Extend(...);

@gabrielschulhof
Copy link
Contributor

@verwaest I guess the parameters for the new API would be the same or almost the same as for v8::FunctionTemplate::New().

@maierfelix
Copy link

Any updates on this?

@mhdawson
Copy link
Member

mhdawson commented Sep 20, 2019

@gabrielschulhof one concern I have is that if we are asking for additional V8 functions, what is the likelyhood that other JS engines have the same functionality. We don't want to add something that precludes running on other engines...

gabrielschulhof pushed a commit to gabrielschulhof/node that referenced this issue Nov 17, 2020
Add the ability to define a subclass of a JS class by generating a JS
snippet that uses the `extends` syntax and calls the native bindings
which are passed in as parameters to the generated function responsible
for defining the subclass.

An example of a generated JS snippet might be:

```js
(function(parent, ctor, getSuperParams, prop0, prop1) {
  'use strict';
  class NativeSubclass extends parent {
    constructor() {
      super(...getSuperParams.apply(null, arguments));
      ctor.apply(this, arguments);
    }
    subMethod() {
      if (!(this instanceof NativeSubclass))
        throw new Error('Illegal invocation');
      return prop0.apply(this, arguments);
    }
    chainableMethod() {
      if (!(this instanceof NativeSubclass))
        throw new Error('Illegal invocation');
      return prop1.apply(this, arguments);
    }
  }

  return NativeSubclass;
})
```

where `ctor`, `getSuperParams`, `prop0`, and `prop1` are native
functions supplied to `napi_define_subclass`.

Signed-off-by: Gabriel Schulhof <[email protected]>
Refs: nodejs/node-addon-api#229
@gabrielschulhof
Copy link
Contributor

@mhdawson given that this can be implemented without support from the engine, as nodejs/node#36148 shows, I think it would be absolutely beneficial to have engine support for extending arbitrary JS classes.

@verwaest-zz @hashseed I'm not sure you are any longer involved with this subject, but hopefully you can ping the right folks to show them nodejs/node#36148 as an example of the kind of functionality we would need from V8. Basically, nodejs/node#36148 implements #229 (comment), but "a clear API on Function" would still help immensely to reduce the complexity of the implementation, not least because the extra indirection in nodejs/node#36148 reduces the performance of calling the native function even further.

@gabrielschulhof
Copy link
Contributor

CC: @verwaest

@github-actions
Copy link
Contributor

This issue is stale because it has been open many days with no activity. It will be closed soon unless the stale label is removed or a comment is made.

@ajihyf
Copy link

ajihyf commented Apr 27, 2022

I've managed to make inheritance work in two steps.

First, make one ObjectWrap hold all native objects by a base class pointer. Do not inherit different ObjectWrap, otherwise you will encounter the problem of diamond inheritance. And in this way, the property descriptors of the parent class can be copied into the child class safely.

class Clazz {
 public:
  virtual ~Clazz() {}
};

class ScriptWrappable : public Napi::ObjectWrap<ScriptWrappable> {
 public:
  ScriptWrappable(const Napi::CallbackInfo& info)
      : Napi::ObjectWrap<ScriptWrappable>(info) {
    // set _instance from meta data inside info.Data
  }

  template <typename T, Napi::Value (T::*cb)(const Napi::CallbackInfo&)>
  Napi::Value InstanceMethodCallback(const Napi::CallbackInfo& info) {
    T* t = static_cast<T*>(_instance.get());
    return (t->*cb)(info);
  }

  std::unique_ptr<Clazz> _instance;
};

class Base : public Clazz {
 public:
  Napi::Value Hello(const Napi::CallbackInfo& info);
};

class Sub : public Base {
 public:
  Napi::Value Hi(const Napi::CallbackInfo& info) {
    return Napi::String::New(info.Env(), "hi");
  }

  static Napi::Function DefineClass(Napi::Env env) {
    return ScriptWrappable::DefineClass(
        env,
        "Sub",
        {ScriptWrappable::InstanceMethod<
             &ScriptWrappable::InstanceMethodCallback<Base, &Base::Hello>>(
             "hello"),
         ScriptWrappable::InstanceMethod<
             &ScriptWrappable::InstanceMethodCallback<Sub, &Sub::Hi>>("hi")}
        // attach class meta data here
    );
  }
};

And then, monkey patch the prototype to make instanceof works

  Napi::Object global = env.Global();
  Napi::Object Object = global.Get("Object").As<Napi::Object>();
  Napi::Function setPrototypeOf =
      Object.Get("setPrototypeOf").As<Napi::Function>();

  Napi::Value clazz_proto = clazz.Get("prototype");
  Napi::Value parent_proto = parent_clazz.Get("prototype");

  setPrototypeOf.Call({clazz_proto, parent_proto});
  setPrototypeOf.Call({clazz, parent_clazz});

To make constructor and other things work correctly, class meta data should be attached as the data pointer of DefineClass call. I've created a lib node-addon-api-helper to simplify these things and hope it will help.

@ApsarasX
Copy link

ApsarasX commented Apr 29, 2022

@ajihyf The example C++ code in your comment cannot work well (some compilation error).

@ajihyf
Copy link

ajihyf commented Apr 30, 2022

@ajihyf The example C++ code in your comment cannot work well (some compilation error).

Sorry it's a code snippet I copied from my lib to show the basic idea and I didn't check its compilation outside the lib. It should work now.

@mmomtchev
Copy link

@devongovett, I have settled on a method which I think is the least bad one: https://mmomtchev.medium.com/c-class-inheritance-with-node-api-and-node-addon-api-c180334d9902

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests