Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wip #1

Closed
wants to merge 18 commits into from
Closed

wip #1

wants to merge 18 commits into from

Conversation

ryzokuken
Copy link
Owner

/cc @hashseed

@@ -122,14 +122,9 @@ var modulePaths = [];
Module.globalPaths = [];

Module.wrap = function(script) {
return Module.wrapper[0] + script + Module.wrapper[1];
return vm.wrapFunction(script);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed, let's rename this. Maybe to getWrappedFunction.

Copy link

@hashseed hashseed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, the old way offered code caching. We should make this possible too. v8::ScriptCompiler::CompileFunctionInContext actually provides code caching functionality.

};

TryCatch try_catch(env->isolate());
Environment::ShouldNotAbortOnUncaughtScope no_abort_scope(env);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need this. TryCatch catches the exception.


Local<Function> fun = maybe_fun.ToLocalChecked();
CHECK(!fun.IsEmpty());
CHECK(!try_catch.HasCaught());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be written as

Local<Function> fun;
if (maybe_fun.ToLocal(&fun)) {
  // set return value to fun
} else {
  // rethrow exception
}

CHECK(!fun.IsEmpty());
CHECK(!try_catch.HasCaught());

Local<String> result = fun->ToString(context).ToLocalChecked();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is precisely what we do not want to do. Do not turn the function into a string. We want the function itself.


// vm.wrapFunction
{
assert.strictEqual(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More tests for

  • calling the function with arguments and check that the arguments can be used as expected, e.g. can be returned.
  • another syntax error with random garbage as source

@ryzokuken
Copy link
Owner Author

Would you expect the user to send Cache as a second argument to the function?

displayErrors: true
});
const func = vm.getWrappedFunction(content, false);
var compiledWrapped = func(); // FIXME(ryzokuken): This won't work.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you don’t want to call func yet, but instead have compiledWrapper = vm.getWrappedFunction(…) and leave the rest?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

originally, compilerWrapper would return a v8::Value that'd be returned as the result of your script, which it'll receive by calling script->Run().

All that I think needs to be done here is somehow calling the function (func->Call()) with the same context (env->context()). All the needed vars (require, __dirname, etc) will be in there, right?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax on second thought, you're probably right.

@ryzokuken ryzokuken force-pushed the module-17396 branch 2 times, most recently from 9c0cfb8 to 2accbab Compare June 22, 2018 16:49
env->module_parameter_module(),
env->module_parameter_filename(),
env->module_parameter_dirname()
};

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might actually be nice if we could pass these in as arguments to for GetWrappedFunction()?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would be nice, yes. That way this function only provides a primitive to build upon.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax like... accepting the params array as arguments? Do I fallback to these values as defaults, or do I pass these into the primitive function which requires them?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do I pass these into the primitive function which requires them?

Yes, that. (Assuming “requires” = “accepts as arguments”)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, by require I mean that you need to pass them in as opposed to optionally specifying them in which case they fallback to defaults.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an empty array would make a sensible default here

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax cool. So should I add the said primitive in this PR, or should I follow up with a PR that refactors most of the current code into the primitive, with extra code to handle params as args to the new function would just call the primitive with these params as an arg?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure which parts you are referring to by “current code”, but generally, you can use a single PR that contains two logically separate commits:

  • One that adds the CompileFunctionInContext binding to the vm module and does not touch the module load
  • One that makes the CJS loader use that binding and doesn’t touch the VM module

Does that help?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does.

@ryzokuken
Copy link
Owner Author

@addaleax @hashseed while all complications seem to have gone away, ESLint seems to be failing:

Cannot read config file: /Users/ryzokuken/Code/nodejs/node/.eslintrc.js
Error: The "path" argument must be of type string. Received type undefined
TypeError [ERR_INVALID_ARG_TYPE]: Cannot read config file: /Users/ryzokuken/Code/nodejs/node/.eslintrc.js
Error: The "path" argument must be of type string. Received type undefined
    at assertPath (path.js:39:11)
    at Object.dirname (path.js:1270:5)
    at module.exports (<anonymous>:11:34)
    at loadJSConfigFile (<anonymous>:154:16)
    at loadConfigFile (<anonymous>:209:22)
    at loadFromDisk (<anonymous>:495:18)
    at Object.load (<anonymous>:559:20)
    at Config.getLocalConfigHierarchy (<anonymous>:227:44)
    at Config.getConfigHierarchy (<anonymous>:179:43)
    at Config.getConfigVector (<anonymous>:286:21)

any idea how any of my changes could trigger this?

@ryzokuken
Copy link
Owner Author

We're somehow not helping with the file paths? undefined:752

@ryzokuken
Copy link
Owner Author

Wait, it's probably because I'm not using the filename option in _compile anymore, but where do I use it? Anywhere we can enter filepaths while compiling functions?

@addaleax
Copy link

@ryzokuken Maybe look at the existing usage of ScriptOrigin?

(In the end, you probably want a function that resembles the existing ContextifyScript::New() pretty closely, and maybe refactor out some common bits)

@ryzokuken
Copy link
Owner Author

@addaleax tests run now finally, yay.

ArrayBuffer::Contents contents = cached_data_buf->Buffer()->GetContents();
uint8_t* data = static_cast<uint8_t*>(contents.Data());
cached_data = new ScriptCompiler::CachedData(
data + cached_data_buf->ByteOffset(), cached_data_buf->ByteLength());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this can really be done inside the if-block where you checked args[2]. That way you can also define cached_data_buf inside the block.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did it in this particular fashion to mimic what New did, so I have no preferences here whatsoever. Will do this.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. In that case it's alright then.


ScriptCompiler::CompileOptions options;
if (source.GetCachedData() == nullptr) {
options = ScriptCompiler::kNoCompileOptions;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This option can also be set inside the if-block for args[2].

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, will do.

env,
reinterpret_cast<const char*>(cached_data->data),
cached_data->length);
fun->Set(env->cached_data_string(), buf.ToLocalChecked());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax do you have a suggestions as to where the resulting buffer could be better stored?

Alternatively, we don't offer this at all, and expose a separate binding for ScriptCompiler::CreateCodeCacheForFunction similar to how we also recently exposed ScriptCompiler::CreateCodeCache. That way this function gets a bit simpler, but then it would also lose symmetry to ContextifyScript::New.

Copy link

@devsnek devsnek Jun 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since the option for ContextifyScript::New is deprecated in my pr i think its best not to mimic it.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@devsnek could you drop me a link to it? Currently it Sets the two on the returned function object itself, which sounds like similar (if not the same) to what we're doing in ContextifyScript::New. Would love to see any alternatives you have in mind.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@devsnek cool, this is exactly what @hashseed pointed out, I could do this. That said, it's still just a documentation-level deprecation, right?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well... it will be docs-dep for 10, runtime dep for 11, and removed in 12.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case it would be reasonable to not introduce it here in the first place :)

// vm.getWrappedFunction
{
assert.strictEqual(
vm.getWrappedFunction('console.log("Hello, World!");'),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a .toString() here to be explicit about what we are comparing here?

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

definitely. The tests haven't been updated to reflect the latest changes made in the code.

type: SyntaxError,
message: 'Unexpected token }'
});
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought you had more tests :)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I commented out these as well, for now. Now that the existing tests all pass, will focus on covering as many cases as possible with tests.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I commented out these as well, for now. Now that the existing tests all pass, will focus on covering as many cases as possible with tests.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I commented out these as well, for now. Now that the existing tests all pass, will focus on covering as many cases as possible with tests.

@hashseed
Copy link

(In the end, you probably want a function that resembles the existing ContextifyScript::New() pretty closely, and maybe refactor out some common bits)

This. The new binding should mirror the old code path as closely as possible.

@ryzokuken
Copy link
Owner Author

ryzokuken commented Jun 22, 2018

The new binding should mirror the old code path as closely as possible.

Agreed. This will definitely be a priority.

BTW, @hashseed @addaleax at this point, all existing tests on master pass, so I personally feel good enough about this, code-wise (we could definitely add more functionality though). LMK whenever you're okay with this going to a public PR.

The TODOs for now are:

  • Add nice tests
  • Add appropriate documentation
  • Use gzemnid, analyze github data, etc to see the usage stats of affected functions, this would help make a nice argument for it once I make the PR
  • Add support for sandboxes (maybe another PR?)
  • Make a primitive variant (maybe another PR?)
  • Make this as similar as possible to New, and then refactor the common bits (maybe another PR?)

@addaleax
Copy link

@ryzokuken I think the priorities might be another way – making this as similar as possible to New wouldbe #1, and a lot of the rest already follows from that?

Context::Scope scope(context);

MaybeLocal<Function> maybe_fun = ScriptCompiler::CompileFunctionInContext(
context, &source, 5, params, 0, nullptr, options);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to also support the extensions param

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@devsnek definitely. I had been thinking of shipping in primitives with much more features (sandboxes, extensions, line_offset and comment_offset and what not) in hopefully a subsequent PR but I'm open to adding it in this one.

@ryzokuken
Copy link
Owner Author

So, I added a few tests that I felt seemed absolutely necessary and made sure that the function works as expected. Let me know if you all have any more test cases in mind.

Copy link

@hashseed hashseed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. Aside from design choices for which I'm not the right person.

env,
reinterpret_cast<const char*>(cached_data->data),
cached_data->length);
fun->Set(env->cached_data_string(), buf.ToLocalChecked());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case it would be reasonable to not introduce it here in the first place :)

Local<Value> val = params_buf->Get(context, n).ToLocalChecked();
if (val->IsString()) {
params[n] = val.As<String>();
} else {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we shouldn't be doing the type checks in c++, just assume they're all strings at this point. (CHECK(val->IsString()) if you must)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could do this, but I'm not entirely convinced that we shouldn't do any error checking here. Perhaps something on the JS level?

return;
}
}
} else {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this condition shouldn't be needed, cjs should be a normal consumer of this function

CHECK(args[1]->IsString());
Local<String> filename = args[1].As<String>();

// Argument 3: line offset
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't think we really need offsets

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we’d probably want this for parity with New()

Copy link
Owner Author

@ryzokuken ryzokuken Jun 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to what @addaleax said. Added the additional args for parity.


// Read params from params buffer
size_t params_len;
Local<String> *params;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

node pointers hang left

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix, didn't notice.

Local<String> *params;
if (!params_buf.IsEmpty()) {
params_len = params_buf->Length();
params = new Local<String>[params_len];
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't forget to free this

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Will do.

@devsnek
Copy link

devsnek commented Jun 25, 2018

it might be a good idea to open this pr on the node repo to get more feedback


// Read params from params buffer
size_t params_len;
Local<String> *params;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be nicer to use a std::vector<Local<String>> – right now, this looks like a memory leak because params is not being deleted anywhere?

(Otherwise, style-nit: left-leaning pointers :))

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax I had been tempted to the that too, but the function needs an array and size value, so why bother putting stuff in a vector just to decompose it later?

Re:delete, it was a oversight and I will delete params appropriately 😅

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because std::vector is basically a cleaner way of what new/delete does, namely it avoids this exact kind of memory leak and is a bit more flexible in general? You can feel free to not do it, but I’d be 90 % sure somebody else would ask for it in the public PR then

params[1] = env->module_parameter_require();
params[2] = env->module_parameter_module();
params[3] = env->module_parameter_filename();
params[4] = env->module_parameter_dirname();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think the default should be an empty array

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@devsnek (#1 (comment)) and @addaleax I thought so at first too, but chose otherwise because I thought eternals would be faster?

Anyway, if you're still both +1 to defaulting to an empty array instead, then I think I'm okay with it.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought eternals would be faster?

I don’t think it makes much of a difference. If you have an empty default parameter list, you don’t have any extra native streams, that’s also a nice thing.

But more importantly, this sets the API right to be self-contained rather than something with the specific purpose of supporting the module wrapper

lib/vm.js Outdated
@@ -294,6 +299,8 @@ module.exports = {
runInNewContext,
runInThisContext,
isContext,
getWrappedFunction,
getWrappedFunctionGeneric,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we only want to have one export of this kind? Maybe this is already implied in the WIP tag here, just want to make sure it doesn’t get lost :)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wanted to expose the generic version in case someone needs it. The getWrappedFunction is a smaller variant with only 2 args and default values.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That said, an options object with defaults sounds better, so I'll probably do that instead.

lib/vm.js Outdated
@@ -286,6 +287,10 @@ function runInThisContext(code, options) {
return createScript(code, options).runInThisContext(options);
}

function getWrappedFunction(code, filename) {
return getWrappedFunctionGeneric(code, filename, 0, 0, undefined, false);
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest as the public API:

vm.compileFunction(code, params = [], {
  filename,
  columnOffset,
  lineOffset,
  cachedData, 
  produceCachedData
} = {})

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, Will do.

lineOffset: 0,
displayErrors: true
});
const compiledWrapper = vm.getWrappedFunction(content, filename);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would there be a way to have the cake and eat it to? As is the removal of Module.wrap use is a no-go for me since CJS tools rely on it. If there is a way to get the benefit + allow Module.wrap tie-ins that would be rad.

This could be done in a variety of ways. For example using the Module.wrapper[0] and Module.wrapper[1] to be used to compile something internally (waves hands as I'm not familiar with V8 internals). Or having setters on Module.wrapper, Module.wrapper[0], and Module.wrapper[1] to trigger the old non-vm.getWrappedFunction use.

Copy link

@jdalton jdalton Jun 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ryzokuken

@jdalton I didn't remove Module.wrap, nor do I intend to (atleast not in this PR). This PR just changes module._compile to not use it internally, technically intended as a semver-major (because new bindings).

If in case somebody does decide to remove Module.wrap, it will have to go through the standard deprecation process that @devsnek pointed out, thus you could be assured that it won't be going anywhere any soon. Plus we won't do anything that hurts the community, would we? ;)

This is PR addresses a minor inconvenience that has existed since Node's creation so I'm not OK dropping any existing CJS tie-ins here. Module.wrap and Module.wrapper have been used by tooling to customize CJS module construction so I'd be wary of removing their use here. CJS is essentially frozen so I'd urge you to reconsider.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdalton Proper syntax error messages that don’t get obfuscated by the added function code are important as well.

If there are specific use cases you’re thinking of, it would be good to know what public API would address those (and possibly whether they can also be addressed by monkey-patching the newly introduced vm method)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdalton what specifically makes Module.wrap special to an extent that CJS tools "rely" on it? I'd love to see code that works with Module.wrap but breaks on what we have right now. Aiming for a semver-minor here, I'll be doing whatever I can to make sure no userland code breaks (and technically, it shouldn't, even right now).

I mean, you very well know at this point about all problems that the original function causes and how this will be an improvement, but again, I'd be making sure that nothing breaks in the process and if something does, let me know.

Copy link

@devsnek devsnek Jun 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or having setters on Module.wrapper, Module.wrapper[0], and Module.wrapper[1] to trigger the old non-vm.getWrappedFunction use.

if we must keep using wrapper, this sounds like the best way. i would fire a deprecation warning too

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jdalton again, I'm removing neither Module.wrap nor Module.wrapper here.

If you could point out which CJS modules/tools broke while changing Module._compile internally, I could make sure it wouldn't.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what John is referring to isn’t that Module.wrap or Module.wrapper don’t work anymore, it’s that replacing/monkey-patching them doesn’t have the effect of actually changing Node’s behaviour anymore?

Copy link
Owner Author

@ryzokuken ryzokuken Jun 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, CJS modules are monkey-patching the Module object and we're holding up PRs because it won't allow them to monkey-patch a core module anymore!?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, CJS modules are monkey-patching the Module object and we're holding up PRs because it won't allow them to monkey-patch a core module anymore!?

Yes, it’s the sensible thing to do for us. We have Node’s past lack of proper hook APIs to blame for this, not users who need to get things to work…

Copy link

@jdalton jdalton Jun 25, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CJS is very monkey-patchable. This particular API and functionality has existed since Jan 2011 (so pretty old). Losing the ability to monkey patch in this way is why I'm raising the concern.

@addaleax
Copy link

Oh, also: The addition of a new vm method is semver-minor, but the module loader change is almost surely going to be considered semver-major.

@ryzokuken
Copy link
Owner Author

@addaleax no tests fail, and I hope no CITGM runs would fail as well. Why not semver-patch then?

@addaleax
Copy link

@ryzokuken It’s semver-major because of what John is talking about – it’s going to be a breaking change, effectively.

@addaleax
Copy link

Btw, one thing that we could do is to implement Module.wrap/Module.wrapper as getter/setter pairs, and fall back to the old mechanism if they are set by userland code. That would allow us to deprecate and eventually them independently, I think?

@ryzokuken
Copy link
Owner Author

ryzokuken commented Jun 25, 2018

@hashseed @addaleax @devsnek made the changes requested. LMK if I'm still missing something, but hopefully this is ready to go to nodejs/node. Will make a PR tomorrow (hopefully) :)

Copy link

@addaleax addaleax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is basically already an in-depth review, with these points addressed I think this would be good to go

Given the most recent bits of discussion I’d recommend splitting this into two PRs, first taking care of everything except the modules change, so that you can focus on the technical side of things there

lib/vm.js Outdated
@@ -26,6 +26,7 @@ const {
kParsingContext,
makeContext,
isContext: _isContext,
getWrappedFunction

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/getWrappedFunction/compileFunction/

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

umm, do you mean I should rename the C++ binding and well? The C++ binding is named getWrappedFunction//GetWrappedFunction while the public API function is called vm.compileFunction.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that’s what I mean :)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! Will do.


// Read params from params buffer
size_t params_len;
Local<String>* params;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’d still really encourage you to use something better than a raw pointer here. There’s a couple options…

  • std::unique_ptr<Local<String>[]>. If you want to stick with custom allocation, this is what you want – You use it almost the same, but it automatically takes care of de-allocation.
  • std::vector<Local<String>>. This also avoids dealing with raw pointers, and is probably the most idiomatic thing C++-wise.
  • MaybeStackBuffer<Local<String>, N> (where N is something like 10 or so). This is a Node.js-specific thing and lacks a lot of the flexibility of built-in container types, but it has the small advantage of avoiding a heap allocation for the common case.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, I'll switch to a vector.


for (int32_t n = 0; n < params_len; n++) {
Local<Value> val = params_buf->Get(context, n).ToLocalChecked();
CHECK(val->IsString());

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, Github cut off review here because my connection was flaky :/

For one, we don’t currently do any checking that val is actually a string in JS land, so right now passing in an array with non-string elements would crash the process (as far as I can tell)

Then, params_buf->Get(…) can fail if the array element is defined via a getter, so the .ToLocalChecked() may crash too – you probably want something like if (!params_buf->Get(context, n).ToLocal(&val)) return; (but just as a heads up, that would leak memory unless you go with an RAII option for managing params like the ones suggested above)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this had been troubling me too, and the check was quite different originally. Will do this once I make the switch to vectors.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'd rather we did the argument handling in jsland and passed just raw arguments like ContextifyScript::New

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@devsnek totally doable. Plus then we could make it throw descriptive errors instead of just crashing.

@ryzokuken
Copy link
Owner Author

@addaleax @devsnek @hashseed done, please take a final look before I make this public (for real). I think all concerns have been addressed.

@ryzokuken
Copy link
Owner Author

ryzokuken commented Jun 26, 2018

@addaleax @devsnek done. PTAL? Let's get this where it needs to be within like... an hour or so.

}
fun->Set(
env->cached_data_produced_string(),
Boolean::New(isolate, cached_data_produced));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both of these should use the non-deprecated overloads of Set() (and ideally also do error handling for these cases)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't check those out, will do this ASAP.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax I suppose that by non-deprecated you mean ones that take context, key, val instead of plain key, val? How do I know which functions are deprecated?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are marked V8_DEPRECATE_SOON or V8_DEPRECATED in v8.h :)

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, thanks.

std::vector<Local<String>> params;
if (!params_buf.IsEmpty()) {
for (uint32_t n = 0; n < params_buf->Length(); n++) {
Local<Value> val = params_buf->Get(context, n).ToLocalChecked();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier comment here about ToLocalChecked still stands, btw

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Umm, we're checking that each item in params is a string, so that's covered. ToLocalChecked can fail, so I guess we could return if it does. I hope there's no scope of a memory leak anymore, right?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we tested that it’s a string (although a sufficiently evil getter could also return a string while typechecking and then later return something else)

What I meant was that if it’s a getter, it can always fail, so we should return back to JS if it does

I hope there's no scope of a memory leak anymore, right?

I don’t think so, no. :)

@ryzokuken
Copy link
Owner Author

@addaleax I guess this does it?

@ryzokuken ryzokuken closed this Jul 8, 2018
ryzokuken pushed a commit that referenced this pull request Oct 23, 2018
Reverting this enables us to provide slower, but longer-lasting
replacements for the deprecated APIs.

Original commit message:

    Put back deleted V8_DEPRECATE_SOON methods

    This partially reverts
    https://chromium-review.googlesource.com/c/v8/v8/+/1177861,
    which deleted many V8_DEPRECATE_SOON methods rather than moving them to
    V8_DEPRECATED first. This puts them back and marks them V8_DEPRECATED.

    Note V8_DEPRECATED that were deleted in the same CL stay deleted.

    NOTRY=true
    NOPRESUBMIT=true
    NOTREECHECKS=true

    Bug: v8:7786, v8:8240
    Cq-Include-Trybots: luci.chromium.try:linux_chromium_rel_ng
    Change-Id: I00330036d957f98dab403465b25e30d8382aac22
    Reviewed-on: https://chromium-review.googlesource.com/1251422
    Commit-Queue: Dan Elphick <[email protected]>
    Reviewed-by: Yang Guo <[email protected]>
    Reviewed-by: Michael Hablich <[email protected]>
    Cr-Commit-Position: refs/branch-heads/7.0@{nodejs#49}
    Cr-Branched-From: 6e2adae6f7f8e891cfd01f3280482b20590427a6-refs/heads/7.0.276@{#1}
    Cr-Branched-From: bc08a8624cbbea7a2d30071472bc73ad9544eadf-refs/heads/master@{nodejs#55424}

Refs: v8/v8@9136dd8
Refs: nodejs#23122

PR-URL: nodejs#23158
Reviewed-By: Yang Guo <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants