overriding standard types

[nadako]

I tried to add haxe.io.Bytes implementation, but there's a problem that in a macro it also tries to use the overriden class and then complains about using js.node.Buffer in a macro context.

What can we do with that? One option is to just move nodejs sys stuff into Haxe itself (and fence with #if nodejs), another one is to make it possible to override types only for target output context, not the macro one.

Also, @:coreApi metadata doesn't work on our overriden types.

cc @Simn @ncannasse

[nadako]

Bytes implementation 72d7b83

[nadako]

Oh, and thanks to @andyli and Travis CI we can see that @:coreApi actually worked in 3.2.0: https://travis-ci.org/HaxeFoundation/hxnodejs/jobs/79463579

[eduardo-costa]

I didn't get yet whay @coreapi does :)
Em 09/09/2015 10:05, "Dan Korostelev" [email protected] escreveu:

Oh, and thanks to @andyli https://github.com/andyli and Travis CI we
can see that @:coreApi actually worked in 3.2.0:
https://travis-ci.org/HaxeFoundation/hxnodejs/jobs/79463579

—
Reply to this email directly or view it on GitHub
#42 (comment)
.

[nadako]

@eduardo-costa it checks whether all fields are defined correctly in the overriden std class. it's used in haxe std for providing target implementations. it does check field existance, type, access modifiers, method signatures, etc.

[eduardo-costa]

Got it! Thanks!
We should describe more of thia in he docs!
Em 09/09/2015 11:23, "Dan Korostelev" [email protected] escreveu:

@eduardo-costa https://github.com/eduardo-costa it checks whether all
fields are defined correctly in the overriden std class. it's used in haxe
std for providing target implementations. it does check field existance,
type, access modifiers, method signatures, etc.

—
Reply to this email directly or view it on GitHub
#42 (comment)
.

[ncannasse]

Yes I'm aware of the issue with Bytes redefinition. I actually opened HaxeFoundation/haxe#4249 because of that. Not sure if that's the right solution or not.

I actually thing we need to find better ways for libraries to override/implement some std classes as well as configure which implementation is used depending on the compiled platform (in a safe manner, ie like we do for @:coreApiin stdlib)

[waneck]

I wish we implemented the _std magic for all classpaths - not just the std classpath

[ncannasse]

This currently breaks haxe.io.BytesBuffer compilation, and breaks some WebGL code I'm having in Node Webkit as well (since I need to have Bytes that maps to JS ArrayBuffer). Should we not drop the haxe.io.Bytes <-> NodeBuffer support since it seems recent NodeJS version does uses ArrayBuffer ? (I didn't test the later)

[ncannasse]

Up again : should we remove this and instead add NodeBuffer <-> Bytes conversion (in NodeBuffer class) since they are both based on ArrayBuffer but uses different APIs ?

[nadako]

To be honest, I'd remove anything but node.js externs (to reduce scope of this particular lib), and then make a separate lib that provides sys stuff.

[clemos]

@nadako +1

[ncannasse]

Maybe, but that just moves the problem away, no ?
I think it would be nice for Haxe users to have a crossplatform sys API for Node. Even more if that's a complex problem to solve so that users don't have to reinvent the wheel.

But looking at the Buffer API it seems there's no way to easy convert from/to ArrayBuffer (with no copying being done in the process). I don't then understand why they say that in 4.0+, Buffer is backed by UInt8Array. Unless there's a trick, can Node users confirm this ?

If there's no easy way, I agree we should move it out of hxnodejs for now, until we come up with a solution.

[nadako]

Looking at this code: https://github.com/nodejs/node/blob/master/lib/buffer.js#L43-L62, it seems that Buffer actually IS Uint8Array

[ncannasse]

Oh. then that makes things easy, right ?

[nadako]

I don't know, will have to look into it? What's the decision anyway? Support 4.0+ and provide only Sys/sys.* stuff?

[ncannasse]

What I'm proposing (open for discuss):

• support 4.0+ Buffer (which extends UInt8Array)
• do not redefine haxe.io.Bytes and especially not BytesData (breaks several std classes and 3rd party libraries)
• add API in Buffer (using inline methods?) which allows from/to Bytes, I can help with this
• implement sys/Sys

[nadako]

Sounds sane, I'll try to find time to look into it today.

[ncannasse]

BTW it might be possible to have the node Buffer be a reference to an ArrayBuffer (with memory sharing) if created by passing an ArrayBuffer, to confirm by experiment, see
https://github.com/nodejs/node/blob/master/lib/buffer.js#L142

[ncannasse]

(passing an UInt8Array for instance will create a copy)

[nadako]

Okay, I removed haxe.io stuff and added conversion helper methods. Please review.

[nadako]

We have a problem with Haxe 3.2.0 however: the byteLength static method of node's Buffer class has the same name as Uint8Array non-static property, so it fails to compile (https://travis-ci.org/HaxeFoundation/hxnodejs/jobs/88176317). It's allowed on Haxe development though.

[nadako]

Also, can we get rid of js.html.compat stuff for nodejs somehow?

[eduardo-costa]

@nadako +1 to keep hxnodejs extern-only
We have the hxnodelibs repo that can fit extra libs for sys and externs for other npm tools.

[nadako]

I did that, pls review!

[clemos]

Looks good to me

[ncannasse]

ok, everything seems good to me!
shall we move forward with a proper release?
or do anyone want to contribute more sys/Sys implementation (quickly) before that?

Also, we need to decide on the name. We can have the library named either "nodejs" or "hxnodejs", as soon as we have a proper -D nodejs added as well so users can #if nodejs .... In all cases, it will be the best to replace / take down no longer maintained node libraries at the same time, such as http://lib.haxe.org/p/nodejs and http://lib.haxe.org/p/nodehx if their respective authors agree with it?

[eduardo-costa]

I'm good!
I own the nodehx lib! Feel free to put it down!

2015-11-03 18:18 GMT-02:00 Nicolas Cannasse [email protected]:

ok, everything seems good to me!
shall we move forward with a proper release?
or do anyone want to contribute more sys/Sys implementation (quickly)
before that?

Also, we need to decide on the name. We can have the library named either
"nodejs" or "hxnodejs", as soon as we have a proper -D nodejs added as
well so users can #if nodejs .... In all cases, it will be the best to
replace / take down no longer maintained node libraries at the same time,
such as http://lib.haxe.org/p/nodejs and http://lib.haxe.org/p/nodehx if
their respective authors agree with it?

—
Reply to this email directly or view it on GitHub
#42 (comment)
.

[image: Imagem inline 1]

[nadako]

@ncannasse

Also, can we get rid of js.html.compat stuff for nodejs somehow?

Because any Buffer usage right now outputs a ton of unused compat stuff in the output js.

[ncannasse]

@nadako Sorry, I've let this pass. Can you not simply #if !nodejs it ?

[dionjwa]

You can have http://lib.haxe.org/p/nodejs!

[nadako]

Okay, I added #if !nodejs guards to haxe std, seems quite clean (although it still adds empty haxe.io.Bytes constructor for some reason, but we can live with it, I guess). As for the name, I'd release it as hxnodejs and upload a deprecation release for nodejs and nodehx that point to the new lib.

[kevinresol]

hxnodejs seems consistent with hxjava, hxcs, hxcpp, but only if it includes the sys stuff?

[nadako]

Honestly, I'm okay with any name.

[kevinresol]

what if we name it hxnodejs and confine its scope as node extern + haxe std library?
Anything else will go to another nodelib repo

[eduardo-costa]

We have hxnodelibs for that..
I vote for hxnodejs as extern-only
Em 04/11/2015 23:38, "Kevin Leung" [email protected] escreveu:

what if we name it hxnodejs and confine its scope as node extern + haxe
std library?
Anything else will go to another nodelib repo

—
Reply to this email directly or view it on GitHub
#42 (comment)
.

[nadako]

@kevinresol +1

If there'll be no further discussion, I can make a haxelib release.

[andyli]

@nadako One question: Does the JS output runs equally fine on browsers when -D nodejs? Ref. HaxeFoundation/haxe#4524

[nadako]

Oh, I gotta read that...

[nadako]

So, that topic seems to be Haxe-related which means it shouldn't block release of hxnodejs. @ncannasse waiting for you to say the word.

[nadako]

I'll close this issue, as we really should use #40 for release discussion. The Bytes problem is solved now.

[nadako]

Actually, let's leave this open for now, since the general issue is still there. Maybe it should go to Haxe's issues?

[ncannasse]

hxnodejs is ok for me. Let's release! (unless there's something else to discuss before that?)

[back2dos]

@nadako The issue with the type collision can be solved by making two implementations in js.node.Bytes and js.node.BytesData and then call some --macro in extraParams.hxml to Context.defineType to alias haxe.io.Bytes and haxe.io.BytesData to them. That way the implementation will not leak into macro context.