New asys APIs

[Aurel300]

Rendered version

See implementation draft.

Changelog:

• 23. 5. - changed paths to use FilePath abstract
• 27. 6. - changed Event to Signal
• 11. 9. - changed to separate asys package, updated feature set

[RealyUniqueName]

I don't like the idea of storing a specially encoded file names in a plain string. It's a potential source of errors and confusion, because any string in an application can suddenly become an encoded gibberish instead of a valid string.
At least let's make it something like this:

abstract FilePath(String) from String {
  @:from static public function encode(bytes:Bytes):FilePath;
  public function decode():Bytes;
}

//and then in sys api:
static public function readDirectory(path:FilePath):Array<FilePath>;

//usage
readDirectory('someDir');
readDirectory(nonUnicodeBytes);

And that FilePath could potentially make concatenating/splitting paths easier if provided with an appropriate API.

[Gama11]

A FilePath abstract would also be awesome simply because of the additional semantic information it carries, very much in the spirit of @nadako's Abstracting Primitives blog post. This would be very useful for improved IDE support:

If we're inside of a string literal in a place where a FilePath is expected, we could provide auto-completion from the file system after path separators like / and \ . This is what for instance the VSCode LaTeX extension does for commands like \includegraphics:

[gene-pavlovsky]

You've done a big work, I would like to know your thoughts behind some of the design changes.
Why did you choose to create your own events and callbacks, rather than using already popular and proven solutions (such as tink's signals and promises)?
Btw, have you researched https://github.com/benmerckx/asys ?

Haxe has a lot of other targets besides Node.JS, do we really want to "copy" Node's API?

[Aurel300]

Why did you choose to create your own events and callbacks, rather than using already popular and proven solutions (such as tink's signals and promises)?

My original idea for these APIs was to have the promise variants only. I personally usually use async / await when using Node.js, and I thought it would be best if we also exposed promises as the unified API and later wrapped it in coroutines.

After some discussions (some of which are in the #58 promises proposal), I decided it is indeed best not to go for promises and try to have as minimal of an API as needed for asynchronous operations, which is simply callbacks.

Some good arguments against A+ promises (the standard Javascript ones) are here. We could fix some of these without changing the interface too much, but some are really tricky. The "Eager, not lazy" problem as described in the article, for example, cannot be solved without having a .run-type method. Adding such a method would not be a big deal, but it would completely miss the point of having an interface similar to A+ and it would only cause confusion for people familiar with A+ promises. Additionally A+ promises suffer from type parameter issues, although that is a problem that I could also only "sidestep" by introducing haxe.NoData + the @:from methods in haxe.async.Callback.

Regarding any other style of promises – it is easy for a library to provide a façade over a callback-style API and expose promises instead. The opposite much less so – if you wanted callbacks and only had a promise-based API, you could make it work, but it would be very wasteful, since under the hood the promises would still need to be allocated, garbage collected, etc etc.

Regarding tink specifically – I acknowledge tink is a good set of libraries and a lot of work has been put into them. I am sure the asynchrony primitives in tink are well thought out. However, tink is far from atomic, and introducing tink asynchrony into the Haxe standard library would introduce a lot of new concepts that are very specific to tink (Surprise, Noise, Outcome ...?).

So instead this API has the bare minimum needed, making it easy for libraries to wrap it however you want. Both callbacks and events are abstracts, to at runtime callbacks are simply functions with a specific function signature (always error first), and events are just an array of event handlers. There may be issues with both (please comment if you think of specifics!) but I don't think simply replacing it with a "popular and proven solution" would be the correct way to go.

Btw, have you researched https://github.com/benmerckx/asys ?

I have not until now :) I can see it uses tink promises and futures. More importantly, it is just a wrapper which calls the asynchronous versions of functions on Node.js, or uses a runloop (queued synchronous operations) for any other target.

The point of this proposal is to provide an interface for asynchronous operations which will be implemented properly (i.e. using native asynchronous calls) on all sys targets.

Haxe has a lot of other targets besides Node.JS, do we really want to "copy" Node's API?

While I mostly cite Node.js as the reference API here, the actual functions come from libuv. Libuv is a C library, so its API would feel extremely clunky if taken directly into Haxe. Javascript on the other hand has a lot of similarities to Haxe, and the Node.js APIs are simply a very nice way to bring the libuv APIs to Javascript – hence they make good references for Haxe APIs.

Very importantly, it would be best to avoid re-inventing the wheel on all targets when implementing these APIs. Simply using libuv on as many sys targets as possible would be ideal.

[benmerckx]

Btw, have you researched https://github.com/benmerckx/asys ?

Asys is more of a crutch until something like this comes along anyway :) We can easily create a new wrapper that offers all of the async std api using tink's Promises (as mentioned).

[nanjizal]

I don't see any mention of AIR api was it studied and considered, I am not really following what Adobe are doing with AIR. But even quite recently Heaps was using AIR so old code may still need maintaining at the very least? See some AIR sys stuff here in relation to your proposal:
https://help.adobe.com/en_US/as3/dev/WS5b3ccc516d4fbf351e63e3d118666ade46-7fe4.html

Event would clash with https://api.openfl.org/openfl/events/Event.html and with Flash target which is still in use, while you can clarify with packages paths this is clumsy for instance dispatching a new type Event from a Sprite? Would SignalEvent be more suitable term? Certainly Signals seems to be a very common term familar with many even in Robert Penners classic book on Signals and Tweens ( tweens are now everywhere but largely they became popular due to the as3 book ).
https://github.com/robertpenner/as3-signals
To me Event seems rather strangely use of the keyword in your example?

Error does this keyword have implications, if it contains data then would ErrorData, or similar be more suitable and reduce any confusion with native targets?

[nanjizal]

Well irrespective of if it could require patches for flash/air a target that you are unlikely to want to fix, I still wonder if Signal is not a more correct term in the wider industry ( outside of whatever node does ) for the use illustrated. Certainly it looked like a simple Signal to me.

[Aurel300]

But even quite recently Heaps was using AIR so old code may still need maintaining at the very least?

Yes, there is still Flash- and AIR-specific code in Heaps, and maybe other Haxe libraries. Since they use native Flash classes, this proposal does not affect them at all. Even libraries which use the current "pure Haxe" system APIs will not be affected, since the old APIs will not even be deprecated for some time.

Event would clash with https://api.openfl.org/openfl/events/Event.html and with Flash target which is still in use, while you can clarify with packages paths this is clumsy for instance dispatching a new type Event from a Sprite?

I don't think this is much of a problem. This would cause issues with existing code only if there was an import haxe.async.*, which is a package that is new to the stdlib. Using the events
provided by the sys APIs does not require an import of haxe.async.Event.

Would SignalEvent be more suitable term?

Well, technically, haxe.async.Event is a signal, I suppose. SignalEvent is just plain confusing. The semantics/use cases of signals and events are the same, which is why I don't think it is wrong to call the class an event. The only distinction as far as I can see is the identifier and type safety?

Error does this keyword have implications, if it contains data then would ErrorData, or similar be more suitable and reduce any confusion with native targets?

Please clarify. Why would it introduce confusing with native targets? It is also in the haxe package.

[Gama11]

A thought on the naming convention of event fields: using event as a prefix rather than a suffix sounds pretty awkward to me. For instance, I think eventClose should be named closeEvent instead.

[tienery]

From what I can tell, these async versions are a complete replacement of the current synchronise options? Please tell me I'm wrong, because I'm in the mind that constantly using async everywhere is a bad idea, and should only be used for intensive tasks on targets or situations where multi-threading is not possible.

[Aurel300]

@tienery No, there are both synchronous and asynchronous versions in this proposal. They are meant to eventually replace the current APIs which are only synchronous.

[Simn]

New sys API as a whole has been accepted. Not every detail here is necessarily what we'll end up with 100%, but it still makes sense to merge this as-is.