Switch from Reify to SWC for ensuring compatability with older browsers.

[ILOVEPIE]

Description

Created a utility that runs esbuild and SWC to build the library.
This supports a compat profile that works as far back as safari 8, maybe earlier.

This closes #619

Motivation and Context

The library is limited in compatibility by the number of new features we are using, this PR allows us to keep using those features while dropping reify and also supporting even older browsers.

How Has This Been Tested?

I tested the compat outputs using Browser Stack on multiple old browsers including safari 8 and some ancient versions of chrome. I also tested the other builds using more modern browsers.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• I did npm run test and all tests passed green (including code styling checks).
• I have added tests to cover my changes.
• My change requires a change to the documentation.
• I have updated the README accordingly.
• I have read the CONTRIBUTING document.

[ILOVEPIE]

@yne does this look OK or do we need to make changes?

[yne]

to be honest

• I think it's a step in the wrong direction because I expect the opentype v2 to move to a modern stack with less complexity
• If you want to use older browser, use older opentype.js
• WebKit/apple are well known for being anti-web so I doubt they users (who exactly ?) expect anything
• This commit add even more build script to maintain which I'm fighting against ( eslint-local-rules.js I'm also looking at you)
• esbuild support transpiling to older ES version https://esbuild.github.io/api/#supported and I don't see any explanation why it is not used
• This commit remove mocha tests ?!

So no, I won't accept it.

[ILOVEPIE]

to be honest

• I think it's a step in the wrong direction because I expect the opentype v2 to move to a modern stack with less complexity
• If you want to use older browser, use older opentype.js
• WebKit/apple are well known for being anti-web so I doubt they users (who exactly ?) expect anything
• This commit add even more build script to maintain which I'm fighting against ( eslint-local-rules.js I'm also looking at you)
• esbuild support transpiling to older ES version https://esbuild.github.io/api/#supported and I don't see any explanation why it is not used
• This commit remove mocha tests ?!

So no, I won't accept it.

The problem is older versions of OpenType.js don't work properly. Also, we already discussed all this in #619.

Also, esbuild does not support transpiling. It only errors when you use a feature that isn't supported by the target. I did a lot of research into this. Also, old versions of Safari are not the only thing targeted by the compat profile. It also supports old versions of Firefox and Chrome and old browsers like Internet Explorer. While you may think there is no use in supporting older browsers, I think there is a large amount of developers out there who work with legacy code who would appreciate being able to use a modern library. Also, do you know how many modern smart TVs are still running old versions of WebKit? Nearly all of them. So if somebody wanted to use the library in a smart TV app, then they're SOL without this. I'm not suggesting we make this the default version of opentype. I'm just saying it's okay to support older browsers for those who need it. And besides, this allows us to drop reify for a dependency that's actually up to date: SWC.

Hell, I'll even take over complete responsibility for maintaining the compat profile if you're that against including the profile.

It's not like I'm asking for Netscape Navigator support.

@Jolg42 & @Connum should weigh in with their opinions before a decision is made.

[ILOVEPIE]

• This commit remove mocha tests ?!

Yeah, that was a mistake. I just saw the word "reify" and removed it. I was pretty tired at the time.

[yne]

Also, esbuild does not support transpiling. It only errors when you use a feature that isn't supported by the target. I did a lot of research into this.

I'm confused, the esbuild documentation state that

[--target] tells esbuild to transform JavaScript syntax that is too new for these environments into older JavaScript syntax that will work in these environments. For example, the ?? operator was introduced in Chrome 80 so esbuild will convert it into an equivalent (but more verbose) conditional expression when targeting Chrome 79 or earlier.

So I would have expected some kind of rational/justification explaining why esbuild was not the chosen solution.

I'm not against allowing users to build a "compat" version of opentype.js. I'm against complexifing our codebase/buildflow for that.
if it would only have required a --target=es5 to the esbuild I wouldn't have minded.

[yne]

also, the increasing number of variability that we have to test may require some re-focus

• esm/cjs
• low-memory
• min
• legacy

sum up to 16 possible build profile combination to test

maybe low memory and legacy are for the same purpose ?

I hope that even if we disagree on this subject you'll still ping me some other times 😀

[ILOVEPIE]

Also, esbuild does not support transpiling. It only errors when you use a feature that isn't supported by the target. I did a lot of research into this.

I'm confused, the esbuild documentation state that

[--target] tells esbuild to transform JavaScript syntax that is too new for these environments into older JavaScript syntax that will work in these environments. For example, the ?? operator was introduced in Chrome 80 so esbuild will convert it into an equivalent (but more verbose) conditional expression when targeting Chrome 79 or earlier.

So I would have expected some kind of rational/justification explaining why esbuild was not the chosen solution.

I'm not against allowing users to build a "compat" version of opentype.js. I'm against complexifing our codebase/buildflow for that.
if it would only have required a --target=es5 to the esbuild I wouldn't have minded.

esbuild does transform some syntax but it's a very limited transform and it's not as thorough as SWC. Also SWC does a code optimization pass to make the code smaller and more performant by rearranging elements of the code.

Also:

Note that this is only concerned with syntax features, not APIs. It does not automatically add polyfills for new APIs that are not used by these environments. You will have to explicitly import polyfills for the APIs you need (e.g. by importing core-js). Automatic polyfill injection is outside of esbuild's scope.

SWC does automatic ponyfill injection. So for example we have usages of map that are not supported in older browsers because map was introduced later on. SWC fixes this automatically by injecting a ponyfill of Array.prototype.map that gets used if the function is not available in the browser. It also only injects the ponyfills needed to target a specific browser version at a minimum. So what it does is it only injects the minimum number of ponyfills needed for that target.

[ILOVEPIE]

The reason I was talking about the erroring was because you originally only linked the supported flag, which is not the same as the target flag.

[Connum]

I actually like this PR, and as @ILOVEPIE stated, we already had a discussion about it. Getting rid of reify will finally allow us to use modern syntax and language features, such as private class fields and methods. iirc it was also holding us back from switching upgrading to the most recent version of mocha. So it actually is a step to more modern development. SWC is also a modern build tool, and many active projects provide legacy versions (may using a "hail Mary" approach and not even running the tests on them again) so I frankly don't get that argument at all. 😄

I don't really care about increased build times, and the complexity is no really increasing that much. There are still so many projects relying on webpack, babel and a great number of plugins for them... So what we'd have now is already way better than that.

Regarding the local eslint rules you mentioned: these are for specific performance improvements that we found made an impact on the library, and there were no existing packages for them. You write them once, nothing to maintain except extending them maybe, they're only used for lining, so I don't really see an issue with that either. I think it's even better to have control over them instead of having to rely on yet another third party package.

So I'm very happy to see this happening and I'm all for it!

[Connum]

Furthermore, I strongly believe that built complexity or maintenance never was and never will be the reason why opentype.js development is slow. It's our work and private lifes and the lack of sponsorship. 😉

[yne]

If this allows removing the barrier on mocha then I'll be for the best. ✔️

Is the Closure Compiler externs file still relevant once transpiling will work ?

Echoing to #675 and the last line of #693 (comment) I think one day we shall have a clear view of what opentype.js vision is: "sleek & modern" or "battery-included works-everywhere " (kind of a mac vs windows)

[ILOVEPIE]

If this allows removing the barrier on mocha then I'll be for the best. ✔️

Is the Closure Compiler externs file still relevant once transpiling will work ?

Echoing to #675 and the last line of #693 (comment) I think one day we shall have a clear view of what opentype.js vision is: "sleek & modern" or "battery-included works-everywhere " (kind of a mac vs windows)

the two approaches aren't mutually exclusive. That closure compiler externs file is outdated and needs fixes, it basically tells google closure compiler what our API is so it doesn't get mangled or optimized away when people using closure compiler use opentype.js. Closure compiler is like SWC's less hip but way stronger older brother. I opted for SWC because it's newer but that doesn't mean our users will be using it over closure compiler.

[yne]

• This commit remove mocha tests ?!

Yeah, that was a mistake. I just saw the word "reify" and removed it. I was pretty tired at the time.

@ILOVEPIE : I've cloned your branch and re-enabled unit-tests (so without --reify) and it failed.

Did you created the PR without running the unit-test ? Or where you using a solution that you didn't commited ?

If the later (which I really hope), could you share it because the only solution we had for now was to simply declare module:true in package.json but you refused it as too modern (because it require NodeJS v15.3 from 2020?)

Also, I setup SWC from scratch by

• adding "@swc/cli": "^0.3.12" in "devDependencies"
• adding "compat": "swc -C jsc.target=es3 ./dist/opentype.js -o ./dist/opentype.compat.js", in script

and it worked.

So what was the motivation for creating all those swc wrapper and why was this wrapper even used on non compat builds ?

[ILOVEPIE]

There was a few issues with the PR. I'm going to fix them later today. I was pretty tired when I finalized the PR and didn't fully take everything into account. The reason why I have the wrapper is because it gives us finer control over the targeting of the builds and also allows us to specifically enable or disable features of SWC that are causing issues in a specific build etc. The solution for fixing the tests I'm probably going to go with is renaming the source files to .mjs because that should let Mocha run them properly. Unless you want me to make a wrapper that runs mocha (just kidding). Also SWC assumes that it's output will be inside a IIFE or module, which means it's better to apply it before bundling.

[ILOVEPIE]

Also the reason why it was used on non-compat builds was because that way the code would be ES6 compatible even if new features are added and we use them. For example, the spread operator is not ES6 compatible, but we can use it with SWC and still be ES6 compatible.

[ILOVEPIE]

I fixed some of the test stuff to work better with modules, but I'm still getting a timeout with the HTTP and HTTPS modules.

[ILOVEPIE]

This also closes #692 as it required reworking the same stuff.

[ILOVEPIE]

I fixed the issues, @yne if you want to take a look, this PR is now using latest mocha.

[ILOVEPIE]

@Connum @yne This PR has been reworked.

[ILOVEPIE]

@Connum @yne When either of you get the chance can you do a review of this PR?

[yne]

when this PR will be rebased :)

[ILOVEPIE]

when this PR will be rebased :)

It's already been re-based. The only thing that hasn't been updated is the package JSON.

[yne]

I'm asking because there are 7300+ line added in 55 files... which is not just the kind of "I've added 5 line in package.json to build with reify" PR I was expecting...

All thoses changes are mandatory for reify to work ?

[ILOVEPIE]

I'm asking because there are 7300+ line added in 55 files... which is not just the kind of "I've added 5 line in package.json to build with reify" PR I was expecting...

All thoses changes are mandatory for reify to work ?

I'll go back through it and try and simplify, but we're not adding reify. We're removing reify and adding support for SWC for version compatibility.

[Connum]

I don't understand why the mjs files are listed as completely new and there are no deletions of their counterparts listed (which should be evaluated as renaming anyway). I'm on my phone though, so maybe I missed something.

Also the build fails, which needs to be fixed before we can start the review.

edit: and there are also conflicts with the main branch, so that's probably why @yne said it's not rebased yet.

[yne]

I'm asking because there are 7300+ line added in 55 files... which is not just the kind of "I've added 5 line in package.json to build with reify" PR I was expecting...
All thoses changes are mandatory for reify to work ?

I'll go back through it and try and simplify, but we're not adding reify. We're removing reify and adding support for SWC for version compatibility.

Sorry I was tired, I meant SWC.
What a week huh?

[ILOVEPIE]

Should be fixed now.

[ILOVEPIE]

Now it's fixed.
Edit: It appears I made an error with my modifications to the build script, I'll ammend the last commit.

[yne]

There is still a new test/tables/gasp.mjs which shall be test/tables/gasp.spec.mjs

[ILOVEPIE]

Finally i got the missing dependency, polyfill issues, tests and eslint errors fixed and working.

[yne]

I'm still unsure about some part especially the coreJs part and the argument parsing, but that's could be improved later (if any)

but the overall gain (being able to use any ES feature without being scared of breaking some old browser/node) worth it.

[ILOVEPIE]

There are a few code cleanup changes that I'd like to make before we actually go through with this.

[ILOVEPIE]

I will probably get some time to make those code cleanup changes on Saturday. I"ve been a bit busy between work and preparing to give a workshop on taking over control of other running processes in both Windows and Linux at DEFCON 32 in Las Vegas.