improve: wasm build #3074

Uzlopak · 2024-04-09T00:35:30Z

should shrink the wasm file. removes build artifacts and other files we dont need.

build/wasm.js

codecov-commenter · 2024-04-09T00:43:14Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.06%. Comparing base (7a94682) to head (4a8f4a7).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3074   +/-   ##
=======================================
  Coverage   94.06%   94.06%           
=======================================
  Files          89       89           
  Lines       24327    24331    +4     
=======================================
+ Hits        22883    22887    +4     
  Misses       1444     1444

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Uzlopak · 2024-04-09T00:46:45Z

@mcollina

Can you please this PR on your special benchmarking PC please :)?

mcollina · 2024-04-09T06:19:58Z

how does it shrink the wasm file?

Uzlopak

@mcollina

By using -04 we optimize the wasm further.

Uzlopak · 2024-04-13T12:47:08Z

build/wasm.js

 // These are relevant for undici and should not be overridden
 WASM_CFLAGS += ' -Ofast -fno-exceptions -fvisibility=hidden -mexec-model=reactor'
 WASM_LDFLAGS += ' -Wl,-error-limit=0 -Wl,-O3 -Wl,--lto-O3 -Wl,--strip-all'
 WASM_LDFLAGS += ' -Wl,--allow-undefined -Wl,--export-dynamic -Wl,--export-table'
 WASM_LDFLAGS += ' -Wl,--export=malloc -Wl,--export=free -Wl,--no-entry'

+const WASM_OPT_FLAGS = '-O4 --converge --strip-debug --strip-dwarf --strip-producers'


@mcollina

This is where we define O4 for optimizing the wasm file.

mcollina

lgtm

mochaaP · 2024-05-21T18:28:35Z

Hi @Uzlopak,

The external wasm codepath was used by Fedora due to our package policy. This is blocking us from updating to v6.14.0 and onwards.

If you think some code / files is redundant, please explain it further to me and I'll be glad to update my code!

Uzlopak · 2024-05-21T19:36:47Z

What do you mean with external wasm codepath?

mochaaP · 2024-05-22T06:09:42Z

build/wasm.js

-  writeFileSync(join(WASM_OUT, 'wasm_build_env.txt'), buildInfo)
-}
-
-const writeWasmChunk = EXTERNAL_PATH
-  ? (path, dest) => {
-      const base64 = readFileSync(join(WASM_OUT, path)).toString('base64')
-      writeFileSync(join(WASM_OUT, dest), `
-const { Buffer } = require('node:buffer')
-
-module.exports = Buffer.from('${base64}', 'base64')
-`)
-    }
-  : (path, dest) => {
-      writeFileSync(join(WASM_OUT, dest), `
-const { fs } = require('node:fs')
-
-module.exports = fs.readFileSync(require.resolve('./${basename(path)}'))
-`)
-    }
-
-// Build wasm binary
-execSync(`${WASM_CC} ${WASM_CFLAGS} ${WASM_LDFLAGS} \
- ${join(WASM_SRC, 'src')}/*.c \
- -I${join(WASM_SRC, 'include')} \
- -o ${join(WASM_OUT, 'llhttp.wasm')} \
- ${WASM_LDLIBS}`, { stdio: 'inherit' })
-
-writeWasmChunk('llhttp.wasm', 'llhttp-wasm.js')
-
-// Build wasm simd binary
-execSync(`${WASM_CC} ${WASM_CFLAGS} -msimd128 ${WASM_LDFLAGS} \
- ${join(WASM_SRC, 'src')}/*.c \
- -I${join(WASM_SRC, 'include')} \
- -o ${join(WASM_OUT, 'llhttp_simd.wasm')} \
- ${WASM_LDLIBS}`, { stdio: 'inherit' })
-
-writeWasmChunk('llhttp_simd.wasm', 'llhttp_simd-wasm.js')
-
-if (EXTERNAL_PATH) {
-  writeFileSync(join(ROOT, 'loader.js'), `
-'use strict'
-
-globalThis.__UNDICI_IS_NODE__ = true
-module.exports = require('node:module').createRequire('${EXTERNAL_PATH}/loader.js')('./index-fetch.js')
-delete globalThis.__UNDICI_IS_NODE__
-`)


^ I meant this.

Ok, then it has to be added back + a big fat comment to not remove it because of fedora

Uzlopak added 2 commits April 9, 2024 02:33

improve: wasm build

1695a15

Merge branch 'main' into wasm-builder

211e1d5

github-advanced-security bot found potential problems Apr 9, 2024

View reviewed changes

build/wasm.js Dismissed Show dismissed Hide dismissed

build/wasm.js Dismissed Show dismissed Hide dismissed

build/wasm.js Dismissed Show dismissed Hide dismissed

build/wasm.js Dismissed Show dismissed Hide dismissed

Merge branch 'main' into wasm-builder

4a8f4a7

Uzlopak commented Apr 13, 2024

View reviewed changes

mcollina approved these changes Apr 22, 2024

View reviewed changes

mcollina requested a review from ronag April 22, 2024 13:10

mcollina merged commit 3f9e147 into nodejs:main Apr 22, 2024
26 checks passed

github-actions bot mentioned this pull request Apr 22, 2024

[Release] v6.14.0 #3147

Merged

Uzlopak deleted the wasm-builder branch April 22, 2024 13:31

mochaaP reviewed May 22, 2024

View reviewed changes

renovate bot mentioned this pull request Jun 11, 2024

chore(deps): update all non-major dependencies unjs/node-fetch-native#123

Open

1 task

mhdawson mentioned this pull request Jun 19, 2024

Ability to externalize WASM was broken #3345

Closed

richardlau mentioned this pull request Jul 26, 2024

fix: restore externalized Node.js dep compatibility #3421

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

improve: wasm build #3074

improve: wasm build #3074

Uzlopak commented Apr 9, 2024

codecov-commenter commented Apr 9, 2024 •

edited

Loading

Uzlopak commented Apr 9, 2024

mcollina commented Apr 9, 2024

Uzlopak left a comment

Uzlopak Apr 13, 2024

mcollina left a comment

mochaaP commented May 21, 2024

Uzlopak commented May 21, 2024

mochaaP May 22, 2024

Uzlopak May 22, 2024

improve: wasm build #3074

improve: wasm build #3074

Conversation

Uzlopak commented Apr 9, 2024

codecov-commenter commented Apr 9, 2024 • edited Loading

Codecov Report

Uzlopak commented Apr 9, 2024

mcollina commented Apr 9, 2024

Uzlopak left a comment

Choose a reason for hiding this comment

Uzlopak Apr 13, 2024

Choose a reason for hiding this comment

mcollina left a comment

Choose a reason for hiding this comment

mochaaP commented May 21, 2024

Uzlopak commented May 21, 2024

mochaaP May 22, 2024

Choose a reason for hiding this comment

Uzlopak May 22, 2024

Choose a reason for hiding this comment

codecov-commenter commented Apr 9, 2024 •

edited

Loading