Skip to content

Commit

Permalink
Include artefact sizes in printout (#3308)
Browse files Browse the repository at this point in the history
# Motivation
We would like to monitor Wasm sizes. We agreed that it might make sense
to include the Wasm size printout that we currently have for merges to
main in the nightly build, so that we would have a place where we can
get the size periodically.
([GIX-1857](https://dfinity.atlassian.net/browse/GIX-1857)) It turned
out, though, that the nice printout we have when the build job is run is
not readily accessible in a programmatic way. We actually have plenty of
runs, the trouble is access to the data.

# Changes
* Log Wasm sizes.
* Include the list of Wasm sizes in release artefacts.

Before the assets printout looked like this:
```
02e20e8e4313facfd0b6caa0e7eca0c453c1884b878bce0e52611b1709dbc193  out/sns_aggregator.wasm.gz
e90b508eb38de35f2a836f610b12bd12e26432bf3ba98a92e5a76c08257891ef  out/nns-dapp.wasm.gz
bbd2d0d6e4301fe863b7255b4979dff4c56d8b94812ab1d39d4cf9ea844b8e15  out/deployment-config.json
ac35b41554d819694c8b9faf052a5368b2cd0630b34f05a4f96e317d017c1d68  out/nns-dapp-arg-mainnet.bin
2bf5c0ff8e4e6e299a166744a61ecb90dc7031bf3cc6c1ffb261c2b869c3d6a0  out/nns-dapp_noassets.wasm.gz
c5b6d21dca7235531d3b1730f774e7a61d3025b6c15ce8f3352f9cff79e6c900  out/nns-dapp-arg-mainnet.did
74691d9e093ec6fc5462ee69b65b8a0a241ffd1fd3d5c36ce5aa5e795f29e5d7  out/frontend-config.sh
97773a6a45a34f171bc22034f9a8d0aeab4cc50c827de5a9de670358ec1bf7c8  out/sns_aggregator_dev.wasm.gz
2ec5cf46caba0a6c1c96ebaff90d1d03fbd340501579f3f26129023d640d679c  out/nns-dapp.wasm
a0d46f20f6bab723c3b5d5ce2c847b14a4513eee708b65759362bf895d15673d  out/sourcemaps.tar.xz
9dc6dfbbce074cab1d669811bcb7177add19255121f4e31ff3bb1e729e509203  out/assets.tar.xz
```
After:
```
===START_ARTEFACTS===
sha256                                                            size     filename
02e20e8e4313facfd0b6caa0e7eca0c453c1884b878bce0e52611b1709dbc193  334848   out/sns_aggregator.wasm.gz
e90b508eb38de35f2a836f610b12bd12e26432bf3ba98a92e5a76c08257891ef  3077987  out/nns-dapp.wasm.gz
bbd2d0d6e4301fe863b7255b4979dff4c56d8b94812ab1d39d4cf9ea844b8e15  1152     out/deployment-config.json
ac35b41554d819694c8b9faf052a5368b2cd0630b34f05a4f96e317d017c1d68  988      out/nns-dapp-arg-mainnet.bin
2bf5c0ff8e4e6e299a166744a61ecb90dc7031bf3cc6c1ffb261c2b869c3d6a0  840468   out/nns-dapp_noassets.wasm.gz
c5b6d21dca7235531d3b1730f774e7a61d3025b6c15ce8f3352f9cff79e6c900  1446     out/nns-dapp-arg-mainnet.did
74691d9e093ec6fc5462ee69b65b8a0a241ffd1fd3d5c36ce5aa5e795f29e5d7  942      out/frontend-config.sh
97773a6a45a34f171bc22034f9a8d0aeab4cc50c827de5a9de670358ec1bf7c8  334621   out/sns_aggregator_dev.wasm.gz
2ec5cf46caba0a6c1c96ebaff90d1d03fbd340501579f3f26129023d640d679c  5098433  out/nns-dapp.wasm
a0d46f20f6bab723c3b5d5ce2c847b14a4513eee708b65759362bf895d15673d  1388352  out/sourcemaps.tar.xz
9dc6dfbbce074cab1d669811bcb7177add19255121f4e31ff3bb1e729e509203  2237816  out/assets.tar.xz
==END_ARTEFACTS==
```

# Tests
* See CI:
### The section with the assets can be extracted from any run:
```
gh run view --job 16715948272 --log | sed -nr '/ ===START_ARTEFACTS===/,/ ==END_ARTEFACTS==/{p}'
build	Build nns-dapp repo	2023-09-12T13:21:05.0027820Z ===START_ARTEFACTS===                                                      
build	Build nns-dapp repo	2023-09-12T13:21:05.0028278Z sha256                                                            size     filename
build	Build nns-dapp repo	2023-09-12T13:21:05.0029059Z ab2315619bddd0e7752a944fa7dd294dee9ee1c380b1e6d83ba29f8054eb303e  1414     out/nns-dapp-arg-mainnet.did
build	Build nns-dapp repo	2023-09-12T13:21:05.0029920Z cad1c87519603a9bcb6eaae507339226d6738793f4766695939fb3330cc60218  1116     out/deployment-config.json
build	Build nns-dapp repo	2023-09-12T13:21:05.0030526Z e31ef45bb051a494c454e7fdf9208878a3949fbc5b1e38875e900569c9b3e86e  1403580  out/sourcemaps.tar.xz
[SNIP]
2d1c919d9dd9efc2cfa7ab28155014ae0e7945786694fa7be635d37bb6f8e315  354199   out/sns_aggregator_dev.wasm.gz
build	Build nns-dapp repo	2023-09-12T13:21:05.0036246Z ==END_ARTEFACTS=
```
### The production wasm size can be extracted from the logs
With something like:
```
grep -Eo '[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9:.]+Z [a-f0-9]{64} +[0-9]+ +\S+' | grep out/nns-dapp_production.wasm.gz | awk '{print $3}'
```
So, some work is needed but as we have timestamp and wasm size available
for every run we can make charts, dashboards and so on.

# Todos

- [x] Add entry to changelog (if necessary).


[GIX-1857]:
https://dfinity.atlassian.net/browse/GIX-1857?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: Max Murphy <[email protected]>
  • Loading branch information
bitdivine and bitdivine authored Sep 13, 2023
1 parent 3eb8d98 commit 791ced7
Show file tree
Hide file tree
Showing 3 changed files with 9 additions and 2 deletions.
8 changes: 7 additions & 1 deletion .github/actions/build_nns_dapp/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -82,4 +82,10 @@ runs:
DFX_NETWORK=local ./config.sh && cp nns-dapp-arg-local.did out && didc encode "$(cat out/nns-dapp-arg-local.did)" | xxd -r -p >out/nns-dapp-arg-local.bin
- name: Hash artefacts
shell: bash
run: find out/ -type f | xargs sha256sum
# Note:
# * sha256sum prints the hash of each file;
# * The `wc -c` in the awk code gets the size in bytes of each file;
# * The AWK code also adds a header and footer;
# * `column -t` formats the data as a table;
# * `tee` saves a copy of the printout in a file; that file will be included in releases.
run: find out/ -type f | xargs sha256sum | awk '{"wc -c " $2 | getline size; print $1, size}BEGIN{print "===START_ARTEFACTS===\nsha256 size filename"}END{print "==END_ARTEFACTS=="}' | column -t | tee out/filelist.txt
2 changes: 1 addition & 1 deletion .github/actions/release_nns_dapp/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ runs:
cd "${{ inputs.assets_dir }}"
daily_build_name="nns-dapp-$(git rev-parse HEAD).wasm.gz"
cp nns-dapp.wasm.gz "$daily_build_name"
artefacts=(nns-dapp.wasm.gz nns-dapp_noassets.wasm.gz nns-dapp_production.wasm.gz nns-dapp_test.wasm.gz sns_aggregator.wasm.gz sns_aggregator_dev.wasm.gz assets.tar.xz "$daily_build_name" nns-dapp-arg-{mainnet,local}.{did,bin})
artefacts=(nns-dapp.wasm.gz nns-dapp_noassets.wasm.gz nns-dapp_production.wasm.gz nns-dapp_test.wasm.gz sns_aggregator.wasm.gz sns_aggregator_dev.wasm.gz assets.tar.xz "$daily_build_name" nns-dapp-arg-{mainnet,local}.{did,bin} filelist.txt)
ls -l "${artefacts[@]}"
for tag in $(git tag --points-at HEAD) ; do
: Creates or updates a release for the tag
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG-Nns-Dapp.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ The NNS Dapp is released through proposals in the Network Nervous System. Theref

#### Changed
* Specify the `snsdemo` version in `dfx.json`.
* Make the file list stats accessible in logs and as a file in the release artefacts.
* Support using the nns-dapp downgrade-upgrade test with Wasms other than prod.

#### Deprecated
Expand Down

0 comments on commit 791ced7

Please sign in to comment.