ci - build-artifacts - generate sesify-viz for inspecting deps #7151
Conversation
|
Here is an example of the generated deps viewer https://105532-42009758-gh.circle-artifacts.com/0/build-artifacts/deps-viz/background/index.html its quite simple right now and I intend to improve it as we go |
|
Adds about 40sec (+4.5%) to test-passing builds. may be able to parallelize in future work. |
| }]) | ||
|
|
||
| // remove html comments that SES is alergic to | ||
| const removeHtmlComment = makeStringTransform('remove-html-comment', { excludeExtension: ['.json'] }, (content, _, cb) => { |
There was a problem hiding this comment.
So SES has some problem with HTML comments (even when they're embedded in JavaScript)? I'm be interested to hear more about whatever problem this is solving.
It should be harmless enough for this dependency visualization build I suppose
There was a problem hiding this comment.
I think it's related to this monster
1 + <!-- 2
3
// => 4I haven't encountered html comment openings in the wild, but I have encountered html comment closings --> as part of diagrams in comments. SES has a regex that throws if present.
This isn't actually needed for the viz tho, but for sesify build runtimes. Part of this PR is sneaking in all the components for sesify so that the PR to actually enable it in prod will be quite small.
There was a problem hiding this comment.
I think the main reason it's blacklisted is that it's behavior is not standardized
There was a problem hiding this comment.
Welp - that does seem monstrous.
Maybe there's a way to make this transformation more targeted then. I'm concerned that if this transformation was used as-is for a proper sesify build, it could break HTML markup containing comments that was inserted via JavaScript.
There was a problem hiding this comment.
Yeah that's a good concern. Maybe it should warn when it encounters one.
| @@ -229,6 +229,13 @@ jobs: | |||
| - store_artifacts: | |||
| path: test-artifacts | |||
| destination: test-artifacts | |||
| # important: generate sesify viz AFTER uploading builds as artifacts | |||
There was a problem hiding this comment.
I'm guessing this needs to be run after because otherwise it'd override the "real" build artifacts. I guess this is the main obstacle to running this in parallel - writing the output to a different location, so it can be done earlier.
There was a problem hiding this comment.
Correct. Could add a flag to build dist to a different dir. Though worth noting that all jobs past all-tests-pass are optional and don't block merges
|
The dependency visualization is neat, though I do wish there was a way to see the package information other than via tooltips. The visualization is great for the big picture but it's a bit tedious to look at each tooltip one at a time - maybe there's an easier way I don't know of. Where is the source for |
|
@Gudahtt yeah my next work will improve the explorer including the code base and distribution. The viz was thrown together for a hackathon and then again for a demo, so it's a hot mess. Gimme your feature requests. |
…evelop * origin/develop: (31 commits) Performance: Delivery optimized images (#7176) Add `appName` message to each locale Remove the disk store (#7170) Update @hapi/subtext as per security advisory (#7172) Add fixes for German translations (#7168) Fix recipient field of approve screen (#7171) 3box integration 2.0 (#6972) ci - metamaskbot - include links to dep-viz and all artifacts (#7155) Replace `undefined` selectedAddress with `null` (#7161) Add polyfill for AbortController (#7157) Remove redundant error logging (#7158) Set minimum Firefox version to v56.2 to support Waterfox (#7156) ci - install deps with "--har" flag to capture network activity (#7143) ci - create source-map-explorer build-artifacts (#7141) ci - build-artifacts - generate sesify-viz for inspecting deps (#7151) Publish GitHub release from master branch (#7136) fix rinkeby spelling (#7148) deps - move gulp-terser-js to devDeps test:integration - fix renamed test data file lint fix ...
The `build-artifacts` directory is created on CI (as of #7151), and is used to store the sesify visualization and dependency logs. It's useful to have this ignored locally as well, for when those scripts are being tested.
The `build-artifacts` directory is created on CI (as of #7151), and is used to store the sesify visualization and dependency logs. It's useful to have this ignored locally as well, for when those scripts are being tested.
The `build-artifacts` directory is created on CI (as of #7151), and is used to store the sesify visualization and dependency logs. It's useful to have this ignored locally as well, for when those scripts are being tested.
note:
This is a step towards adding sesify/lavamoat integration into
develop.This does not add sesify to our builds.
Using some tools developed for sesify/lavamoat, we add a dependency explorer for each build. While still early, this tool helps give us an idea of what is consuming what, and what platform api it seems to be consuming. Note that the platform usage is not enforced without integrating sesify/lavamoat.