Small-artifact bloat reports

#### Problem The current bloat report requires preserving large binaries, and has trouble matching tree parents for comparison. #### Change overview This change makes example builds generate small artifacts containing json file(s) containing build item identification and section sizes. Artifacts names have the form “Size,PR,WORKFLOW,CURRENT_SHA,PARENT_SHA”; these contain one or more json files, each containing a report on one build target. The reporting scripts then examines the list of available artifacts to list of artifacts to identify pairs with the same $WORKFLOW where one's $CURRENT_SHA is the other's $PARENT_SHA, and only then downloads and processes those artifacts to generate comparisons for matching builds. - Changes to `examples-…` workflows: - Adds some `GH_EVENT_…` variables to workflow `env`, to identify the current run. - Uses the `gh_sizes.py` script to generate size report json files. - Uploads size report artifacts. - Changes to `scripts/tools/memory`: - Adds minimal platform config files for recently-added platforms. - Adds markdown output options (required for github comments) along with some associated cleanup. - Adds a script `scripts/tools/memory/gh_sizes.py` for use by workflows; this is similar to `report_summary.py` with a suitable consistent set of arguments. - Adds a script `scripts/tools/memory/gh_report.py` to analyze size report artifacts. - Modifies `bloat_check.py` to ignore the size report artifacts. Note that this PR does _not_ include a change to actually send size report comments on github; this is left to a followup after that code has been verified on real-world size artifacts. #### Testing Fork CI and offline runs on resulting artifacts.
project-chip · Sep 7, 2021 · 379650a · 379650a
1 parent 9b976c0
commit 379650a
Show file tree

Hide file tree

Showing 27 changed files with 1,722 additions and 182 deletions.
diff --git a/.github/workflows/examples-efr32.yaml b/.github/workflows/examples-efr32.yaml
@@ -30,6 +30,9 @@ jobs:
         env:
             EFR32_BOARD: BRD4161A
             BUILD_TYPE: gn_efr32
+            GH_EVENT_PR: ${{ github.event_name == 'pull_request' && github.event.number || 0 }}
+            GH_EVENT_HASH: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+            GH_EVENT_PARENT: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
 
         runs-on: ubuntu-latest
         if: github.actor != 'restyled-io[bot]'
@@ -57,25 +60,29 @@ jobs:
                    .environment/pigweed-venv/*.log
             - name: Build example EFR32 Lock App for BRD4161A
               timeout-minutes: 10
-              run:
-                  scripts/examples/gn_efr32_example.sh examples/lock-app/efr32/
-                  out/lock_app_debug BRD4161A
+              run: |
+                  scripts/examples/gn_efr32_example.sh examples/lock-app/efr32/ out/lock_app_debug BRD4161A
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py efr32 BRD4161A lock-app \
+                    out/lock_app_debug/BRD4161A/chip-efr32-lock-example.out /tmp/bloat_reports/
             - name: Build example EFR32 Lighting App for BRD4161A
               timeout-minutes: 10
-              run:
-                  scripts/examples/gn_efr32_example.sh
-                  examples/lighting-app/efr32/ out/lighting_app_debug BRD4161A
+              run: |
+                  scripts/examples/gn_efr32_example.sh examples/lighting-app/efr32/ out/lighting_app_debug BRD4161A
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py efr32 BRD4161A lighting-app \
+                    out/lighting_app_debug/BRD4161A/chip-efr32-lighting-example.out /tmp/bloat_reports/
             - name: Build example EFR32 Lighting App for BRD4161A with RPCs
               timeout-minutes: 10
-              run:
-                  scripts/examples/gn_efr32_example.sh
-                  examples/lighting-app/efr32/ out/lighting_app_debug_rpc BRD4161A
-                  -args='import("//with_pw_rpc.gni")'
+              run: |
+                  scripts/examples/gn_efr32_example.sh examples/lighting-app/efr32/ out/lighting_app_debug_rpc BRD4161A \
+                    -args='import("//with_pw_rpc.gni")'
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py efr32 BRD4161A+rpc lighting-app \
+                    out/lighting_app_debug_rpc/BRD4161A/chip-efr32-lighting-example.out /tmp/bloat_reports/
             - name: Build example EFR32 Window Covering for BRD4161A
               timeout-minutes: 10
-              run:
-                  scripts/examples/gn_efr32_example.sh examples/window-app/efr32/
-                  out/window_app_debug BRD4161A
+              run: |
+                  scripts/examples/gn_efr32_example.sh examples/window-app/efr32/ out/window_app_debug BRD4161A
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py efr32 BRD4161A window-app \
+                    out/window_app_debug/BRD4161A/chip-efr32-window-example.out /tmp/bloat_reports/
             - name: Binary artifact suffix
               id: outsuffix
               uses: haya14busa/[email protected]
@@ -94,3 +101,9 @@ jobs:
                       out/lock_app_debug/BRD4161A/chip-efr32-lock-example.out.map
                       out/lighting_app_debug_rpc/BRD4161A/chip-efr32-lighting-example.out
                       out/lighting_app_debug_rpc/BRD4161A/chip-efr32-lighting-example.out.map
+            - name: Uploading Size Reports
+              uses: actions/upload-artifact@v2
+              with:
+                  name: Size,EFR32-Examples,${{ env.GH_EVENT_PR }},${{ env.GH_EVENT_HASH }},${{ env.GH_EVENT_PARENT }}
+                  path: |
+                      /tmp/bloat_reports/
diff --git a/.github/workflows/examples-esp32.yaml b/.github/workflows/examples-esp32.yaml
@@ -30,6 +30,9 @@ jobs:
 
         env:
             BUILD_TYPE: esp32
+            GH_EVENT_PR: ${{ github.event_name == 'pull_request' && github.event.number || 0 }}
+            GH_EVENT_HASH: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+            GH_EVENT_PARENT: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
 
         runs-on: ubuntu-latest
         if: github.actor != 'restyled-io[bot]'
@@ -64,6 +67,10 @@ jobs:
                   mkdir -p example_binaries/$BUILD_TYPE-build
                   cp examples/all-clusters-app/esp32/build/chip-all-clusters-app.elf \
                      example_binaries/$BUILD_TYPE-build/chip-all-clusters-app.elf
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                     esp32 m5stack all-clusters-app \
+                     example_binaries/$BUILD_TYPE-build/chip-all-clusters-app.elf \
+                     /tmp/bloat_reports/
             - name: Build example All Clusters App C3
               timeout-minutes: 10
               run: scripts/examples/esp_example.sh all-clusters-app sdkconfig_c3devkit.defaults
@@ -72,6 +79,10 @@ jobs:
                   mkdir -p example_binaries/$BUILD_TYPE-build
                   cp examples/all-clusters-app/esp32/build/chip-all-clusters-app.elf \
                      example_binaries/$BUILD_TYPE-build/chip-all-clusters-app.elf
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                     esp32 c3devkit all-clusters-app \
+                     example_binaries/$BUILD_TYPE-build/chip-all-clusters-app.elf \
+                     /tmp/bloat_reports/
             - name: Build example Pigweed App
               timeout-minutes: 5
               run: scripts/examples/esp_example.sh pigweed-app sdkconfig.defaults
@@ -80,6 +91,10 @@ jobs:
                   mkdir -p example_binaries/$BUILD_TYPE-build
                   cp examples/pigweed-app/esp32/build/chip-pigweed-app.elf \
                      example_binaries/$BUILD_TYPE-build/chip-pigweed-app.elf
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                     esp32 default pigweed-app \
+                     example_binaries/$BUILD_TYPE-build/chip-pigweed-app.elf \
+                     /tmp/bloat_reports/
             - name: Build example Lock App
               timeout-minutes: 5
               run: scripts/examples/esp_example.sh lock-app sdkconfig.defaults
@@ -88,6 +103,10 @@ jobs:
                   mkdir -p example_binaries/$BUILD_TYPE-build
                   cp examples/lock-app/esp32/build/chip-lock-app.elf \
                      example_binaries/$BUILD_TYPE-build/chip-lock-app.elf
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                     esp32 default lock-app \
+                     example_binaries/$BUILD_TYPE-build/chip-lock-app.elf \
+                     /tmp/bloat_reports/
             - name: Build example Bridge App
               timeout-minutes: 5
               run: scripts/examples/esp_example.sh bridge-app
@@ -96,6 +115,10 @@ jobs:
                   mkdir -p example_binaries/$BUILD_TYPE-build
                   cp examples/bridge-app/esp32/build/chip-bridge-app.elf \
                      example_binaries/$BUILD_TYPE-build/chip-bridge-app.elf
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                     esp32 default bridge-app \
+                     example_binaries/$BUILD_TYPE-build/chip-bridge-app.elf \
+                     /tmp/bloat_reports/
             - name: Build example Persistent Storage App
               timeout-minutes: 5
               run: scripts/examples/esp_example.sh persistent-storage sdkconfig.defaults
@@ -104,6 +127,10 @@ jobs:
                   mkdir -p example_binaries/$BUILD_TYPE-build
                   cp examples/persistent-storage/esp32/build/chip-persistent-storage.elf \
                      example_binaries/$BUILD_TYPE-build/chip-persistent-storage.elf
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                     esp32 default persistent-storage \
+                     example_binaries/$BUILD_TYPE-build/chip-persistent-storage.elf \
+                     /tmp/bloat_reports/
             - name: Build example Shell App
               timeout-minutes: 5
               run: scripts/examples/esp_example.sh shell sdkconfig.defaults
@@ -112,6 +139,10 @@ jobs:
                   mkdir -p example_binaries/$BUILD_TYPE-build
                   cp examples/shell/esp32/build/chip-shell.elf \
                      example_binaries/$BUILD_TYPE-build/chip-shell.elf
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                     esp32 default shell \
+                     example_binaries/$BUILD_TYPE-build/chip-shell.elf \
+                     /tmp/bloat_reports/
             - name: Build example Temperature Measurement App
               timeout-minutes: 5
               run: scripts/examples/esp_example.sh temperature-measurement-app sdkconfig.optimize.defaults
@@ -120,6 +151,10 @@ jobs:
                   mkdir -p example_binaries/$BUILD_TYPE-build
                   cp examples/temperature-measurement-app/esp32/build/chip-temperature-measurement-app.elf \
                      example_binaries/$BUILD_TYPE-build/chip-temperature-measurement-app.elf
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                     esp32 optimize temperature-measurement-app \
+                     example_binaries/$BUILD_TYPE-build/chip-temperature-measurement-app.elf \
+                     /tmp/bloat_reports/
             - name: Build example IPv6 Only App
               timeout-minutes: 5
               run: scripts/examples/esp_example.sh ipv6only-app sdkconfig.defaults
@@ -128,6 +163,10 @@ jobs:
                   mkdir -p example_binaries/$BUILD_TYPE-build
                   cp examples/ipv6only-app/esp32/build/chip-ipv6only-app.elf \
                      example_binaries/$BUILD_TYPE-build/chip-ipv6only-app.elf
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                     esp32 default ipv6only-app \
+                     example_binaries/$BUILD_TYPE-build/chip-ipv6only-app.elf \
+                     /tmp/bloat_reports/
             - name: Binary artifact suffix
               id: outsuffix
               uses: haya14busa/[email protected]
@@ -145,3 +184,8 @@ jobs:
                       ${{ env.BUILD_TYPE }}-example-build-${{
                       steps.outsuffix.outputs.value }}
                   path: /tmp/output_binaries/${{ env.BUILD_TYPE }}-build
+            - name: Uploading Size Reports
+              uses: actions/upload-artifact@v2
+              with:
+                  name: Size,ESP32-Examples,${{ env.GH_EVENT_PR }},${{ env.GH_EVENT_HASH }},${{ env.GH_EVENT_PARENT }}
+                  path: /tmp/bloat_reports/
diff --git a/.github/workflows/examples-infineon.yaml b/.github/workflows/examples-infineon.yaml
@@ -28,6 +28,11 @@ jobs:
         name: Infineon examples building
         timeout-minutes: 30
 
+        env:
+            GH_EVENT_PR: ${{ github.event_name == 'pull_request' && github.event.number || 0 }}
+            GH_EVENT_HASH: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+            GH_EVENT_PARENT: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
+
         runs-on: ubuntu-latest
         if: github.actor != 'restyled-io[bot]'
 
@@ -55,4 +60,13 @@ jobs:
               run: |
                 scripts/run_in_build_env.sh \
                   "scripts/build/build_examples.py --no-log-timestamps --platform infineon --app lock build"
+                .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                    p6 default lock-app \
+                    out/infineon-p6board-lock/chip-p6-lock-example.out
 
+            - name: Uploading Size Reports
+              uses: actions/upload-artifact@v2
+              with:
+                  name: Size,P6-Examples,${{ env.GH_EVENT_PR }},${{ env.GH_EVENT_HASH }},${{ env.GH_EVENT_PARENT }}
+                  path: |
+                    out/infineon-p6board-lock/p6-default-lock-app-sizes.json
diff --git a/.github/workflows/examples-k32w.yaml b/.github/workflows/examples-k32w.yaml
@@ -29,6 +29,9 @@ jobs:
 
         env:
             BUILD_TYPE: gn_k32w
+            GH_EVENT_PR: ${{ github.event_name == 'pull_request' && github.event.number || 0 }}
+            GH_EVENT_HASH: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}
+            GH_EVENT_PARENT: ${{ github.event_name == 'pull_request' && github.event.pull_request.base.sha || github.event.before }}
 
         runs-on: ubuntu-latest
         if: github.actor != 'restyled-io[bot]'
@@ -56,16 +59,28 @@ jobs:
                    .environment/pigweed-venv/*.log
             - name: Build example K32W Lock App
               timeout-minutes: 5
-              run: scripts/examples/k32w_example.sh
-                   examples/lock-app/k32w out/lock_app_debug
+              run: |
+                  scripts/examples/k32w_example.sh examples/lock-app/k32w out/lock_app_debug
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                    k32w k32w061+debug lock-app \
+                    out/lock_app_debug/chip-k32w061-lock-example \
+                    /tmp/bloat_reports/
             - name: Build example K32W Shell App
               timeout-minutes: 5
-              run: scripts/examples/k32w_example.sh
-                   examples/shell/k32w out/shell_app_debug
+              run: |
+                  scripts/examples/k32w_example.sh examples/shell/k32w out/shell_app_debug
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                    k32w k32w061+debug shell \
+                    out/shell_app_debug/chip-k32w061-shell-example \
+                    /tmp/bloat_reports/
             - name: Build example K32W Lighting App with Secure Element
               timeout-minutes: 5
-              run: scripts/examples/k32w_se_example.sh
-                   examples/lighting-app/k32w out/lighting_app_se_release
+              run: |
+                  scripts/examples/k32w_se_example.sh examples/lighting-app/k32w out/lighting_app_se_release
+                  .environment/pigweed-venv/bin/python3 scripts/tools/memory/gh_sizes.py \
+                    k32w k32w061+se05x+release lighting-app \
+                    out/lighting_app_se_release/chip-k32w061-light-example \
+                    /tmp/bloat_reports/
             - name: Binary artifact suffix
               id: outsuffix
               uses: haya14busa/[email protected]
@@ -82,3 +97,9 @@ jobs:
                   path: |
                       out/lock_app_debug/chip-k32w061-lock-example.out
                       out/lock_app_debug/chip-k32w061-lock-example.out.map
+            - name: Uploading Size Reports
+              uses: actions/upload-artifact@v2
+              with:
+                  name: Size,K32W-Examples,${{ env.GH_EVENT_PR }},${{ env.GH_EVENT_HASH }},${{ env.GH_EVENT_PARENT }}
+                  path: |
+                      /tmp/bloat_reports/