Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start separating generated files in unique folder instead of inside the src/example tree #9124

Merged
merged 45 commits into from
Aug 20, 2021

Conversation

andy31415
Copy link
Contributor

@andy31415 andy31415 commented Aug 19, 2021

Problem

ZAP generated files are in-tree and this has undesired sideffects:

  • code review is harder as every change updates different distinct path. If zap updates are mixed with code updates, reviewers have to jump through files to filter out genreated vs non-generated files
  • relative paths are used extensively and include odities (e.g. data_model generated has same file names as example generated making include order important) go easier unnoticed

Change overview

  • created a 'zzz_generated' folder (for alpha-sorting to go at the end) for all example generated files. Explicitly named folders as 'zap-generated' instead of 'gen' to keep track of them easier in the future (a gen/include is used for build time include generation, would like zap to be explicitly different)
  • added 'output folder' support in the regen all script (make all examples go to the zzz folder)
  • Migrated examples and other builds to the new structure
  • Used <....> includes instead of "..."

Testing

Verified that things build (except pump/pump-controller which does not build in ToT yet), CI will also validate things.
Checked that zap-regen-all does not change any output.

@todo
Copy link

todo bot commented Aug 19, 2021

order of includes is important here as the data model headers

# TODO: order of includes is important here as the data model headers
# often overlap with the app include headers.
#
# Generated file name uniqueness would be very useful here (or have alternate
# subdirectory paths)
if (!_controller_model) {
public_configs += [ "${chip_root}/src/controller/data_model:data_model_config" ]
}
}
}


This comment was generated by todo based on a TODO comment in dd801a6 in #9124. cc @andy31415.

@github-actions
Copy link

Size increase report for "esp32-example-build" from 155d61f

File Section File VM
chip-shell.elf .flash.text 56 56
Full report output
BLOAT REPORT

Files found only in the build output:
    report.csv

Comparing ./master_artifact/chip-lock-app.elf and ./pull_artifact/chip-lock-app.elf:

sections,vmsize,filesize
.debug_str,0,11
.debug_line,0,9

Comparing ./master_artifact/chip-pigweed-app.elf and ./pull_artifact/chip-pigweed-app.elf:

sections,vmsize,filesize

Comparing ./master_artifact/chip-shell.elf and ./pull_artifact/chip-shell.elf:

sections,vmsize,filesize
.flash.text,56,56
[Unmapped],0,-56

Comparing ./master_artifact/chip-bridge-app.elf and ./pull_artifact/chip-bridge-app.elf:

sections,vmsize,filesize
.debug_str,0,5
.debug_line,0,3

Comparing ./master_artifact/chip-ipv6only-app.elf and ./pull_artifact/chip-ipv6only-app.elf:

sections,vmsize,filesize

Comparing ./master_artifact/chip-persistent-storage.elf and ./pull_artifact/chip-persistent-storage.elf:

sections,vmsize,filesize

Comparing ./master_artifact/chip-all-clusters-app.elf and ./pull_artifact/chip-all-clusters-app.elf:

sections,vmsize,filesize
.riscv.attributes,0,-2
.debug_line,0,-15
.debug_str,0,-15

Comparing ./master_artifact/chip-temperature-measurement-app.elf and ./pull_artifact/chip-temperature-measurement-app.elf:

sections,vmsize,filesize
.debug_line,0,225
.debug_str,0,223


@andy31415 andy31415 merged commit 0fcf173 into project-chip:master Aug 20, 2021
sharadb-amazon pushed a commit to sharadb-amazon/connectedhomeip that referenced this pull request Aug 23, 2021
…he src/example tree (project-chip#9124)

* Start creating a zzz_generated folder

* Allow output folder argument in zap generation python script. Verified that regenall was a noop

* moved all generated files from examples into zzz_generated (by running regen-all)

* Update gen folder as "zap-generated"

* Rename gen to zap-generated in includes

* More generated path renaming

* update the cluster template generation to use the new zap-generated name

* Rename more zap generated files

* more renames and zapt updates. Fix build rule dependency

* Add config dependency on data model: only need include paths

* Fix include order. Lighting app on linux compiles

* Add TODO regarding config path ordering

* Prepare example generated files: nrf builds compile

* Make lock app compile on nrf with the new directory structure

* update telink makefile to match the new  zap generated outputs

* More compile logic updates

* esp32 compiles now

* Make the bridge app  compile on esp32

* Exclude zap generated files from doxygen

* Fix typo in CHIPCert, move the darwin header/mm files

* Update a few more gen folder usages

* Update java to zap-generated, move more files around

* Restyle fixes

* Update path inside pbxproj

* Fix up lighting app cmake for mbed

* Update lock generated files for esp32 lock app

* Update paths for generated file for lock app cmakefiles

* One more cmake fix for esp32

* Update CMakeLists.txt for the temperature measurement app

* Add back af-gen-event.h even though they are NOT currently generated

* Undo the data_model include logic

* Move ota-provider-app to zzz_generated

* Fix android build

* Fix ota provider app build path for zap generated files

* Restyle fixes

* Fix up ota build

* Updated fix for ota provider app build rules

* Create ota-provider-app config globally, to fix up includes for common paths

* Restyle fixes

* Update relative paths into ota-provider-common to  use top level include

* Do not use relative path in includes in ota provider common
nikita-s-wrk pushed a commit to nikita-s-wrk/connectedhomeip that referenced this pull request Sep 23, 2021
…he src/example tree (project-chip#9124)

* Start creating a zzz_generated folder

* Allow output folder argument in zap generation python script. Verified that regenall was a noop

* moved all generated files from examples into zzz_generated (by running regen-all)

* Update gen folder as "zap-generated"

* Rename gen to zap-generated in includes

* More generated path renaming

* update the cluster template generation to use the new zap-generated name

* Rename more zap generated files

* more renames and zapt updates. Fix build rule dependency

* Add config dependency on data model: only need include paths

* Fix include order. Lighting app on linux compiles

* Add TODO regarding config path ordering

* Prepare example generated files: nrf builds compile

* Make lock app compile on nrf with the new directory structure

* update telink makefile to match the new  zap generated outputs

* More compile logic updates

* esp32 compiles now

* Make the bridge app  compile on esp32

* Exclude zap generated files from doxygen

* Fix typo in CHIPCert, move the darwin header/mm files

* Update a few more gen folder usages

* Update java to zap-generated, move more files around

* Restyle fixes

* Update path inside pbxproj

* Fix up lighting app cmake for mbed

* Update lock generated files for esp32 lock app

* Update paths for generated file for lock app cmakefiles

* One more cmake fix for esp32

* Update CMakeLists.txt for the temperature measurement app

* Add back af-gen-event.h even though they are NOT currently generated

* Undo the data_model include logic

* Move ota-provider-app to zzz_generated

* Fix android build

* Fix ota provider app build path for zap generated files

* Restyle fixes

* Fix up ota build

* Updated fix for ota provider app build rules

* Create ota-provider-app config globally, to fix up includes for common paths

* Restyle fixes

* Update relative paths into ota-provider-common to  use top level include

* Do not use relative path in includes in ota provider common
@andy31415 andy31415 deleted the separate_generated branch October 28, 2021 14:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
app documentation Improvements or additions to documentation examples review - approved scripts
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants