Switch to using DecodableList for list-typed attribute reads #10681

bzbarsky-apple · 2021-10-20T02:25:38Z

Problem

Attribute reads are using a non-spec data format.

Change overview

Use DecodableList for list-typed attribute reads.
Update various codegen bits (chip-tool, Android, Darwin, python) to handle getting a DecodableList
Enable the complex-typed-attributes define on the server to send lists encoded per spec.

Testing

Verified that existing Darwin and yaml tests (which read attributes on TestCluster and Descriptor at the very least) pass in the new setup and fail if I just enable the per-spec encoding on the server without the corresponding client changes.

src/controller/java/templates/CHIPClusters-JNI.zapt

github-actions · 2021-10-20T19:20:32Z

PR #10681: Size comparison from 447cdd7 to a9dce16

8 builds (for k32w, p6, qpg, telink)

platform	target	config	section	`447cdd7`	`a9dce16`	change
k32w	lock-app	k32w061+debug	.bss	69052	69060	8
			.data	1864	1864	0
			.text	515536	515552	16
	shell	k32w061+debug	.bss	55080	55080	0
			.data	672	672	0
			.text	357332	357332	0
	lighting-app	k32w061+se05x+release	.bss	78576	78584	8
			.data	1900	1900	0
			.text	614304	614320	16
p6	lock-app	default	.bss	68216	68216	0
			.data	2416	2416	0
			.heap	962712	962712	0
			.text	1126552	1126568	16
qpg	lighting-app	qpg6100+debug	.bss	53552	53552	0
			.data	996	996	0
			.text	486624	486640	16
	lock-app	qpg6100+debug	.bss	52488	52496	8
			.data	952	952	0
			.text	462692	462708	16
	persistent-storage-app	qpg6100+debug	.bss	17778	17778	0
			.data	280	280	0
			.text	102704	102704	0
telink	lighting-app	tlsr9518adk80d	bss	71000	71000	0
			noinit	33216	33216	0
			text	458318	458350	32

8 builds (for linux)

platform	target	config	section	`447cdd7`	`a9dce16`	change	% change
linux	all-clusters-app	debug	.bss	52176	52176	0	0.0
			.data	978	978	0	0.0
			.data.rel.ro	58688	58688	0	0.0
			.dynamic	592	592	0	0.0
			.got	4080	4080	0	0.0
			.init	27	27	0	0.0
			.init_array	512	512	0	0.0
			.rodata	136373	136245	-128	-0.1
			.text	1343314	`1342018`	-1296	-0.1
	chip-tool	debug	.bss	17552	17552	0	0.0
			.data	1584	1584	0	0.0
			.data.rel.ro	89344	89344	0	0.0
			.dynamic	592	592	0	0.0
			.got	4344	4344	0	0.0
			.init	27	27	0	0.0
			.init_array	416	416	0	0.0
			.rodata	184476	184932	456	0.2
			.text	`3425061`	`3436965`	11904	0.3
	ota-provider-app	debug	.bss	37472	37472	0	0.0
			.data	752	752	0	0.0
			.data.rel.ro	23176	23176	0	0.0
			.dynamic	592	592	0	0.0
			.got	4008	4008	0	0.0
			.init	27	27	0	0.0
			.init_array	440	440	0	0.0
			.rodata	110024	110024	0	0.0
			.text	`1009714`	1010002	288	0.0
	ota-requestor-app	debug	.bss	205728	205728	0	0.0
			.data	752	752	0	0.0
			.data.rel.ro	24488	24488	0	0.0
			.dynamic	592	592	0	0.0
			.got	4136	4136	0	0.0
			.init	27	27	0	0.0
			.init_array	512	512	0	0.0
			.rodata	128744	128744	0	0.0
			.text	`1129298`	`1129586`	288	0.0
	shell	debug	.bss	16072	16072	0	0.0
			.data	242	242	0	0.0
			.data.rel.ro	35120	35120	0	0.0
			.dynamic	592	592	0	0.0
			.got	3496	3496	0	0.0
			.init	27	27	0	0.0
			.init_array	336	336	0	0.0
			.rodata	71823	71823	0	0.0
			.text	569842	569842	0	0.0
	tv-app	debug	.bss	216400	216592	192	0.1
			.data	2032	2032	0	0.0
			.data.rel.ro	55872	56096	224	0.4
			.dynamic	592	592	0	0.0
			.got	4400	4400	0	0.0
			.init	27	27	0	0.0
			.init_array	608	608	0	0.0
			.rodata	152264	153352	1088	0.7
			.text	1461602	1442850	-18752	-1.3
	bridge-app	debug+rpc	.bss	52912	52912	0	0.0
			.data	976	976	0	0.0
			.data.rel.ro	25784	25784	0	0.0
			.dynamic	592	592	0	0.0
			.got	3944	3944	0	0.0
			.init	27	27	0	0.0
			.init_array	400	400	0	0.0
			.rodata	110620	110620	0	0.0
			.text	`1051093`	1051077	-16	-0.0
	lighting-app	debug+rpc	.bss	42232	42232	0	0.0
			.data	1106	1106	0	0.0
			.data.rel.ro	52496	52496	0	0.0
			.dynamic	608	608	0	0.0
			.got	4104	4104	0	0.0
			.init	27	27	0	0.0
			.init_array	528	528	0	0.0
			.rodata	127825	127825	0	0.0
			.text	1251314	1251314	0	0.0

github-actions · 2021-10-20T19:54:47Z

PR #10681: Size comparison from 447cdd7 to 1817486

20 builds (for efr32, k32w, linux, p6, qpg, telink)

platform	target	config	section	`447cdd7`	`1817486`	change	% change
efr32	lighting-app	BRD4161A	.bss	118036	118036	0	0.0
			.data	1800	1800	0	0.0
			.text	782896	782928	32	0.0
	lock-app	BRD4161A	.bss	115892	115892	0	0.0
			.data	1760	1760	0	0.0
			.text	762040	762056	16	0.0
	window-app	BRD4161A	.bss	116212	116212	0	0.0
			.data	1764	1764	0	0.0
			.text	762948	762964	16	0.0
	lighting-app	BRD4161A+rpc	.bss	131364	131364	0	0.0
			.data	1852	1852	0	0.0
			.text	762636	762652	16	0.0
k32w	lock-app	k32w061+debug	.bss	69052	69060	8	0.0
			.data	1864	1864	0	0.0
			.text	515536	515552	16	0.0
	shell	k32w061+debug	.bss	55080	55080	0	0.0
			.data	672	672	0	0.0
			.text	357332	357332	0	0.0
	lighting-app	k32w061+se05x+release	.bss	78576	78584	8	0.0
			.data	1900	1900	0	0.0
			.text	614304	614320	16	0.0
linux	all-clusters-app	debug	.bss	52176	52176	0	0.0
			.data	978	978	0	0.0
			.data.rel.ro	58688	58688	0	0.0
			.dynamic	592	592	0	0.0
			.got	4080	4080	0	0.0
			.init	27	27	0	0.0
			.init_array	512	512	0	0.0
			.rodata	136373	136245	-128	-0.1
			.text	1343314	`1342018`	-1296	-0.1
	chip-tool	debug	.bss	17552	17552	0	0.0
			.data	1584	1584	0	0.0
			.data.rel.ro	89344	89344	0	0.0
			.dynamic	592	592	0	0.0
			.got	4344	4344	0	0.0
			.init	27	27	0	0.0
			.init_array	416	416	0	0.0
			.rodata	184476	184932	456	0.2
			.text	`3425061`	`3436965`	11904	0.3
	ota-provider-app	debug	.bss	37472	37472	0	0.0
			.data	752	752	0	0.0
			.data.rel.ro	23176	23176	0	0.0
			.dynamic	592	592	0	0.0
			.got	4008	4008	0	0.0
			.init	27	27	0	0.0
			.init_array	440	440	0	0.0
			.rodata	110024	110024	0	0.0
			.text	`1009714`	1010002	288	0.0
	ota-requestor-app	debug	.bss	205728	205728	0	0.0
			.data	752	752	0	0.0
			.data.rel.ro	24488	24488	0	0.0
			.dynamic	592	592	0	0.0
			.got	4136	4136	0	0.0
			.init	27	27	0	0.0
			.init_array	512	512	0	0.0
			.rodata	128744	128744	0	0.0
			.text	`1129298`	`1129586`	288	0.0
	shell	debug	.bss	16072	16072	0	0.0
			.data	242	242	0	0.0
			.data.rel.ro	35120	35120	0	0.0
			.dynamic	592	592	0	0.0
			.got	3496	3496	0	0.0
			.init	27	27	0	0.0
			.init_array	336	336	0	0.0
			.rodata	71823	71823	0	0.0
			.text	569842	569842	0	0.0
	tv-app	debug	.bss	216400	216592	192	0.1
			.data	2032	2032	0	0.0
			.data.rel.ro	55872	56096	224	0.4
			.dynamic	592	592	0	0.0
			.got	4400	4400	0	0.0
			.init	27	27	0	0.0
			.init_array	608	608	0	0.0
			.rodata	152264	153352	1088	0.7
			.text	1461602	1442850	-18752	-1.3
	bridge-app	debug+rpc	.bss	52912	52912	0	0.0
			.data	976	976	0	0.0
			.data.rel.ro	25784	25784	0	0.0
			.dynamic	592	592	0	0.0
			.got	3944	3944	0	0.0
			.init	27	27	0	0.0
			.init_array	400	400	0	0.0
			.rodata	110620	110620	0	0.0
			.text	`1051093`	1051077	-16	-0.0
	lighting-app	debug+rpc	.bss	42232	42232	0	0.0
			.data	1106	1106	0	0.0
			.data.rel.ro	52496	52496	0	0.0
			.dynamic	608	608	0	0.0
			.got	4104	4104	0	0.0
			.init	27	27	0	0.0
			.init_array	528	528	0	0.0
			.rodata	127825	127825	0	0.0
			.text	1251314	1251314	0	0.0
p6	lock-app	default	.bss	68216	68216	0	0.0
			.data	2416	2416	0	0.0
			.heap	962712	962712	0	0.0
			.text	1126552	1126568	16	0.0
qpg	lighting-app	qpg6100+debug	.bss	53552	53552	0	0.0
			.data	996	996	0	0.0
			.text	486624	486640	16	0.0
	lock-app	qpg6100+debug	.bss	52488	52496	8	0.0
			.data	952	952	0	0.0
			.text	462692	462708	16	0.0
	persistent-storage-app	qpg6100+debug	.bss	17778	17778	0	0.0
			.data	280	280	0	0.0
			.text	102704	102704	0	0.0
telink	lighting-app	tlsr9518adk80d	bss	71000	71000	0	0.0
			noinit	33216	33216	0	0.0
			text	458318	458350	32	0.0

2 builds (for mbed)

platform	target	config	section	`447cdd7`	`1817486`	change
mbed	lighting-app	CY8CPROTO_062_4343W+release	.bss	172164	172164	0
			.data	5464	5464	0
			.heap	858816	858816	0
			.text	1220144	`1220208`	64
	lock-app	CY8CPROTO_062_4343W+release	.bss	171084	171084	0
			.data	5432	5432	0
			.heap	859928	859928	0
			.text	1198040	1198040	0

github-actions · 2021-10-20T20:40:35Z

Size increase report for "gn_qpg-example-build" from b06b55c

File	Section	File	VM
chip-qpg6100-lighting-example.out	.text	16	16

Full report output

BLOAT REPORT

Files found only in the build output:
    report.csv

Comparing ./master_artifact/chip-qpg6100-lighting-example.out and ./pull_artifact/chip-qpg6100-lighting-example.out:

sections,vmsize,filesize
.debug_info,0,35460
.debug_line,0,194
.strtab,0,70
.symtab,0,32
.debug_abbrev,0,26
.text,16,16
.shstrtab,0,2
[Unmapped],0,-16
.debug_ranges,0,-32
.debug_loc,0,-68

Comparing ./master_artifact/chip-qpg6100-lighting-example.out.map and ./pull_artifact/chip-qpg6100-lighting-example.out.map:

BLOAT EXECUTION FAILED WITH CODE 1:
bloaty: unknown file type for file './pull_artifact/chip-qpg6100-lighting-example.out.map'

github-actions · 2021-10-20T20:54:55Z

PR #10681: Size comparison from b06b55c to af655f9

22 builds (for efr32, k32w, linux, mbed, p6, qpg, telink)

platform	target	config	section	`b06b55c`	`af655f9`	change	% change
efr32	lighting-app	BRD4161A	.bss	118036	118036	0	0.0
			.data	1800	1800	0	0.0
			.text	782896	782928	32	0.0
	lock-app	BRD4161A	.bss	115892	115892	0	0.0
			.data	1760	1760	0	0.0
			.text	762040	762056	16	0.0
	window-app	BRD4161A	.bss	116212	116212	0	0.0
			.data	1764	1764	0	0.0
			.text	762948	762964	16	0.0
	lighting-app	BRD4161A+rpc	.bss	131364	131364	0	0.0
			.data	1852	1852	0	0.0
			.text	762636	762652	16	0.0
k32w	lock-app	k32w061+debug	.bss	69052	69060	8	0.0
			.data	1864	1864	0	0.0
			.text	515536	515552	16	0.0
	shell	k32w061+debug	.bss	55080	55080	0	0.0
			.data	672	672	0	0.0
			.text	357332	357332	0	0.0
	lighting-app	k32w061+se05x+release	.bss	78576	78584	8	0.0
			.data	1900	1900	0	0.0
			.text	614304	614320	16	0.0
linux	all-clusters-app	debug	.bss	52176	52176	0	0.0
			.data	978	978	0	0.0
			.data.rel.ro	58688	58688	0	0.0
			.dynamic	592	592	0	0.0
			.got	4080	4080	0	0.0
			.init	27	27	0	0.0
			.init_array	512	512	0	0.0
			.rodata	136373	136245	-128	-0.1
			.text	1343314	`1342018`	-1296	-0.1
	chip-tool	debug	.bss	17552	17552	0	0.0
			.data	1584	1584	0	0.0
			.data.rel.ro	89344	89344	0	0.0
			.dynamic	592	592	0	0.0
			.got	4344	4344	0	0.0
			.init	27	27	0	0.0
			.init_array	416	416	0	0.0
			.rodata	184604	185060	456	0.2
			.text	3428261	3439989	11728	0.3
	ota-provider-app	debug	.bss	37472	37472	0	0.0
			.data	752	752	0	0.0
			.data.rel.ro	23176	23176	0	0.0
			.dynamic	592	592	0	0.0
			.got	4008	4008	0	0.0
			.init	27	27	0	0.0
			.init_array	440	440	0	0.0
			.rodata	110024	110024	0	0.0
			.text	`1009714`	1010002	288	0.0
	ota-requestor-app	debug	.bss	205728	205728	0	0.0
			.data	752	752	0	0.0
			.data.rel.ro	24488	24488	0	0.0
			.dynamic	592	592	0	0.0
			.got	4136	4136	0	0.0
			.init	27	27	0	0.0
			.init_array	512	512	0	0.0
			.rodata	128776	128776	0	0.0
			.text	1129506	1129794	288	0.0
	shell	debug	.bss	16072	16072	0	0.0
			.data	242	242	0	0.0
			.data.rel.ro	35120	35120	0	0.0
			.dynamic	592	592	0	0.0
			.got	3496	3496	0	0.0
			.init	27	27	0	0.0
			.init_array	336	336	0	0.0
			.rodata	71823	71823	0	0.0
			.text	569842	569842	0	0.0
	tv-app	debug	.bss	216400	216592	192	0.1
			.data	2032	2032	0	0.0
			.data.rel.ro	55872	56096	224	0.4
			.dynamic	592	592	0	0.0
			.got	4400	4400	0	0.0
			.init	27	27	0	0.0
			.init_array	608	608	0	0.0
			.rodata	152296	153384	1088	0.7
			.text	1463170	1444418	-18752	-1.3
	bridge-app	debug+rpc	.bss	52912	52912	0	0.0
			.data	976	976	0	0.0
			.data.rel.ro	25784	25784	0	0.0
			.dynamic	592	592	0	0.0
			.got	3944	3944	0	0.0
			.init	27	27	0	0.0
			.init_array	400	400	0	0.0
			.rodata	110620	110620	0	0.0
			.text	`1051093`	1051077	-16	-0.0
	lighting-app	debug+rpc	.bss	42232	42232	0	0.0
			.data	1106	1106	0	0.0
			.data.rel.ro	52496	52496	0	0.0
			.dynamic	608	608	0	0.0
			.got	4104	4104	0	0.0
			.init	27	27	0	0.0
			.init_array	528	528	0	0.0
			.rodata	127825	127825	0	0.0
			.text	1251314	1251314	0	0.0
mbed	lighting-app	CY8CPROTO_062_4343W+release	.bss	172164	172164	0	0.0
			.data	5464	5464	0	0.0
			.heap	858816	858816	0	0.0
			.text	1220144	`1220208`	64	0.0
	lock-app	CY8CPROTO_062_4343W+release	.bss	171084	171084	0	0.0
			.data	5432	5432	0	0.0
			.heap	859928	859928	0	0.0
			.text	1198040	1198040	0	0.0
p6	lock-app	default	.bss	68216	68216	0	0.0
			.data	2416	2416	0	0.0
			.heap	962712	962712	0	0.0
			.text	1126552	1126568	16	0.0
qpg	lighting-app	qpg6100+debug	.bss	53552	53552	0	0.0
			.data	996	996	0	0.0
			.text	486624	486640	16	0.0
	lock-app	qpg6100+debug	.bss	52488	52496	8	0.0
			.data	952	952	0	0.0
			.text	462692	462708	16	0.0
	persistent-storage-app	qpg6100+debug	.bss	17778	17778	0	0.0
			.data	280	280	0	0.0
			.text	102704	102704	0	0.0
telink	lighting-app	tlsr9518adk80d	bss	71000	71000	0	0.0
			noinit	33216	33216	0	0.0
			text	458318	458350	32	0.0

10 builds (for nrfconnect)

platform	target	config	section	`b06b55c`	`af655f9`	change
nrfconnect	lighting-app	nrf52840dk_nrf52840	bss	112480	112480	0
			rodata	97776	97776	0
			text	577784	577800	16
	lock-app	nrf52840dk_nrf52840	bss	111544	111544	0
			rodata	94112	94112	0
			text	559272	559288	16
	pigweed-app	nrf52840dk_nrf52840	bss	51824	51824	0
			rodata	45772	45772	0
			text	339456	339456	0
	pump-app	nrf52840dk_nrf52840	bss	111612	111616	4
			rodata	95092	95092	0
			text	562428	562444	16
	pump-controller-app	nrf52840dk_nrf52840	bss	111552	111552	0
			rodata	94172	94172	0
			text	559064	559080	16
	shell	nrf52840dk_nrf52840	bss	107316	107316	0
			rodata	71640	71640	0
			text	519032	519032	0
	lighting-app	nrf52840dk_nrf52840+rpc	bss	108720	108720	0
			rodata	88560	88560	0
			text	550984	551000	16
		nrf5340dk_nrf5340_cpuapp	bss	113852	113852	0
			rodata	93016	93016	0
			text	507248	507264	16
	lock-app	nrf5340dk_nrf5340_cpuapp	bss	112916	112920	4
			rodata	89372	89372	0
			text	488732	488748	16
	shell	nrf5340dk_nrf5340_cpuapp	bss	108300	108300	0
			rodata	66284	66284	0
			text	439632	439632	0

2 builds (for esp32)

platform	target	config	section	`b06b55c`	`af655f9`	change	% change
esp32	all-clusters-app	c3devkit	.dram0.bss	60296	60296	0	0.0
			.dram0.data	16192	16192	0	0.0
			.flash.rodata	198992	198856	-136	-0.1
			.flash.text	873186	872686	-500	-0.1
			.iram0.text	57330	57330	0	0.0
		m5stack	.dram0.bss	62792	62792	0	0.0
			.dram0.data	32084	32084	0	0.0
			.flash.rodata	207736	207608	-128	-0.1
			.flash.text	904423	903959	-464	-0.1
			.iram0.text	125115	125115	0	0.0

github-actions · 2021-10-20T21:28:36Z

Size increase report for "nrfconnect-example-build" from b06b55c

File	Section	File	VM
chip-lock.elf	text	16	16

Full report output

BLOAT REPORT

Files found only in the build output:
    report.csv

Comparing ./master_artifact/chip-shell.elf and ./pull_artifact/chip-shell.elf:

sections,vmsize,filesize

Comparing ./master_artifact/chip-lock.elf and ./pull_artifact/chip-lock.elf:

sections,vmsize,filesize
.strtab,0,70
.debug_line,0,49
.symtab,0,32
.debug_info,0,19
text,16,16
.debug_loc,0,4
.shstrtab,0,2
.debug_ranges,0,-32

This was missed in project-chip#10681

This was missed in #10681

This was missed in project-chip#10681

boring-cyborg bot added app controller darwin examples lib labels Oct 20, 2021

austinh0 approved these changes Oct 20, 2021

View reviewed changes

src/controller/java/templates/CHIPClusters-JNI.zapt Outdated Show resolved Hide resolved

msandstedt approved these changes Oct 20, 2021

View reviewed changes

bzbarsky-apple added 7 commits October 20, 2021 16:03

Convert list attributes to DecodableList for chip-tool yaml tests.

9094b26

Convert list attributes to DecodableList for Darwin

01f022b

Convert list attributes to DecodableList for python controller

9f77095

Convert list attributes to DecodableList for the Android controller

b158c26

Improve comment per review feedback

ee95646

Fix tv app to produce lists with the right encoding.

b0ceab8

Address review comment to fix Android attribute reading

af655f9

bzbarsky-apple force-pushed the fix-attributes branch from 1817486 to af655f9 Compare October 20, 2021 20:13

jmartinez-silabs approved these changes Oct 20, 2021

View reviewed changes

pullapprove bot added review - approved and removed review - pending labels Oct 20, 2021

woody-apple approved these changes Oct 20, 2021

View reviewed changes

bzbarsky-apple merged commit b5a9acb into project-chip:master Oct 20, 2021

bzbarsky-apple deleted the fix-attributes branch October 20, 2021 22:36

yufengwangca mentioned this pull request Oct 26, 2021

[OperationalCredentials]: Support read TrustedRootCertificates attribute #10943

Merged

bzbarsky-apple mentioned this pull request Nov 4, 2021

Use of Variable Length Arrays in generated code #8558

Closed

bzbarsky-apple added a commit to bzbarsky-apple/connectedhomeip that referenced this pull request Nov 4, 2021

Remove unpaired pragma diagnostic pop.

cee972e

This was missed in project-chip#10681

bzbarsky-apple mentioned this pull request Nov 4, 2021

Remove unpaired pragma diagnostic pop. #11407

Merged

woody-apple pushed a commit that referenced this pull request Nov 4, 2021

Remove unpaired pragma diagnostic pop. (#11407)

56e15c5

This was missed in #10681

PSONALl pushed a commit to PSONALl/connectedhomeip that referenced this pull request Dec 3, 2021

Remove unpaired pragma diagnostic pop. (project-chip#11407)

7d2ab78

This was missed in project-chip#10681

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switch to using DecodableList for list-typed attribute reads #10681

Switch to using DecodableList for list-typed attribute reads #10681

bzbarsky-apple commented Oct 20, 2021

github-actions bot commented Oct 20, 2021 •

edited

Loading

github-actions bot commented Oct 20, 2021 •

edited

Loading

github-actions bot commented Oct 20, 2021

github-actions bot commented Oct 20, 2021 •

edited

Loading

github-actions bot commented Oct 20, 2021

Switch to using DecodableList for list-typed attribute reads #10681

Switch to using DecodableList for list-typed attribute reads #10681

Conversation

bzbarsky-apple commented Oct 20, 2021

Problem

Change overview

Testing

github-actions bot commented Oct 20, 2021 • edited Loading

github-actions bot commented Oct 20, 2021 • edited Loading

github-actions bot commented Oct 20, 2021

github-actions bot commented Oct 20, 2021 • edited Loading

github-actions bot commented Oct 20, 2021

github-actions bot commented Oct 20, 2021 •

edited

Loading

github-actions bot commented Oct 20, 2021 •

edited

Loading

github-actions bot commented Oct 20, 2021 •

edited

Loading