Introducing CLI tv-casting-app for Linux

[sharadb-amazon]

Problem

The CHIP SDK did not have an app that implemented the flow where a commissionable node discovers a commissioner and requests it for commissioning. See InitiatingSetup.adoc#24-commissioner-discovery-from-an-on-network-device

What is being fixed?
Added a CLI based CHIP TV Casting app for Linux with the following steps implemented (from the flow referenced above):

• discovers TVs on the local network that act as commissioners
• lets the user select one commissioner TV
• Enters commissioning mode and opens commissioning window
• Advertises itself as a commissionable node
• sends the TV a User Directed Commissioning request to the selected commissioner TV
• Displays the onboarding payload to the user

TBD: Implementation to actually send a casting command to a TV will be done in a separate PR.

Change overview

• New tv-casting-app example (which will act as the commissionable node)
• New RequestCommissioning method in the CHIPCommissionableNodeController
• tv-casting-app leverages the CHIPCommissionableNodeController to execute all the steps mentioned in the above section

Testing

A. Test cases added for commissioner discovery

B. Tested the tv-casting-app on Linux (VSCode/docker) by using the tv-app, minimal-mdns-client and chipTool:

1. Started running the tv-app and then the tv-casting-app in parallel, which discovers the commissioner TV over DNS-SD as follows.
   $ out/host/chip-tv-casting-app

...
[1628206267.250223][40533:40533] CHIP:ZCL: Discovered Commisioner #1
[1628206267.250266][40533:40533] CHIP:DIS: Device Name: Test TV
[1628206267.250282][40533:40533] CHIP:DIS: Vendor ID: 9050
[1628206267.250324][40533:40533] CHIP:DIS: Product ID: 65279
[1628206267.250340][40533:40533] CHIP:DIS: Device Type: 35
[1628206267.250381][40533:40533] CHIP:DIS: Hostname: 025000000001
[1628206267.250397][40533:40533] CHIP:DIS: Port: 5540
[1628206267.250439][40533:40533] CHIP:DIS: Number of IP addresses: 2. IP Adddress(es):
[1628206267.250458][40533:40533] CHIP:DIS: fe80::50:ff:fe00:1,
[1628206267.250502][40533:40533] CHIP:DIS: 192.168.65.3,
[1628206267.250517][40533:40533] CHIP:DIS: Commissioning Mode: 0
[1628206267.250561][40533:40533] CHIP:ZCL: 1 commissioner(s) discovered

2. User selects a commissioner TV over the CLI.

[1628206267.250577][40533:40533] CHIP:ZCL: Choose a commissioner TV (by number# above) to request commissioning from:
1

3. Opens commissioning window, enters commissioning mode and advertises itself as a commissionable node over DNS-SD (verified advertisement with out/host/minimal-mdns-client -q _matterc._udp.local)

[1628194259.587022][11128:11128] CHIP:ZCL: Using ZAP configuration...
...snip...
[1628194259.595609][11128:11128] CHIP:IN: UDP::Init bind&listen port=5540
[1628194259.595678][11128:11128] CHIP:IN: UDP::Init bind&listen port=5540
...snip...
[1628194259.605502][11128:11128] CHIP:SC: Waiting for PBKDF param request
[1628194259.605533][11128:11128] CHIP:DIS: Start dns-sd server
[1628194259.606382][11128:11128] CHIP:DIS: CHIP minimal mDNS started advertising.
[1628194259.606514][11128:11128] CHIP:DIS: Replying to DNS-SD service listing request
...snip...
[1628194259.609278][11128:11128] CHIP:SVR: Server Listening...

4. Sends User directed commissioning request to the selected commissioner TV (verified UDC commands were received on the tv-app console)

[1628194259.609290][11128:11128] CHIP:SVR: SendUserDirectedCommissioningRequest2
[1628194259.609301][11128:11128] CHIP:SVR: instanceName=DC514873944A5CFF
[1628194259.609317][11128:11128] CHIP:IN: Sending UDC msg
[1628194259.610781][11128:11128] CHIP:IN: UDC msg send status ../../third_party/connectedhomeip/src/inet/IPEndPointBasis.cpp:915: No Error
[1628194259.610850][11128:11128] CHIP:SVR: Send UDC request success
...snip...

5. Displays the onboarding payload

[1628206286.424116][40533:40533] CHIP:ZCL: Onboarding Payload::
[1628206286.424207][40533:40533] CHIP:ZCL: PIN: 34567890
[1628206286.424225][40533:40533] CHIP:ZCL: Discriminator: 2976
[1628206286.424253][40533:40533] CHIP:ZCL: Vendor ID: 9050
[1628206286.424306][40533:40533] CHIP:ZCL: Product ID: 65279

6. Receives onnetwork commissioning command from chipTool (out/host/chip-tool pairing onnetwork 0 34567890 2976 192.168.65.4 5540) and gets commissioned successfully.

[1628194320.944094][11128:11128] CHIP:EM: Received message of type 0x20 with vendorId 0x0000 and protocolId 0x0000 on exchange 39017
...snip...
[1628194320.956272][11128:11128] CHIP:SC: Sent PBKDF param response
[1628194320.967469][11128:11128] CHIP:EM: Received message of type 0x22 with vendorId 0x0000 and protocolId 0x0000 on exchange 39017
...snip...
[1628194320.971048][11128:11128] CHIP:SC: Sent spake2p msg2
[1628194320.979742][11128:11128] CHIP:EM: Received message of type 0x24 with vendorId 0x0000 and protocolId 0x0000 on exchange 39017
...snip...
[1628194321.277809][11128:11128] CHIP:IN: CASE Session established. Setting up the secure channel.
[1628194321.280133][11128:11128] CHIP:IN: Marking old secure session for device 0x000000000001B669 as expired
[1628194321.280173][11128:11128] CHIP:IN: New secure session created for device 0x000000000001B669, key 254!!
[1628194321.280251][11128:11128] CHIP:IN: CASE secure channel is available now.
[1628194321.280269][11128:11128] CHIP:IN: CASE Server enabling CASE session setups
[1628194321.280298][11128:11128] CHIP:EM: Sending Standalone Ack for MsgId:00000000
[1628194321.280446][11128:11128] CHIP:EM: Flushed pending ack for MsgId:00000000
[1628194444.416157][11128:11128] CHIP:DL: Inet Layer shutdown
[1628194444.416263][11128:11128] CHIP:DL: BLE layer shutdown
[1628194444.416311][11128:11128] CHIP:DL: System Layer shutdown

[andy31415]

Could you separate out commits between generated files and hand-edited code?

This is a 12K line added PR and it would help a lot to have the 'generated files' separated out.

[sharadb-amazon]

Could you separate out commits between generated files and hand-edited code?

This is a 12K line added PR and it would help a lot to have the 'generated files' separated out.

Sure, I just separated the changes into different commits (zap/gen code and handwritten code): https://github.com/project-chip/connectedhomeip/pull/8824/commits - is that better?

[chrisdecenzo]

please see comments

[tcarmelveilleux]

While I can understand that the current state of some of the API surfaces make the desired change harder to do, the solution used here to add a lot of application-dependant #ifdef will not scale. This is just one more application desiring to implement a given feature, and I can imagine N other such applications with slightly different requirements. Instead of adding more #ifdef that enshrine more assumptions (some of which are temporary), there should be some effort to refactor or at least discuss the obstacles found in API usage that cause the need to include all this specific steering of code.

The pervasive new usage of #if CONFIG_DEVICE_LAYER ifdefs really indicate to me that some APIs are contradict some the assumptions of the author in terms of what their application ought to have to provide or implement. Rather than #ifdef, let's discuss how to improve the API for everyone.

Finally, this PR touches much code, including code not related to its own requirements, and does so without adding or modifying a single unit test or integration test. Manual testing of a feature of this size is insufficient to maintain it in the tree.

[andy31415]

Flagging as SDK Discussion required concering the approach: sudo usage (is it BT, something else)?, usage of ifdefs, split between functionality and example app development. Generally looking for a path forward to have incremental development instead of large patches mixing examples and functionality code.

[msandstedt]

Minor nit inline, but this looks good to me.

[github-actions]

Size increase report for "gn_qpg-example-build" from deb227f

File	Section	File	VM
chip-qpg6100-lighting-example.out	.text	32	32

Full report output

BLOAT REPORT

Files found only in the build output:
    report.csv

Comparing ./master_artifact/chip-qpg6100-lighting-example.out.map and ./pull_artifact/chip-qpg6100-lighting-example.out.map:

BLOAT EXECUTION FAILED WITH CODE 1:
bloaty: unknown file type for file './pull_artifact/chip-qpg6100-lighting-example.out.map'

Comparing ./master_artifact/chip-qpg6100-lighting-example.out and ./pull_artifact/chip-qpg6100-lighting-example.out:

sections,vmsize,filesize
.debug_info,0,142
.debug_str,0,58
.text,32,32
.debug_line,0,25
.debug_abbrev,0,-2
.debug_ranges,0,-16
[Unmapped],0,-32
.debug_loc,0,-247

[github-actions]

Size increase report for "esp32-example-build" from deb227f

File	Section	File	VM
chip-all-clusters-app.elf	.flash.text	40	40
chip-shell.elf	.flash.text	40	40

Full report output

BLOAT REPORT

Files found only in the build output:
    report.csv

Comparing ./master_artifact/chip-persistent-storage.elf and ./pull_artifact/chip-persistent-storage.elf:

sections,vmsize,filesize

Comparing ./master_artifact/chip-all-clusters-app.elf and ./pull_artifact/chip-all-clusters-app.elf:

sections,vmsize,filesize
.debug_info,0,567
.debug_line,0,76
.debug_str,0,57
.flash.text,40,40
.debug_loc,0,11
.debug_ranges,0,8
.riscv.attributes,0,3
.debug_abbrev,0,-26
[Unmapped],0,-40

Comparing ./master_artifact/chip-lock-app.elf and ./pull_artifact/chip-lock-app.elf:

sections,vmsize,filesize
.debug_info,0,283
.debug_str,0,55
.debug_line,0,1
.debug_loc,0,-11
.debug_abbrev,0,-24

Comparing ./master_artifact/chip-pigweed-app.elf and ./pull_artifact/chip-pigweed-app.elf:

sections,vmsize,filesize

Comparing ./master_artifact/chip-shell.elf and ./pull_artifact/chip-shell.elf:

sections,vmsize,filesize
.debug_info,0,269
.debug_line,0,98
.debug_str,0,54
.flash.text,40,40
.debug_loc,0,29
.debug_ranges,0,16
.debug_abbrev,0,-26
[Unmapped],0,-40

Comparing ./master_artifact/chip-ipv6only-app.elf and ./pull_artifact/chip-ipv6only-app.elf:

sections,vmsize,filesize

Comparing ./master_artifact/chip-bridge-app.elf and ./pull_artifact/chip-bridge-app.elf:

sections,vmsize,filesize
.debug_line,0,1
.debug_str,0,-1

Comparing ./master_artifact/chip-temperature-measurement-app.elf and ./pull_artifact/chip-temperature-measurement-app.elf:

sections,vmsize,filesize
.debug_str,0,3
.debug_line,0,1
.debug_loc,0,-24

[github-actions]

Size increase report for "nrfconnect-example-build" from deb227f

File	Section	File	VM
chip-lock.elf	text	28	28
chip-lock.elf	device_handles	4	4
chip-shell.elf	text	28	28
chip-shell.elf	device_handles	4	4

Full report output

BLOAT REPORT

Files found only in the build output:
    report.csv

Comparing ./master_artifact/chip-lock.elf and ./pull_artifact/chip-lock.elf:

sections,vmsize,filesize
.debug_info,0,142
.debug_str,0,57
text,28,28
.debug_line,0,12
device_handles,4,4
.debug_abbrev,0,-2
.debug_ranges,0,-16
.debug_loc,0,-65

Comparing ./master_artifact/chip-shell.elf and ./pull_artifact/chip-shell.elf:

sections,vmsize,filesize
.debug_info,0,219
.debug_str,0,57
text,28,28
.debug_line,0,21
.debug_abbrev,0,20
device_handles,4,4
.debug_ranges,0,-16
.debug_loc,0,-237

[sharadb-amazon]

Does that have something different with chip-device-ctrl > discover

In TE4_TestPlanVerificationSteps_V5, G34

Put the device in commisioning mode, Verify device advertising its ability for commissioning by running 'discover' command in the python controller along with the QR code or '-v vendorID' .. etc look for discover command usage and pass appropriate param to have commissioner discover the commissionable node . Look for the device information in the below command.
chip-device-ctrl > discover -qr MT:YNJV7VSC00KA0648G00
Waiting for device responses...
[1626844356.667520][1256856:1256864] CHIP:DL: Avahi browse: cache exhausted
[1626844356.770602][1256856:1256864] CHIP:DL: Avahi browse: cache new
[1626844356.771830][1256856:1256864] CHIP:DL: Avahi browse: cache new
[1626844357.669397][1256856:1256864] CHIP:DL: Avahi browse: all for now
[1626844357.879577][1256856:1256864] CHIP:DL: Avahi resolve found
[1626844357.881913][1256856:1256864] CHIP:DL: Avahi resolve found
[1626844358.060376][1256856:1256856] CHIP:DIS: Commissionable Node 0
[1626844358.060452][1256856:1256856] CHIP:DIS: Host name: DCA6328D7072
[1626844358.060526][1256856:1256856] CHIP:DIS: Long discriminator: 3840
[1626844358.060591][1256856:1256856] CHIP:DIS: Vendor ID: 9050
[1626844358.060656][1256856:1256856] CHIP:DIS: Product ID: 65279
[1626844358.060748][1256856:1256856] CHIP:DIS: Additional Pairing 0
[1626844358.060808][1256856:1256856] CHIP:DIS: Commissioning Mode 1
[1626844358.060884][1256856:1256856] CHIP:DIS: Device Type 257
[1626844358.060950][1256856:1256856] CHIP:DIS: Device Name Test Bulb
[1626844358.061028][1256856:1256856] CHIP:DIS: Rotating Id 00007C5F6E176CD40F68685D100A1CF8A98B
[1626844358.061089][1256856:1256856] CHIP:DIS: Pairing Instruction
[1626844358.061160][1256856:1256856] CHIP:DIS: Pairing Hint 0x21
[1626844358.061219][1256856:1256856] CHIP:DIS: Address 0: 192.168.4.223
chip-device-ctrl >

I see, this feature is not supported by chip-tool, but do we really need a standalone tv-casting-app? Or cloud added a command in chip tool?

I think specific commands like sending a UDC request can be modeled as a chip-tool command as well. But this is an example of an end-to-end tv-casting-app which implements this whole commissioning flow: InitiatingSetup.adoc#24-commissioner-discovery-from-an-on-network-device and is then supposed to send TV specific casting commands (TBD).

We will also port this over to separate Android/iOS apps later that can be used as reference by any content provider to implement commissioning of itself and casting in their respective apps.