-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add FOTA support and enable MCUboot logs #14
Conversation
### Summary Get's a FOTA MVP up and running so we can get the latest and greatest on our nRF91 Thingy:91s Change Highlights -- more details inline in the code: 1. I was going insane trying to debug MCUboot when no logs were being emitted. Tinkered around with Kconfig options to get this working. Trade off is I disabled support for mcumgr serial DFU but I do not believe anyone is using this atm anyway. 2. Fixing 1, pointed out that due to the size of the thingy91 image we are hitting an edge case with MCUboot's "swap move" algorithm blocking a FOTA from completing succesfully. I switched to OVERWRITE strategy to work around this. 3. First pass at a FOTA check. The algo works as follows: - Check for FOTA upon first connection since boot - Check for FOTA every 12 hours there-after - If FOTA check on 12 hour mark fails, check upon next successful connection (to accomodate situation where we were not connected) 4. There's some kind of corruption taking place when sending data to cloud _and_ doing a FOTA. We should figure out what's going wrong but for now I fixed it by adding retry logic in our own custom FOTA handler (`CONFIG_MEMFAULT_FOTA_DOWNLOAD_CALLBACK_CUSTOM=y`) ### Internal Documentation https://www.notion.so/memfault/WeFault-Thingy91-Fleet-Quickstart-394ab5bd97ab4a49af86f40423f51982 ### Test Plan 1. Build the new image ``` west build -b [email protected] memfault-asset-tracker -- -DCONFIG_MEMFAULT_NCS_PROJECT_KEY=\"${PROJECT_KEY}\" -DOVERLAY_CONFIG=overlay-memfault.conf -DCONFIG_BUILD_OUTPUT_META=n -Dmcuboot_CONFIG_BUILD_OUTPUT_META=n -DCONFIG_MEMFAULT_FOTA=y -DCONFIG_MEMFAULT_NCS_FW_VERSION_PREFIX=\"0.0.2+\" ``` 2. Flash the image on the board ``` west flash --erase ``` 3. Re-run 1) but with `-DCONFIG_MEMFAULT_NCS_FW_VERSION_PREFIX=\"0.0.3+\" 4. Upload `app_update.bin` from `build/zephyr` folder into wefault project: https://app.memfault.com/organizations/memfault/projects/thingy91-fleet/releases/0.0.3%2Bd8aaaa 5. Confirm FOTA completes successfully by watching console: ``` [00:00:26.520,416] <inf> mflt: FOTA Update Available. Starting Download! [00:00:26.521,820] <inf> download_client: Downloading: https://ota-cdn.memfault.com/3993/6539/11764076539?token=rFQ0Owi7Cbbf7VP9AOQimNndKD16VO_DdjLDl9ANrYg&expires=1701064800 [0] [00:00:26.521,881] <inf> mflt: FOTA In Progress [00:00:26.714,202] <inf> download_client: Setting up TLS credentials, sec tag count 3 [00:00:26.714,324] <inf> download_client: Connecting to https://ota-cdn.memfault.com/3993/6539/11764076539?token=rFQ0Owi7Cbbf7VP9AOQimNndKD16VO_DdjLDl9ANrYg&expires=1701064800 [00:00:28.714,447] <inf> download_client: Downloaded 1024/424428 bytes (0%) [...] [00:07:33.470,214] <inf> download_client: Downloaded 316416/424428 bytes (74%) [00:07:35.357,604] <inf> app_event_manager: LOCATION_MODULE_EVT_CLOUD_LOCATION_DATA_READY [00:07:35.358,337] <inf> app_event_manager: DATA_EVT_DATA_READY [00:07:35.364,715] <inf> app_event_manager: DATA_EVT_CLOUD_LOCATION_DATA_SEND [00:07:35.382,873] <inf> app_event_manager: DATA_EVT_DATA_SEND_BATCH [00:07:35.383,758] <inf> app_event_manager: CLOUD_EVT_DATA_SEND_QOS [00:07:35.384,094] <inf> app_event_manager: CLOUD_EVT_CLOUD_LOCATION_UNKNOWN [00:07:35.384,857] <inf> app_event_manager: LOCATION_MODULE_EVT_INACTIVE [00:07:35.386,108] <inf> app_event_manager: CLOUD_EVT_DATA_SEND_QOS [00:07:40.273,132] <inf> download_client: Downloaded 317440/424428 bytes (74%) [00:07:41.878,082] <err> download_client: Unexpected HTTP response: 400 bad request [00:07:41.878,112] <err> fota_download: Download client error [00:07:41.878,143] <inf> dfu_target_mcuboot: MCUBoot image upgrade aborted. [00:07:41.878,173] <inf> dfu_target_mcuboot: MCUBoot image upgrade aborted. [00:07:41.879,608] <err> mflt: FOTA failed -- trying again ... [00:07:50.251,220] <inf> mflt: FOTA Update Available. Starting Download! [...] [00:10:32.878,387] <inf> download_client: Downloaded 421888/424428 bytes (99%) [00:10:33.554,199] <inf> download_client: Downloaded 422912/424428 bytes (99%) uart:~$ *** Booting nRF Connect SDK dcaa0fbee306 *** I: Starting bootloader I: Image index: 0, Swap type: test I: Image 0 upgrade secondary slot -> primary slot I: Erasing the primary slot I: Image 0 copying the secondary slot to the primary slot: 0x679f0 bytes D: writing magic; fa_id=2 off=0x68ff0 (0x74ff0) D: erasing secondary header D: erasing secondary trailer I: Bootloader chainload address offset: 0xc000 I: Jumping to the first image slot ```
Some checks failed. Please fix and resubmit. checkpatch (informational only, not a failure)
Gitlint issues3: B6 Body message is missing Identity/Emails issues3c481ed: author email (Gillian Minnehan [email protected]) needs to match one of the signed-off-by entries. Tip: The bot edits this comment instead of posting a new one, so you can check the comment's history to see earlier messages. |
@noahp I am unlikely to get to this during the week, could you commandeer to help get it through? |
# | ||
# For now let's flip on MCUBOOT_OVERWRITE_ONLY mode which has less wonky | ||
# flash requirements (and imo is better for long term flash health) | ||
CONFIG_BOOT_UPGRADE_ONLY=y |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
my kingdom for a single kconfig flag 🥹
This worked like a charm for me!! 😍 So excited. Thank you @chrisc11 for finding the solution on this one. I applied a few clean up changes but otherwise looks good to me. @memfault/owners-mcu eyes would be great! Once I merge it, I will update our quickstart guide, set up a new release in the app, and prompt the rest of the team to flash with a new image once we merge. Can't wait! |
Appears I can't approve my own PR 😆 but this LGTM once CI passes. Thanks for the patch ups |
Summary
Get's a FOTA MVP up and running so we can get the latest and greatest on our nRF91 Thingy:91s
Change Highlights -- more details inline in the code:
CONFIG_MEMFAULT_FOTA_DOWNLOAD_CALLBACK_CUSTOM=y
)Internal Documentation
https://www.notion.so/memfault/WeFault-Thingy91-Fleet-Quickstart-394ab5bd97ab4a49af86f40423f51982
Test Plan
Re-run 1) but with `-DCONFIG_MEMFAULT_NCS_FW_VERSION_PREFIX="0.0.3+"
Upload
app_update.bin
frombuild/zephyr
folder into wefault project: https://app.memfault.com/organizations/memfault/projects/thingy91-fleet/releases/0.0.3%2Bd8aaaaConfirm FOTA completes successfully by watching console:
Resolves: MFLT-12431