Twister "FLASH overflow" does not account for imgtool trailer.

[fabiobaltieri]

Describe the bug
Twister has a check for skipping (rather than failing) builds that trigger a flash overflow error. That does not work for board that call imgtool through, where the build image is under the flash size, but the image + imgtool trailer is not. The build fails CI with an error like:

...
Memory region         Used Size  Region Size  %age Used
           FLASH:       97780 B        96 KB     99.47%
            SRAM:       27544 B       136 KB     19.78%
        USB_SRAM:          0 GB        16 KB      0.00%
        IDT_LIST:          0 GB         2 KB      0.00%
Usage: wrapper.py [OPTIONS] INFILE OUTFILE
Try 'wrapper.py --help' for help.

Error: Image size (0x1819e) + trailer (0x1b0) exceeds requested size 0x18000
sign the payload
ninja: build stopped: subcommand failed.

To Reproduce
Steps to reproduce the behavior:

1. git checkout v3.2.0-rc2
2. ./scripts/twister -v -p lpcxpresso55s69_ns -T samples/net/sockets/http_get

Expected behavior
The test should be skipped.

Impact
Annoyance, unrelated CI failures.

Logs and console output
n/a

Environment (please complete the following information):

• OS: Linux
• Toolchain: Zephyr SDK 0.15.0
• Commit SHA: v3.2.0-rc2

[nordicjm]

This links with #46194 in that depending on mcuboot configuration, a generated firmware image might be useless because of mcuboot swap status data, but there would need to be a way for the application (at build time) to know details of the mcuboot configuration

[fabiobaltieri]

Adding mcuboot maintainers, you have any ideas? I'm thinking either reducing the flash size in the board config when imgtool is enabled or detecting the specific error in twister. @nashif

[d3zd3z]

Unfortunately, the trailer size isn't easy to determine as it depends on signature sizes and other things. But, imgtool should fail if the image is too large, is that not happening?

[fabiobaltieri]

It is, the problem is that then twister detects that as a generic "build failure" and fails the test, while normally it would skip instead in case of flash overflow. So I guess the other option is for twister to detect this specific error condition and detect it as flash overflow.

[d3zd3z]

Do you have any ideas on how this could be done? I think that imgtool just raises an exception.

[fabiobaltieri]

Yeah actually it seems fairly easy, something like:

--- a/scripts/pylib/twister/twisterlib/runner.py
+++ b/scripts/pylib/twister/twisterlib/runner.py
@@ -220,10 +220,14 @@ class CMake:
 
             if log_msg:
                 overflow_found = re.findall("region `(FLASH|ROM|RAM|ICCM|DCCM|SRAM|dram0_1_seg)' overflowed by", log_msg)
+                imgtool_overflow_found = re.findall("Error: Image size \(.*\) \+ trailer \(.*\) exceeds requested size", log_msg)
                 if overflow_found and not self.options.overflow_as_errors:
                     logger.debug("Test skipped due to {} Overflow".format(overflow_found[0]))
                     self.instance.status = "skipped"
                     self.instance.reason = "{} overflow".format(overflow_found[0])
+                elif imgtool_overflow_found and not self.options.overflow_as_errors:
+                    self.instance.status = "skipped"
+                    self.instance.reason = "imgtool overflow"
                 else:
                     self.instance.status = "error"
                     self.instance.reason = "Build failure"

I'll send a PR.