compile-time allocation of flash

[tballmsft]

In general, an extension might have a need for extra flash. If a MakeCode app includes several extensions, each of them with some need for a declared constant amount of flash, it would be great for the MakeCode compiler to statically allocate and assign available flash, and then determine how much space is left for the code. This flexibility will allow us to:

• make sure we don't use the same region of flash for difference purposes (one extension overwriting flash used by another)
• don't overallocate

The important point here is that an extension (or feature) should just say how much flash it needs and be parameterized by an address that will be set from outside at compile time. We want to stay away from a feature hardcoding a particular address range, especially if that feature may not be used all the time.

In general, we might have C++ code that wants some flash as well (and is optionally compiled), so would be good to think about this as well. As you know, each time we change set of extensions for an app, we recompile the whole world (with caching), so we can definitely accommodate needs of C++ that is included for a compilation.

[tballmsft]

@finneyj

[tballmsft]

The basic idea here is quite simple, we will make a pass over the packages to collect the amount of flash (default 0) that each needs. This list will be used to allocate addresses (from top of available flash down) and make a call to a known function in the package to inform it of the starting (bottom) address for its flash range.

• requiresFlashBytes?: number // what does this need to be a multiple of
• startingFlashAddr?: (addr: number) => void

[jaustin]

@tballmsft what are the cases for an extension needing dedicated flash, compared to, say, needing a reliable config store, or filesystem? Back in V1 we had the MicroBitStorage which was quite generally useful, but underused because it wasn't persistent.

I worry we'll consume flash quite quickly with an approach like this, as unless extensions co-operate, the minimum allocation granule will be one flash page and the size of that can be platform dependent, so could see a fallback to reserving large chunk.

On micro:bit, some considerations for this:

• target flash will be erased on a drag-and-drop operation
• Different flash page sizes across platforms
• If BLE is enabled, I think we can't expect extensions to be able to safely write to flash without SoftDevice interrupting. Do we need to pass all flash writes through CODAL (so CODAL can handle the SoftDevice side of things?). If so, would a mechanism for requesting flash from CODAL at runtime be better?
• the micro:bit V2 has 128kB of dedicated storage on the interface chip, which is pretty handy for things like logs and potentially assets, but we don't expose this (we currently can't write assets in there with the hex file). This is optionally exposed as the MY_DATA.HTM FILE

@finneyj any other considerations here?

[tballmsft]

Thanks. @jaustin. Yes, a reliable configuration store (which is implemented in settings, used in Arcade, and also by pxt-jacdac) would be very useful. Of course, size might matter. For certain apps, we may need to allocate more flash. For many applications, persistence is only required for that app, so drag and drop of a new app isn't a huge concern. For Jacdac, we persist the mapping between role names and device identifiers. For more ambitious programs, like MicroCode (now ported to makecode.microbit.org), we persist user programs in flash.

My understanding, correct me if I am wrong, is that we have several new features that require some flash:

• MicroBitFileSystem
• MicroBitStorage
• Scratch storage (used by the above)

In the future, will there be need for

• Machine learning
• Jacdac
• TBD

My major point is that a particular micro:bit app might used just a subset of the above features, so it seems problematic to statically allocate regions for features that might not be used by an app. Different features might have different needs/APIs for flash, so not accommodated by a generic key-value store (though maybe it is enough?)

https://github.com/lancaster-university/codal-microbit-v2/blob/master/docs/MemoryMap.md

[tballmsft]

per @jaustin - we probably need to discuss this in larger context of partial flashing - anything that relies on the layout of code/data in flash.