Is there a way to reserve some flash for the periph_flashpage driver to use? My end goal is to use a flash based filesystem on top of mtd_mapper and mtd_flashpage. I see that the the macro FLASH_WRITABLE_INIT is provided, but that the reserved space is part of the image and will change if using riotboot with multiple slots.
Setting aside memory that persists reboots has been requested repeatedly, but AFAIK nobody has proposed any concrete solution (good or bad).
Issues that I think any approach would need to address are:
- How is the size of data area negotiated between applications, or versions of applications?
- Can the size change without rewriting the bootloader?
- How is the format of the data area negotiated between applications, or versions of applications?
- If RIOT modules (rather than apps) want to use it, can they?
- Does riotboot need to be aware of anything more than the reduced sizes of the A and B slots? (Active riotboots like the USB-DFU might offer read and write access for commissioning).
For a minimal version it might suffice to have some space set aside more or less statically (eg. per board), let the applications decide on their own whether or not they can use the data that may be present, and be done with it.
Note that for an application whose persistence format easily changes, it might be practical to (rather than having a shared area) peek into the last version’s known area on the other image to read, update and locally store any to-be-persisted information.
What would a solution look like? Is this some kind of partition table that is imagined? (I want to set aside two pages for persistent variables store; I did some code a few months ago that I’d like to advance… I guess I don’t have a pull request open) If we could agree to such a system of variables, then it alone could be reserved at a known place, and the variables could be say what the rest of the layout it.
I have no clear roadmap here, and especially think we’re lacking a clear set of requirements (for example, will “this can be useful even when upgrading from one application to another” in it?).
As I understand the current status, RIOTBOOT_LEN is fixed per board, and the slot lengths are a fixed function of the available flash, RIOTBOOT_LEN and NUM_SLOTS – change any of them, and non-PIE images (not sure whether we even have them yet). Changing the space outside the images breaks the old one.
So in a sense, the partition table is currently part of the board (or, more precisely, the combination of board + “riotboot was selected”), and statically known. The size (and type) of cross-flash partition might be an extra parameter added to that mix – in that case, there would not be a need for a run-time partition table, because the application knows everything relevant at build time.
Another aspect I’m unsure of is what is best stored there. @enoch247, you mention a flash file system through mtd_flashpage – are they even suitable for the relevant sizes? (I haven’t seen any actual use of off-the-shelf file systems on top of mtd_flashpage). My use case would be primarily storing a few keys and some counters and IDs, which all fit in one page, so I’d just need two erase units to ensure unbrickability, and AFAIK we don’t have any kind of file system that works with these sizes. (“file system” might not even be the right term – as I’d use it, it’s more of a struct). Possibly, FlashDB (which I should really review) could be such a thing.
I am actually looking for something very basic. I just want to tell the build system “don’t use some fixed number of flash pages at a fixed address.” I had planned to manage the contents myself.
Put another way, I want a file system without having to back it with an SD card or MTD. I have already done this using 1/4 of the flash at the “far” end of the flash where I don’t have code, but I want to inform the build system of this, so that it doesn’t spill code into this region and makes the appropriate adjustments to the riotboot_slot module. Is this something other people have done? I assumed there was already a provision in-place as I don’t understand what mtd_flashpage+mtd_mapper could be used for without this ability.
@jue89 : your comment here leads me to believe I’m not alone in doing this.
I think that @enoch247’s requirements are mostly clear to me, and they would work for me. I’d +2 them to have space for my desired two pages of environment variables at the “top”. I want to be able to provision an IDevID very early: yes, it would be nice to be able to have it persist from one application to another.
Having the build system know that the space is unavailable key, but also telling the link system what address things are at.
I was frustrated by the NAND (serial) non-optional flash support: it made me allocate too much ram to buffer things that I shouldn’t have to do. But that’s a different gripe.
IMO it would be best if all boards would create a mtd device for their internal flash and “partition” the mtd using mtd_mappper. The partitioning could either be created statically (using defines) or read from a specific flash location (e.g. esp32/esp8266) for a board/application. Since mtd’s are required to have a constant flash sector size devices with multiple sector sizes could define multiple mtd’s to support this.
Supporting different layouts for different bootloaders (riotboot, mcuboot, espboot, …) and upgrade methods might not be straight forward. A basic partitioning could be: bootloader, images, storages. The bootloader, images and storages could then be further subdivided if needed.
So instead of making a mtd on top of flashpage, mtd is directly calling the flash write routines. This would also move the usage of FLASH_BASE_ADDRESS to the definition of the mtd.
There is one limitation as mtd has a general config option to provide a in place read/modify/write routine. This option is however highly questionable as a interrupted write might lead to a broken system. So it might be good to disable/remove this option.
1 Like
I think that maybe it’s just a question of writing something down (The Roadmap) and communicating it. I’m not sure what format would be appropriate, I guess I will look for some example where a roadmap was established, and make a PR for this.