Cfg page contribution

Development
1

I have made a pull request:

it will naturally get squashed before acceptance, but I didn’t want to do that yet. I have worked too long on this, and I’m relatively disappointed in the result. The core problem I have is the mtd interface is NAND rather than directly addressable NOR flash. I am all for supporting NAND flash, but I think that variables in the way that I’m envisioning them would likely always be in NOR flash.
If I can have some other interface that is directly mapped then for regular use, the ram requirements is probably around 32 bytes.

During the garbage collection phase when the page is full, it is probably possible to avoid any additional RAM.
But, right now, I’m eating another 4K page (ON THE STACK. Horror. I’m bad) to do the work. I’m stomping on the previous key values, since I know it’s ram.
I could probably do without the extra page if I did a lot more mtd transactions.

There is some annoying CBOR internal knowledge, and also some nanocbor internal knowledge which I’m not happy with. There are perhaps a few nanocbor functions that would help, including:

  • how big is the value in front of the cursor?
  • copy the value in front of the cursor to a writer
  • write an int which is of size X (1,1+1,1+2,1+4,1+8).

Keys are currently unsigned integers. It could be expanded to negative integers. I don’t think that other key types are really needed: a particular application will need to keep a registry of what values it needs. Maybe negative integers could be RIOT-wide allocated or something. At present, there is only a routine to get/set text strings. Expanding to byte strings is trivial, and I’ll need that immediate for storing IDevID.

One reason that I went with CBOR encoding is that it is stable across architectures and compilers, and also will make the key/value pairs really easy to spot if you have to do a dump of everything. It also means that it is possible to construct variables at boarding programming time (IEEE OUI mac addresses? IDevIDs, SerialNumbers), which can be written easily. I have some makerdiary boards, nRF62840-MDK on which I’ll be testing this code in production. I want to make it work on ESP32, but there are some flash layout things there that I haven’t yet understood.

I have considered if it would make sense to have a two-level variable storage, as some variables never get changed, so they don’t really need to be re-written each time, do they?

ps: I find the MTD PAGE/SECTOR stuff backwards. I think that PAGES should be made up of multiple SECTORS, where SECTORS are the smallest thing that can be erased. But, I find that it seems to be other way around.

p.p.s: chrysn convinced me that none of this code had to deal in anyway with riotboot, except that I want to put these pages in the part of the flash that riotboot manages. This will likely be a per-board configuration, and I haven’t worked that part out yet.

ppps: I will probably focus on the IDevID and private key storage that I really care about for awhile.

2

I already have one 4K static allocation of ram for the read buffer. I don’t want a second one! Since the GC algorithm finds the “last” value for each key, if it could take deeper knowledge of the CBOR, it could probably just slide all key/value pairs towards the end of the (RAM) page, and then pull them all back to the beginning. That would avoid the second page. But, I was originally thinking that I’d be working on NOR flash, so I didn’t want to memcpy stuff all over. The read routines could live some buffer that was big enough for the largest key/value size. I don’t have to read the entire variables in at once. And the obvious (premature) optimization is to do that read only once.