Runtime Configuration Breakout Session

Notes by Thomas (@tcschmidt):

Convergence strategy:

Find/end up with the best solution. For this

  1. Check functional differences against requirements: what needs to be added on either side for completion resp. a full-featured solution?

  2. Compare usability: Any notable difference?

  3. Measure resource overhead: Use identical configurations to compare (i) baseline overhead and (ii) scaling with configuration entries.

Then select this PR, which meets (1.) to (3.) best, add what is missing and complete.

If a moderator is needed, I suggest to ask José or Leandro, who wrote the initial RDM.

We should also think about “institutional independence” when it comes to moderation. Even though it looks like that, at the moment, moderation will not be needed, it might give a weird picture and lead to unnecessary grudges if one of the parties is working on the same team as the moderators. If moderation is needed, I can offer it and as a second person maybe @bergzand (though he has some requirements with CoRECONF though that might be a plus for the decision process), @oleg or @chrysn?

1 Like

Thanks @miri64: your support is greatly appreciated. José or Leandro were not meant as institutional supporters, but as representatives of the requirement side … but this is documented anyway, hence transparent.

On 20.09.2023 12:43, Martine S. Lenders via RIOT wrote:

[miri64] miri64 https://forum.riot-os.org/u/miri64 Maintainer 20 September

oleg:

If a moderator is needed, I suggest to ask José or Leandro, who
wrote the initial RDM.

We should also think about “institutional independence” when it comes to moderation. Even though it looks like that, at the moment, moderation will not be needed, it might give a weird picture and lead to unnecessary grudges if one of the parties is working on the same team as the moderators. If moderation is needed, I can offer it and as a second person maybe @bergzand https://forum.riot-os.org/u/bergzand (though he has some requirements with CoRECONF though that might be a plus for the decision process), @oleg https://forum.riot-os.org/u/oleg or @chrysn https://forum.riot-os.org/u/chrysn?


Visit Topic https://forum.riot-os.org/t/runtime-configuration-breakout-session/4006/2 or reply to this email to respond.

You are receiving this because you enabled mailing list mode.

To unsubscribe from these emails, click here https://forum.riot-os.org/email/unsubscribe/55b59619c66b6a10347aef9b9117ec0614281e62ff74b01c1eab0daaba9ad6f6.

I can participate if moderation is needed, but beware that I’m having opinions here, and they’re strong like the typing I like in config systems.

As an implementer of CORECONF, I’d very much like to stay part of the discussion, also making me a bit opinionated. As RIOT aims to implement open standards, I’d at least want to ensure that the solution we’re going to use can be used by a CORECONF implementation without having to go through nasty hoops.

I would put a step 0. in front here, which should consist of verifying that the requirements written in 2018 for the RDM are still matching the needs we have these days. A lot has changed in the 5 years so a next iteration of the requirements from the RDM might be required before we start comparing the different solutions. Furthermore the original document probably did not have input on the requirements from industry that voiced their needs during the Summit.

During the breakout session itself there was consensus on the split approach between set/get operations and commit/load/store operations. Meaning in practice that configuration can be applied to the runtime state, and must be explicitly committed to persistent memory to make it persist across reboots.

Other requirements noted was the need to differentiate between instances of same kind, for example multiple LEDs or multiple network interfaces. There was a clear need for this from the group present during the breakout session.

There was no real solution decided on for the internal representation of the different configuration nodes. Different access methods use different identifiers. A shell based configuration method might use string based paths, LwM2M uses namespaced integers and CORECONF uses globally unique 64 bit integers to identify a single node in the configuration tree. The main requirement on the internal representation is that it must be possible to map the different identifier to the identifier used by RIOT. As a side note, the internal representation in RIOT must not depend on the compiled firmware, ensuring that configuration nodes do not suddenly map to different modules after a firmware update.

One other thing that was clarified a bit in the breakout session is that configuration as used here can be distinguished between pure configuration of settings from an administrator, but also the second case where the device itself has some data that needs to be stored in persistent storage between reboots. This last case includes cryptographic nonces or frame counters (possibly used as nonce in crypto), or as really simple case a boot counter.

For both cases the requirements are pretty much identical except that the second case doesn’t need to be configured by the device administrator. During the breakout both cases were treated as ‘configuration’.

1 Like

Quibble, it’s 63-bit integers :slight_smile: The CBOR encoding for the YANG uses a delta encoding with a base SID allocated, and then a map with small integer offsets. Its probably possible to avoid doing the 63-bit addition. The 63-bit base value won’t be >2^32 for some time, but ya never know.

Review the draft: draft-ietf-core-sid-21 - YANG Schema Item iDentifier (YANG SID)
I despair it will never get finished.

1 Like

I see the RDM pretty much fulfilled by both sides. There is the export() API in the RDM, which I confused with bare saving a configuration item. By the RDM I understand that export_cb is more like a “do something with every configuration item”, where printing and saving are only examples. The export() in my implementation only means saving, but a mechanism to iterate over all configuration items is used internally for each API.

To check functional differences, I would propose we gather some criteria and questions in a shared (markdown) document. As a first starting point, this is what I have gathered so far. Or is there already a document?

Comparison of Runtime Configuration Implementations PR19557 and PR19895

Criteria

19557

19895

Basic iInterface and Description

  1. Which public interface is provided?
  2. What is each API supposed to be used for?

Data Model

  1. What internal data structure is used?
  2. How is data/metadata organized?
  3. How are complex data types and arrays handled?

Data Lookup

  1. How is configuration data identified?
  2. How does the lookup procedure work, given a valid identifier?

Data Correctness

  1. How is data assured to be correct, when committed?

Data Appliance

  1. How is new configuration applied to a subsystem?

Locking

  1. Do we need/want to have locking?
  2. If yes, how is it realized? How could it be realized?

External Usage

  1. What must be implemented by external code to integrate new configuration items?

Backend

  1. What is the relation between configuration items and backends?
  2. How is a backend selected?

Compatibility

  1. Given the model of a configuration item changes (same key, different data layout), what has to be done for migration?

How can CORECONF be integrated?

Use Case

  1. Which?
  2. Implement the use case and evaluate
    • RAM overhead
    • ROM overhead

Future Work

  1. What would you say are open implementation tasks to make the configuration system ready for release?

1 Like