Plans for switching over to a new network infrastructure in GNRC

Hi all,

after the release is before the release, so let’s use the drive to continue the good work.

Some of you might have noticed, that we are currently working on a big switch-over for GNRC-internal APIs, so if you are not involved in any direct GNRC programming or need access to the network interfaces this won’t most likely only affect you a little bit (the calls how to get an interface ID will change – actually simplify ;-)) to not at all.

This is quite a massive step, since basically most of the files that are doing operations on, or related to network interfaces (i.e. IPv6, 6LoWPAN, MAC and routing protocols) need to be touched. In addition we decided to integrate this network interface layer directly with the new, shiny and fixed neighbor discovery which saves us a lot of integration work into old stuff, but doesn’t make things easier. Some of that work is already done and in master, but most of it will take some time to adapt.

This is why we (the RIOT maintainers) decided to go a slightly different route than usual: we made a feature branch where all the integration work (and testing) for the new components will happen [1] and made a 1-2 week plan to get this branch re-merged into master as fast as possible. In that time this feature branch exist merging new changes to GNRC into master is highly discouraged to not risk merge conflicts in the gnrc_netif2_integration/master branch. We will notify you again when this embargo is lifted.

So what changes after all this is done (I call the new interface API gnrc_netif2 here, but in the final result it will be called just gnrc_netif):

  • Network interfaces:
  • Simplification of GNRC’s network interface architecture: currently there are 4 APIs (gnrc_netdev, gnrc_netif, gnrc_ipv6_netif, and gnrc_sixlowpan_netif) that represent network interfaces in some kind or form. This makes things complicated if you need e.g. in NDP information that is both device dependent (e.g. link-layer address) and IPv6 dependent (e.g. IPv6 address). This is why gnrc_netif2 merges all these APIs into one single API.
  • Tighter integration into GNRC’s principles: For most device related options interfaces worked like any other GNRC module and used NETAPI, but when it came to other stuff, like IPv6 addresses or 6LoWPAN configuration, other APIs were used. With gnrc_netif2 all options can be set via NETAPI.
  • Better thread synchronization: apart from the the fact that GNRC-externally the options are now set over NETOPT (i.e. in the thread context of the interface) we also use recursive mutexes GNRC-internally now to better synchronize the access to the interface. In the previous implementation normal mutexes were used (requiring some dangerous unlocking in the middle of an operation) or no mutexes were used at all.
  • Easier to extend: If you want to provide a new MAC protocol to GNRC you don’t have to copy all the thread-handler stuff anymore. Just implement the operations provided in gnrc_netif2_ops_t and let gnrc_netif2 do the rest ;-).
  • new neighbor discovery:
  • In comparison to the old neighbor discovery the new was designed more thoughtfully so it easier to maintain.
  • Most issues with the old neighbor discovery were fixed (because with the new design we actually were able to pinpoint issues within minutes, not within days as with the old NDP)
  • For the end-user the shell commands will change slightly: manipulations of the neighbor cache and FIB now are done using the nib command instead of a wide variety of commands.

I hope that’s all.

If you have further questions, don’t be afraid to ask.

Cheers, Martine

[1] https://github.com/RIOT-OS/RIOT/tree/gnrc_netif2_integration/master

Hi,

FYI there is a re-integration PR now: https://github.com/RIOT-OS/RIOT/pull/7925

Cheers, Martine

Hi,

we are at least a week behind schedule with this, so please everyone: have a look at https://github.com/RIOT-OS/RIOT/pull/7925 or https://github.com/RIOT-OS/RIOT/projects/6 and see if you can help with the review and testing.

Kind Regards, Martine

Hi,

thanks to extensive testing by Koen and myself over the last week, #7925 [1] is now in a mergable state (at least Koen and I think so ;-)). However Koen likes to have a second opinion by a more experienced maintainer before merging this PR. So I ask anyone who has some time on their hand for testing this ASAP. Remember that this PR is an aggregation of several PRs so the code review itself already happened. If you notice any drawbacks, I ask you to open an issue to report (on style issues you can comment inline as usual), we will then open a PR against #7925.

After the PR was merged we finally can lift the GNRC merge embargo again.

Kind Regards, Martine

[1] https://github.com/RIOT-OS/RIOT/pull/7925

Hi,

#7925 is now merged, so the GNRC embargo is now officially lifted. Thanks to all contributors and reviewers to this huge endeavor. You are awesome!

Cheers, Martine

Hi,

As #7925 was merged yesterday, there is now a nightly build with the gnrc_netif code merged. From this I present you the build size differences between today (with gnrc_netif) and yesterday (before the merge). Absolute numbers on a single application on a single board can be found by pressing the relevant diff. Selection of the tests, boards and metric (bss, data and text) is at the top of the page.

Some remarks: These are the diffs between the previous and the current build size, not absolute numbers. Furthermore, this is a comparison between nightlies, not from merge to merge, so all merges from yesterday are included in this diff.

Cheers, Koen

Hi,

yep, there is still some major optimization work required. But the feature set provided in #7925 is a little bit larger than the previous NDP implementation was (namely: the UNREACHABLE state as proposed in RFC 7048 is now fully working) + many places that made the old implementation smaller, were just plain wrong assumptions that were major quality defects in the old implementation. What is weird though is, that the 6LN configuration got so much bigger… I would have expected it to be way smaller :-/. (Also MSP430 [not listed] is unexpectedly large, but I blame that on the old GCC doing some bad optimization work).

Cheers, Martine