Last chance for Kconfig dependency modelling

Development
1

The idea of dropping Kconfig as a dependency modelling system has been floating around the community for a while now. After yesterdays VMA, it became clear that the amount of resources needed to complete the migration were just not there and the maintenance burden of supporting what was already migrated was getting too high to just let it linger.

It has been ongoing for 3 years, lots of work was put into it, it also exposed and taught us many things about our current dependency modelling system. Fortunately, it was never used by default and simply avoiding the TEST_KCONFIG=1 flag is essentially the revert (and cleaning up any old files).

I am making this post to see if there is anyone in the community that not only wants Kconfig to be used for dependency modelling but is willing to contribute non-trivial resources to the migration effort. If so, please speak up!

If nobody is willing to invest the time needed by the start of 2024 then we will disable the Kconfig tests from the CI and slowly cleanup what needs to be cleaned.

This does not mean Kconfig will be gone as it is still used for configuration settings (ie. CFLAG += -DCONFIG_*) just the dependency resolution (ie. TEST_KCONFIG=1) will be removed.

2 Likes
2

I’m happy that after more than 4 years we can finally conclude the Kconfig migration.

I hope we learned something through it, even though the work did not end up being fruitful. In my option the biggest problem was the lack of a shared positive outlook on what Kconfig would brings us - in the end it looked like we’d get slower builds with a more brittle config surface. In combination with the pain it caused on everyone submitting a PR with some new functionality who then also had to model it into Kconfig, I’m glad to see it gone.

Will you take care of sending the Kconfig removal PRs? That is

  • disable Kconfig builds in CI (so we can get those PRs stuck on Kconfig issues in)
  • remove all Kconfig / app.config files
  • clean the build system from the remains of Kconfig

Thank you for your work!

3

Huh, we only talked about the Kconfig dependency modeling, not about all kconfig? We still need those for the latter.

4

probably the best is that @MrKevinWeiss summarizes what is needed by the start of 2024, because he started the thread.

5

I don’t understand, what good is Kconfig without dependency modeling? How would that even work - just expose all config options always?

6

Configuration exposure. Just type make menuconfig in any application that has not TEST_KCONFIG=1 defined and you will see that it does work and that it can help finding your way to the labyrinth that is RIOT’s compile-time configuration.

The dependency modeling, i.e. the part that is such a hassle to port (the one activated by TEST_KCONFIG=1 and tested by the CI with the Kconfig tests), is and always has been orthogonal to that as pointed out in @MrKevinWeiss original post:

7

My understanding from the discussion at the VMA was that we “drop Kconfig altogether”, not only the dependency modelling part.

8

The minutes do not say anything in that regard and this was not my understanding from the conclusion. And, as far as I understand, @MrKevinWeiss understood it like that as well?

Where is the harm in keeping the compile-time configuration? On the contrary, I see huge drawbacks when we take the step back from all the configuration exposure @jia200x and @leandrolanzieri did in the beginning.

My understanding was that the dependency modelling and the resulting CI testing was what pains most maintainers. The Kconfig compile-time configuration is so low-profile, I think its worth keeping it and porting it to whatever solution comes up as an alternative to the Kconfig dependency modelling. The latter btw was discussed:

9

Ok, sorry for the delay and happy new year everyone!

In my option the biggest problem was the lack of a shared positive outlook on what Kconfig would brings us

Agreed, when it was first introduced the belief is everyone would be working and contributing to it. However, like so many things in RIOT, there was only one champion at a time which can make things harder when the work isn’t so exciting.

in the end it looked like we’d get slower builds

It looked like that because the testing was so thorough. In fact, the only thing that Kconfig would have affected would be the initial info board-supported command used for the CI to determine what should work or not. If the migration would be complete and the structure would allow for a board wildcard entry (solved by Zephyr) it would be no different that what we currently would have with make. Comparing the hashes (which required multiple builds) did catch many issues not only in dependency modelling but also in varying configurations.

a more brittle config surface

I would say more structured, once one gets used to it failing early and being able to properly describe why something failed is a great asset. There were some constraints with the language that made some things very hard though (no circular dependencies being our biggest issue). The way we implemented it though, opposed to Zephyr which was completely bottom up, still allowed us quite a lot of reliable options…

In combination with the pain it caused on everyone submitting a PR with some new functionality who then also had to model it into Kconfig,

I agree with this… It simply dragged for too long. We should learn from this and rather have something like the i2c embargo where there is a small pause that forces many to finish rather then let something operate along side, forcing duplicate work.

I’m glad to see it gone.

It would have been nice to have all done and working properly but it just would take too many resources for not enough gain for most of the RIOT developers use cases (probably menuconfig would only be used by newer users).

Will you take care of sending the Kconfig removal PRs?

I can do that or at least review others work.

disable Kconfig builds in CI (so we can get those PRs stuck on Kconfig issues in)

That is the most important and should be pretty easy!

remove all Kconfig / app.config files

I think you mean all app.config.test files since we still will use if for generic configurations using the dependency resolution of make to handle the various config values that require knowledge of when modules or packages are being used.

clean the build system from the remains of Kconfig

I can get rid of anything we won’t need.

My understanding from the discussion at the VMA was that we “drop Kconfig altogether”, not only the dependency modelling part.

That isn’t what I had in mind. The config exposure works and is nice, no reason to remove that work.

10

If that works independent from the dependency resolution it’s nice and keeping it sure helps discovering the available config options. My only concern is that this will bit-rot away quickly.