Unit test code coverage using gcov

Hi Guys,

Has anybody succeeded in getting gcov to work to check code coverage of the unit tests?

We will try to add support for this, but I want to make sure we’re not spending our time on unnecessary work.

Greets,

Toon Stegen

Hi,

as there is some discussion to split the unittests into several applications [1] and some stuff isn’t included into them for good reasons (driver test e.g.), I’m not sure if this should be constrained to unittests (though they are indeed a good starting point).

Regards, Martine

PS: I’m not aware of work going into that direction, so +1 in general.

[1] https://github.com/RIOT-OS/RIOT/issues/8941

I have been using gcov to visualize my coverage for the KNX tests. I did it from the command line, so no Makefile hacking, and only for native. Can share my commands that I have been using.

Hi Toon,

are you talking about native tests or embedded ones ?

For embedded I had some POC/wip that I ran with specific local setup in my previous job based on https://www.thanassis.space/coverage.html

https://github.com/iot-lab/openlab/tree/coverage

If there is an effort in this direction I am willing to help.

Cheers, Gaëtan

Hey Toon,

Has anybody succeeded in getting gcov to work to check code coverage of the unit tests?

I managed to get gprof working at some point. AFAIR they're using the same infrastructure (PC sampling, saving it somewhere, then dumping the file) as gcov.

gcc's "-pg" needs library support. For that I have a branch [1]. It is quite old, but maybe it helps.

Kaspar [1] https://github.com/kaspar030/RIOT/tree/add_gcov_support

Did anyone had any luck with that lately? I asked @BasilFX in a PM for the commands he mentioned in

and this is (with his permission) what he provided me with:

Problem is: I don’t even get *.gcda files generated with my test, only *.gcno files:

$ BOARD=native LINKFLAGS="--coverage" CFLAGS="--coverage" make -j flash test
[…]
$ find . -name '*.gcda'
$ find . -name '*.gcno'
./bin/native/gnrc_sixlowpan_frag_sfr/gnrc_sixlowpan_frag_sfr.gcno
./bin/native/gnrc_ndp/gnrc_ndp.gcno
./bin/native/gnrc_ipv6/gnrc_ipv6.gcno
./bin/native/gnrc_sixlowpan/gnrc_sixlowpan.gcno
[…]
./bin/native/gnrc_netif_ieee802154/gnrc_netif_ieee802154.gcno

Same result when using -fprofile-arcs as well (as described here), or when trying to run one of the unittests (in case they do something different than my test).

A hint might be, that when using make term instead of make test and hitting Ctrl+C after the tests completed (still using embunit for the execution), I get

tests/<test name>/bin/native/tests_<test name>.elf: not implemented

spammed to my terminal, but not sure how to solve this or what it means.

I also tried some other methods such as LLVM’s Source-based Code Coverage (for which I don’t get the *.profraw file generated) as well as LLVM’s Sanitizer Coverage (using make all-asan on native) for which after several linking errors where fixed by including modules and features I don’t really need I still got stuck, since the generated *.sancov file did not provide any useful output (e.g. generating a *.symcov file yielded nothing).