Benchmarking of Real-Time Operating Systems for Internet of Things Devices

Development archive
1

Hello RIOT community !

We are two students of the Université catholique de Louvain from Belgium and we work on our master thesis. The thesis is called "Benchmarking of Real-Time Operating Systems for Internet of Things Devices" and is about comparing the different implementations of open-source RTOS's currently available on the market.

In broad outline, we are planning to analyze the scheduling, switching context, memory management and/or any relevant metric dependant of the operating system.

Another goal is to compare networking performances or implementations of the various stacks available. We have currently no concrete planning for this part.

This project will be open source (because we <3 the open source).

Here are some questions we have for you: - "What would you like to see in this benchmarking project?"; - "What RTOS should we benchmark?"; - "What metric should we consider?".

We'd love to hear your recommandations or any help you can provide us with.

Also if you are interested about how the project will evolve, we'll be glad to update you on our progress.

If the project gain lot of interests, we will probably create a communication channel like Slack or Gitter.

Sincerely, Julien Gomez and Trong-Vu Tran

2

Hi guys,

that sounds awesome! Just some hints inline.

Hello RIOT community !

We are two students of the Université catholique de Louvain from Belgium and we work on our master thesis. The thesis is called "Benchmarking of Real-Time Operating Systems for Internet of Things Devices" and is about comparing the different implementations of open-source RTOS's currently available on the market.

In broad outline, we are planning to analyze the scheduling, switching context, memory management and/or any relevant metric dependant of the operating system.

Another goal is to compare networking performances or implementations of the various stacks available. We have currently no concrete planning for this part.

This project will be open source (because we <3 the open source).

Great, same here :-).

Here are some questions we have for you: - "What would you like to see in this benchmarking project?"; - "What RTOS should we benchmark?";

Independent from *RT*OS I'd suggest RIOT, contiki-ng, Zephyr, Mynewt. If you find common ground then FreeRTOS and in case of too much time, add mbed.

- "What metric should we consider?".

Just speaking for the networking part, next to the analysis that you were pointed to off-list, one might want to see comparable numbers for the other OSes. Furthermore a more detailed energy analysis is missing.

Also it would be nice to see how different OSes perform in an "all day" IoT scenario. For example: x devices in a multi-hop topology communicate sensor readings each y seconds. What's the energy consumption, what's the maximum throughput before break-down? And the like ...

Interoperability, feature sets, and device support are important metrics next to community agility which is obviously hard to measure but important when it comes to willingness/speed to migrate bug fixes.

We'd love to hear your recommandations or any help you can provide us with.

Also if you are interested about how the project will evolve, we'll be glad to update you on our progress.

I am interested.

Best Peter

3

PS: You may want to have a look at this work:

https://www.researchgate.net/publication/327561337_System_Architecture_Directions_for_Post-SoC32-bit_Networked_Sensors

Cheers Peter

4

Hi Julien,