Hi all,
Daniel Camara from the ns3 team (in cc) asked me a question that I thought I’d better relay here to gather the most informed opinions:
“Is there any conceivable way for all global variables in RIOT to be declared as thread local? If this could be done adaptation with ns-3 would be MUCH easier.”
Any opinion on this question? Or on how to better formulate the question?
Caveat: Daniel does not know RIOT code well, but knows a lot about ns3, and about porting to ns3 (as he was heavily involved in the porting of Contiki to ns3).
Best,
Emmanuel
Hi Daniel & all,
Daniel Camara from the ns3 team (in cc) asked me a question that I thought
I'd better relay here to gather the most informed opinions:
"Is there any conceivable way for all global variables in RIOT to be
declared as thread local? If this could be done adaptation with ns-3 would
be MUCH easier."
Any opinion on this question? Or on how to better formulate the question?
It would be possible to extend all of RIOTs global variables with a macro
like RIOTGLOBAL and have that expand to _thread for a ns3 port. Not
really nice, but we could do that.
What I don't really see is the performance gain. Maybe you can shed
some light here.
Thread-local storage requires the threading library to update all
global variables and the context on every thread switch. Not using
threads on the other hand would require the system scheduler to update
the context. To me it appears that there is overhead involved in both
possibilities that probably cancel each other out.
Using processes instead of threads has the benefit of separated
address spaces though. That means that memory errors in RIOT would not
lead to the whole simulation crashing.
Furthermore I suspect running RIOT as a thread might lead to conflicts
with RIOTs own pthread implementation.. maybe someone from that domain
can say something about this.
Cheers, Ludwig
Hi Judwig,
To be honest with you, I am not even worried about the performance problems linked to the thread/process trade-off. Even though we, yes, that may play a significant role.
My point is more in terms of “easy to implement”, ideally, if RIOT wasn’t dependent on any globals we could model the riot nodes as objects in a C++ framework, and all the iteration among nodes could be as simple as function calls. However, to minimize changes, the second easiest way would be to have all the nodes modeled as threads. The advantage is that we can get rid of the biggest hassle we have with process that is manipulating the shared memory area. Other advantage is that the debugging process is somehow simpler.
It is not MANDATORY, I was just curious. (OK, yes, you can say I am lazy, and that is true :)… but It is more polite to say… I would like to accomplish my task, with the minimum effort possible. I guess we can say it is an “optimization” problem to where we want to minimize effort. 
However, what you said kind of scared me a little, what is the MTBF of Riot? If crashing is more or less expected… then your concern is more than justifiable and process is maybe the way to go. Even though in a simulation, if a machine crashes, and it wasn’t expected… that may pollute the results and the simulation should be aborted anyway… well at least a BIG warning should be raised… unless it is the homework of networks 101, in that case… no one cares.
Best regards…
Daniel
Hi Daniel,
To be honest with you, I am not even worried about the performance
problems linked to the thread/process trade-off. Even though we, yes, that
may play a significant role.
My point is more in terms of "easy to implement", ideally, if RIOT wasn't
dependent on any globals we could model the riot nodes as objects in a C++
framework, and all the iteration among nodes could be as simple as function
calls.
I have trouble imagining how that would work - even without global
variables. RIOT has a preemptive scheduler and busy loops. That is:
RIOT threads can depend on preemption.
However, to minimize changes, the second easiest way would be to
have all the nodes modeled as threads. The advantage is that we can get rid
of the biggest hassle we have with process that is manipulating the shared
memory area. Other advantage is that the debugging process is somehow
simpler.
It is not MANDATORY, I was just curious. (OK, yes, you can say I am lazy,
and that is true :)... but It is more polite to say.... I would like to
accomplish my task, with the minimum effort possible. I guess we can say it
is an "optimization" problem to where we want to minimize effort. 
OK then - like I said - it is possible.
I think the bigger question is about the feasibility regarding
threading implementation conflicts. (Anyone willing to contribute some
insight on the possibility of running RIOT inside a pthread here?)
However, what you said kind of scared me a little, what is the MTBF of
Riot?
That depends on how fresh the code is.
I mean - the network stack for example is under heavy development at the
moment and during a phase like that unchecked NULL pointers seem to
slip through every once in a while. Of course we will get to a point
where the stack is in sane state, but I assume one might want to use
the simulator to verify some implementation aspects during
development.
If crashing is more or less expected... then your concern is more
than justifiable and process is maybe the way to go. Even though in a
simulation, if a machine crashes, and it wasn't expected.... that may
pollute the results and the simulation should be aborted anyway.... well at
least a BIG warning should be raised.... unless it is the homework of
networks 101, in that case.... no one cares.
Of course, the results will be spoilt. But I don't know ns3 - if it's
no big deal if it crashes that's fine with me as well. If it can cause
damage (e.g. by corrupting files one might still want to have used
afterwards) that might be an argument for separate processes.
Cheers, Ludwig
Hi Ludwig,
I have trouble imagining how that would work - even without global
variables. RIOT has a preemptive scheduler and busy loops. That is:
RIOT threads can depend on preemption.
Humm, scheduler it is easy, in fact, we will NEED to threat any way by
reflecting the Riot scheduler over the ns-3 scheduler, the busy waits...
well... that is another history... it would surely be a problem :(... don't
know I would need to think more over it.
OK then - like I said - it is possible.
I think the bigger question is about the feasibility regarding
threading implementation conflicts. (Anyone willing to contribute some
insight on the possibility of running RIOT inside a pthread here?)
> However, what you said kind of scared me a little, what is the MTBF of
> Riot?
That depends on how fresh the code is.
I mean - the network stack for example is under heavy development at the
moment and during a phase like that unchecked NULL pointers seem to
slip through every once in a while. Of course we will get to a point
where the stack is in sane state, but I assume one might want to use
the simulator to verify some implementation aspects during
development.
That is normal, and it is the price to pay to use the latest version of an
ongoing project. If this is the case, this does not scares me by no means
:).
Of course, the results will be spoilt. But I don't know ns3 - if it's
no big deal if it crashes that's fine with me as well. If it can cause
damage (e.g. by corrupting files one might still want to have used
afterwards) that might be an argument for separate processes.
Ok Ludwig, there are two mind states that need to be clear and distinct
one from the other. There is "yours", as RIOT developer that want to test
your development and verify the behavior of RIOT over a more complex
network setup. That is valid and, I guess, it is the why you, as community,
would be interested in the adaptation. The second state of mind is the guy
who wants to use this framework as a tool to test his own algorithms and
theories. Some one that will use RIOT+ns-3 as as a medium to see other
things, I am not interested in the tool itself it should be transparent to
me. I mean, if the tool does not work correctly it is like you said to
me... "Look I have this hammer here and you can use..... but it has some
wholes on it!!! So it will not work every time!". I may use your hammer,
if I have NO other tool... but just until I find a rock that is nearby :).
Best regards...
Daniel
Hi Daniel,
[…]
Of course, the results will be spoilt. But I don’t know ns3 - if it’s
no big deal if it crashes that’s fine with me as well. If it can cause
damage (e.g. by corrupting files one might still want to have used
afterwards) that might be an argument for separate processes.
Ok Ludwig, there are two mind states that need to be clear and distinct one from the other. There is “yours”, as RIOT developer that want to test your development and verify the behavior of RIOT over a more complex network setup. That is valid and, I guess, it is the why you, as community, would be interested in the adaptation. The second state of mind is the guy who wants to use this framework as a tool to test his own algorithms and theories. Some one that will use RIOT+ns-3 as as a medium to see other things, I am not interested in the tool itself it should be transparent to me. I mean, if the tool does not work correctly it is like you said to me… “Look I have this hammer here and you can use… but it has some wholes on it!!! So it will not work every time!”. I may use your hammer, if I have NO other tool… but just until I find a rock that is nearby :).
We’re on the same wavelength here, and I’m not sure it is worth discussing too much this question.
While the core of RIOT is extremely stable, it’s just that one cannot expect a whole OS + the bleeding edge of full network stacks under development to be as stable as a 1k lines of code implementing a simple network protocol on top of ns-3.
For instance anybody testing Contiki+ some new networked application on ns-3 has realistically exactly the same concerns, which is inherent to this type of approach. If it made sense to port Contiki to ns-3 then it makes sense to port RIOT to ns-3. That’s a simpler way to see it 
So let’s just do it and focus on the “how”. Other questions are irrelevant in this context, in my opinion.
Does it make sense?
Cheers,
Emmanuel
Hi Emanuel,
Don’t get me wrong, I am not judging, complaining, or something like that, just stating the fact that we have two VERY distinct clients here, just that! Even though this obvious for me, it needs to be clear to all. Well, I guess, even if the only clients were RIOT developers, it would already worth the effort… not that cool, we could do better, but probably would already pay the price… and the ways to implement the adaptation could be highly dependent on it.
So yes I agree with you, we need to focus on hows, but I also partially disagree because this specific point may impact the way the adaptation will be made. If you decide the public is just the RIOT development team… and you don’t care for the second class of users (what is completely fair), more “home made” and code dependent options could be considered… , for example, we wouldn’t need something that is fairly independent of versions of ns-3. You could fix your code over, lets say, version 19 and that is it would be more than enough for you.
Once more, don’t get me wrong! Not judging, just trying to understand :).
Hi,
I am confused - I was not arguing for forcing buggy code on people.
I just wanted to say that it might be worth the extra mile (the extra
work of supporting processes instead of threads).
Anyway - let's find out if threads are possible at all first and
talk about this if it's an option at all.
Regarding busy loops - I guess it might be an option to say that the
ns-3 port does not support those, it's not like they are an essential
part of RIOT (at least not to my knowledge, I could only find
occurrences of busy loops in cpu specific code).
Cheers, Ludwig
Hi!
Regarding busy loops - I guess it might be an option to say that the
ns-3 port does not support those, it's not like they are an essential
part of RIOT (at least not to my knowledge, I could only find
occurrences of busy loops in cpu specific code).
And these could/should be probably replaced by some interrupt driven
implementations. The only busy loop that can be definitely not be replaced by
something interrupt driven that I remember by heart is the delay function to
let the LEDs blink at startup (because hwtimer is not initialized then) - and
I guess that's a non-blocker.
Cheers,
Oleg
Even though I have a close relation with the ns-3 community, and I have no information whatsoever, on changes that could affect the adaptation, I can’t guarantee this will not happen in the future. The only way to be fully independent of changes would be to fix a version and go with that for ever, and NO, I don’t think it is a good idea.
The other way to be independent is to create a thin specific control layer, with a generic interface. If something change in any one of the sides (RIOT is also a living project, things will change for sure in the future) just the thin implementation need to be changed, the rest will stay the same.
As I understand the adaptation work should be done is as it was a Middleware between RIOT and ns-3. For RIOT ns-3 will be like a new architecture, a new sensor where it will be running and for ns-3, RIOT will be just a new kind of node, one should be as blind about the behavior of the other as possible. Something like:
Hi Daniel,
I think what you suggest makes sense. RIOT and ns3 are living projects, so will evolve and it is not reasonable to fix one version of any to achieve the port. Fortunately, the core of RIOT and of ns3 should be stable enough for it to be worth it to design the thin layers-based middleware you propose. Let’s refine this idea, I think this is the right direction.
Cheers
Emmanuel