Providing lwIP's HTTPd as RIOT module

Development Networking
1

Hi @miri64 Could you please give me information about how to add lwip_httpd to LWIP_MODULES correctly

2
  1. It should be enough to add a target for an lwip_httpd module to the lwIP pkg’s Makefile. For this lwip_httpd needs to be added to the LWIP_MODULES list here and a target lwip_httpd needs to be defined below that where the other targets are. Something like
    lwip_httpd:
	$(call make_module,$@,$(PKG_SOURCE_DIR)/src/apps/http)
  2. One or more aptly named (RIOT) pseudo-module are required for further inclusions. These should be added to the list of lwIP’s PSEUDOMODULES here.
  3. Dependencies of those pseudo-modules to lwip_httpd should be added to lwIP’s Makefile.dep (and other modules that lwip_httpd might need such as SSL or TCP e.g.)
  4. The respective config options (including the LWIP_HTTPD... activation flags) need to be added to pkg/lwip/include/lwipopts.h, depending on those pseudo-modules:
    #if IS_USED(MODULE_LWIP_HTTPD_...)
/** @brief some doc */
# define LWIP_HTTPD_...   1
# define LWIP_HTTPD_MAX_REQUEST_URI_LEN  ...
#else
# define LWIP_HTTPD_...   0
#endif /* IS_USED(MODULE_LWIP_HTTPD_...) */

Of course, it would be great to have this documented in a more general sense with the pkg (e.g. in it’s doc.txt). However, I currently do not have the time to do that properly at the moment and from experience it is always better if someone with fresh eyes starts something into that direction (and the experts maybe correcting things later). If you like, you can add that as well, based on your experiences of porting lwip_httpd.

1 Like
3

Hi @miri64

Thanks for your clear answer.

How can I exclude a single file from building?

I am getting “multiple definition” errors for the file__index_html, file__404_html and file__img_sics_gif definitions in fsdata.c.

This topic has been discussed before on ST Community. Recommended solution is to remove ‘fsdata.c’ file from the build process. Although my fs.c file doesn’t have an #include ‘fsdata.c’ directive I want to try this solution. However I couldn’t decide how to exclude fsdata.c from build process correctly.

4

You can control the C-files that are build using the SRC macro/environment variable. E.g. by doing:

LWIP_HTTP_DIR=$(PKG_SOURCE_DIR)/src/apps/http

lwip_httpd: export SRC=$(filter-out fsdata.c,$(patsubst $(LWIP_HTTP_DIR)/%.c,%.c,$(wildcard $(LWIP_HTTP_DIR)/*.c)))
lwip_httpd:
	$(call make_module,$@,$(LWIP_HTTP_DIR))

Not sure this is the nicest solution though. Maybe @fjmolinas has some hints on our build system here.

5

Hi @miri64 , thanks.

I tried this change but it didn’t work.

I can’t figure out where the “multiple definition” errors are coming from. I’m researching on the one hand, but I would be very pleased if I could get support from you as well.

EDIT: Previously I said

but this is not true, fs.c file has #include HTTPD_FSDATA_FILE

6

Can you please try a make clean and try again?

1 Like
7

It’s OK now, thanks a lot. I continue my lwip hpptd journey.

1 Like
8

Don’t hesitate to open a (draft) PR in Github, if you have something somewhat workable :slight_smile:.

9

Hello,

Last month, I tried to replace the lua firmware of my esp8266 wifi sensors with riot firmware.

I first tried with lwip httpd ssi / cgi and I was able to make it work but I found it overkill for this use case so I tried to do something simpler using the sock_tcp API but there is still a memory saturation problem that I could not solve.

My working lwip httpd code is here: https://github.com/hugueslarrive/riot_lwip_httpd.

You need to create a Makefile.wificonf and adjust RIOTBASE in the Makefile. The Makefile add a small patch to the the lwip package to enable httpd but patches are applied before it was added the first time you launch make thus it will fail to build. You just need to launch make one more time.

I had to tweak the TCP maximum segment lifetime to avoid memory saturation.

My sock_tcp attempt is here: https://github.com/RIOT-OS/RIOT/pull/16872 but the TCP_MSL tweak does not seem to be enough for this one
 I am looking forward to gnrc_ipv4 because lwip’s dynamic memory allocation is a nightmare!

2 Likes
10

Hello @hugueslarrive ,

Thank you for your valuable contribution to this discussion. I am very pleased to meet you.

I am an engineering manager and therefore my primary interest is to keep my knowledge and skills up to date by practising, rather than producing something.

My experiments with lwip here have been very useful to me in this sense, and they continue to do so. On the other hand, I can only do these studies in my spare time so unfortunately the work goes slowly.
I wanted to reply to you without delay, but I will examine your work in detail within a few days.

I agree with you on this. Yes, the future is in ipv6, but the infrastructure for ipv6 is not fully ready yet. For example, my office modem and my internet service provider do not support ipv6. So I can’t do any of the ipv6 apps, I think RIOT_OS should give more support to Ipv4 for a while. EDIT: Same goes for HTTP too!

BR, Murat

11

Maybe have a look at WIP: Add IPv4 GNRC by wysman · Pull Request #12988 · RIOT-OS/RIOT · GitHub The initial contributor is inactive now and taking his work over would be much appreciated.

1 Like
12

This is great news. I hope someone knowledgeable and talented on this subject will find opportunity to finalize the work :pray:

13

I guess, when you write “memory saturation”, you mean that you run out of memory? I think that many people would write “memory exhaustion”, as “saturation” kinda implies a slightly different thing that memory doesn’t really do (vs other solvents
)

14

riot-lwip-httpd-screen-capture
riot-lwip-httpd-screen-capture836×256 17.9 KB

15

Your data is consistent, since 0 ° C is the water solidus, humidity freezes and falls into frost as in “The day after tomorrow” movie !

If the module stop responding every two seconds, the interface will show some ‘-’ insteed of the digits thus the network part seems to be ok.

Zeros mean that there is probably no sensor wired. The sensor gpio pin is GPIO2 as defined in the Makefile.

Have you tried the munin plugin ?

munin_plugin
munin_plugin1378×960 112 KB

16

No, I mean memory congestion in network context. DEBUG_MEM says “mem_malloc: could not allocate 
 bytes” but some memory will be freed soon thus you can flood it with a too fast request stream, but as soon as you slow down and the requests lifetime expires, some memory will be freed and normal operation will resume.

Let’s say that we have aboute 10 KB of memory and each TCP request need about 330 Bytes during two minutes, this result in a bandwidth of 0,25 request per seconde. Reducing the segment lifetime from one minute to one second lead into increase this bandwidth to 15 requests per second. We could obtain the same rĂ©sult by increasing the amount of memory to 600KB (MEM_SIZE in lwipopts.h).

17

Absolutely :slight_smile:

As you may have guessed, I just wanted to show that your code worked easily with the httpd settings I made earlier.

We just need to decide where and how to define the lwip_httpd flags like LWIP_HTTPD_CGI, SSI, CGI_SSI. I defined them in pkg/lwip/include/lwipopts.h, when you redefined them in Makefile, it caused a “multiple definition” error.

I didn’t look at the sensör side right now because I will deal with the sensor side in detail for another subject; I want to implement a lighting controller with DALI protocol. Dali master driver is ready and has worked very well so far. The user interface seems ok too. I’m trying to handle the web interface now.

No, I didn’t know it but I’ll try, thanks.

I want to make the web interface visually better and more useful. I also want to put a “password” on web access. This password can be built into the firmware, just like esp8266/32 ssid and pass. Also, I want to test esp8266/32 as an access point. I would be very grateful if you have any advice on these matters.

BR,
Murat

18

Moreover, I would not like to see memory as a deposit that could be exhausted.

19

Since this whole thread started to become more about implementation details of lwIP’s HTTPd, I moved it to its own topic in the “Develpoment / Networking” category.

1 Like
20

I think that is something that must be expected in a constrained environment.