How to (re)compile LWIP open source?

[TheOriginalMrWolf]

Hi,

I'm developing on ESP8266 (latest staging version), using ESPAsynTCP etc.

Think I've found a bug with the lwip open source implementation, and have written a small patch to test.

Knocking my head against a wall trying to figure out how to recompile the source rather than using the prebuilt binary. Using either Arduino or PlatformIO IDEs.

Hints & suggestions much appreciated!!!!

Thanks!!!

[ccooper21]

@TheOriginalMrWolf I recently ran into an issue for which I needed to better understand what was happening in the LWIP layer by enabling and adding additional debugging. I was able to copy the LWIP 'CPP' and 'H' files as needed into my sketch directory and make whatever changes I wanted. I think I also changed the '#include's that referenced the modified H files to have quoted names instead of equality symbols. I'm not sure if the latter was strictly necessary though. I was using the Arduino IDE for my work.

[d-a-v]

You also can:

cd esp8266/tools/sdk/lwip/src
make install

it works with linux and macos, not sure about windows.

[TheOriginalMrWolf]

@d-a-v - thanks for the suggestion, probably absolutely the right way to do it! Unfortunately I'm developing on Windows, so no win there I'm afraid :(

@ccooper21 - you put me on the 'right' track - thanks! 👍

(Using PlatformIO) I copied the lwip sources to lib\lwip in the project tree, but then had to flatten the entire structure down into that one directory, and STILL had to use the header files in the original distro directory. Very ugly, and very wrong.... but it worked!! lol...

Bit frustrated by having to do it this way -- would love to hear from someone (maybe @igrr?? hint hint) what the RIGHT way to do this in Windows is!!

[d-a-v]

Bit frustrated by having to do it this way

Install make (cygwin or mingw32 (mingw32-make), or so). The program 'make' is IMHO the only one lacking and needed under windows to rebuild lwip.

[igrr]

@d-a-v gave the correct approach; i'm afraid windows users need to install mingw and use make to compile LwIP. On Linux and Mac you can also select "Core Development Module" as your board, and choose "Open Source" option in "lwIP Variant" submenu. This will invoke make automatically during the sketch build process.

[TheOriginalMrWolf]

@d-a-v - I'll give it a go tomorrow. Thanks!!

@igrr - Thanks so much for all your fabulous work with this ecosystem! Will install & give it a go. Gotta work out how to bundle particular cuts of lwip with a particular project. Is it possible to put lwip into another directory tree (ie bundled with my code) and then have make compile & move the appropriate files into the right tools/sdk/lwip/... directories so that the whole project compiles?

[d-a-v]

I think the easiest way for other users to use your patch without pain and under windows would be making a github-clone of esp8266/Arduino with the proper patches already applied and compiled.

edit: I mean a "fork"

Your users would only have to clone your repo.
You would have to keep your repo sync-ed with master.
Then you might propose your patches as pull-request if you find this useful for the community :)

[TheOriginalMrWolf]

So installed mingw32 & ran make install. Worked beautifully, thanks (first time I've done this sort of stuff, so getting a bit overexcited...)!

Two quick questions:

• in order to get (PlatformIO) to link the correct prebuilt, I had to literally overwrite liblwip_gcc.a with liblwip_src.a. Didn't feel like the best approach... how can I tell the linker which binary to pick up?
• why does this need to be prebuilt - why can't we just treat it like any other arduino lib?

Thanks again!

[igrr]

You possibly need to modify some platform-specific build script in platformio to tell it which library to use, but i think you may be better off asking about this on platformio forums :)

Regarding why it needs to be prebuilt, it was so historically, we were using pre-built lwip which came with the SDK. Another reason is that lwip adds quite a lot to build time, especially because Arduino doesn't do parallel builds. So if you have a PC without an SSD and with low single core performance, there will be a significant difference in build time. Finally, to make LwIP buildable in Arduino we would need to adjust LwIP directory structure and/or include statements, which doesn't make it easier to update it when new SDK comes out.

[TheOriginalMrWolf]

Thanks! How do I tell Arduino IDE which lib to use?

[igrr]

See platform.txt. There is a variable called build.lwip_lib, which is used in the linker command line. By default it is set to -llwip_gcc. Board definitions (in boards.txt) can override this variable and set it to a different value, e.g.

coredev.menu.LwIPVariant.OpenSource=Open Source (gcc)
coredev.menu.LwIPVariant.OpenSource.build.lwip_lib=-llwip_src

[TheOriginalMrWolf]

PERFECT, THANKS!!

This pointed me in the right direction to hack up one of the PlatformIO scripts to force it to link the new binary & it worked perfectly.

I've now put in a request to the PIO guys to try to understand how to do this "properly" :)

[gjt211]

@igrr , thank you for your extremely hard work on this project.
In the file Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/tools/sdk/lwip/include/lwipopts.h I made the following change...

//#define TCP_SND_BUF                     2 * TCP_MSS
#define TCP_SND_BUF                     4 * TCP_MSS

Using your instruction above (from 18 April) I compiled my sketch (Uisng a Mac) set to core development module etc, and noticed many of the files in /lwip/src had their timestamps updated and no errors were seen when I compiled the sketch.
The problem is, I don't see any change in sketch flash or ram usage. I would expect the ram usage to go up.
I also don't see any timestamp changes anywhere else (e.g. in the lwip/include folder). I feel the compile completed ok, but it seems the updated lwipopts.h changes have not filtered through to the normal sketch compile process.
Do I need to do something else to actually use the recompiled lwip?