So my aim was to compile Nginx package, with possibility to add custom modules to it later.
But Nginx is just an example here, with this method you should be able to compile other packages too.
This tutorial is a result of two weeks of trials and errors. I never had any experiences with Linux compiling.
So idea of cross-compiling (especially for such a non-standard platform as Turris Omnia) was for me just a big brainf*ck.
Nevertheless I had a big urge to learn it, because I wanted to have bigger freedom in what I can run on my Omnia.
The biggest problem was that I didn’t even know how to search on this subject, I did’t know how to formulate my questions in Google.
It is also possible that there is somewhere a comprehensive tutorial how accomplish my task. Well, I didn’t find it.
Now, with help of a few friends ( thanks to: glaeken, kixorz, Pinky, brozkeff) I have some small successes, so I decided to publish my findings.
Disclaimer: I’m still a noob in this subject, some of the stages of this process are still a mystery for me - I don’t fully understand them.
So parts of this tutorial are probably not optimal and can be written better - feel free to comment on them, your feedback is welcomed.
The most important is: it worked for me, so I hope it will work for people who will find this post in the future.
So this is how to do it:
You need Ubuntu 14.04.
My first try was on Debian10, and that was a failure. Later on I discovered that I need Ubuntu 14.04:
Unable to build turris openwrt on debian - compile error in freadahead.c in m4
I’m assuming you are starting in /root folder. Create a folder in which you will have your cross compilation stuff
I will call my “cc”, go to that dir
mkdir cc cd cc
- We are now in /root/cc/
- This will download the SDK as a file openwrt-sdk.tar.xz
- Unpack archive, remove archive, change extracted dir name
tar xvf openwrt-sdk.tar.xz
mv openwrt-sdk-* openwrt-sdk
- Now you have the official SDK in folder openwrt-sdk
Now we need to install some prerequisites
apt-get install make libncurses5-dev ccache subversion build-essential zlib1g-dev gettext libssl-dev
I tried to keep them at minimum, but there might be more needed.
If you will encounter any errors, try to install some more as described here:
Let’s download official Turris OS 3.xx repository from NIC
Go to website https://gitlab.labs.nic.cz/turris/openwrt/-/tags
Pick and click on your Turris Os version, (I will pick v3.11.16)
Click on the Download icon, right click on “tar.gz”, and Copy link location, in my case:
Download and unpack:
wget https://gitlab.labs.nic.cz/turris/openwrt/-/archive/v3.11.16/openwrt-v3.11.16.tar.gz tar xvf openwrt-v3.11.16.tar.gz cd openwrt-v3.11.16
- Now you are in the openwrt-v3.11.16 directory, and we will do everything from here as our base directory
And here starts the part in which I’m not entirely sure if all the following steps are needed, and if their order has a meaning
Run the official compiling script - this will probably be not successful at the end, so don’t worry.
This process will configue our compiler to a proper state, and will prepare packages we will need in the next steps
It will start compilation of all the packages, but we only need Nginx package for now, not all of them.
This process almost always failed for me. If you will wait and it will compile for you successfully, then you might not get any errors in points 10. 11.
But if it will fail or you don’t want to wait for the whole process to finish, you can interrupt it in the middle.
It should be safe to cancel the process after the line ‘make -C tools/libtool install’ is finished. Just press CTRL+C whe you will see: make -C tools/gmp compile
This is a mystery for me. I have a feeling that there is a few packages crucial/needed for compilation which can be
picked in menuconfig. And without them successfully compiled, there will be errors in points 10 an 11. I don’t know. Anyway let’s continue.
Now let’s prepare nginx package
./scripts/feeds update -a ./scripts/feeds install nginx make package/nginx/download make package/nginx/prepare
Then open menuconfig to choose which nginx modules you want to compile:
You should see there, first two fields set as shown below:
Target System (Marvell Armada 37x/38x/XP)
Target Profile (Turris Omnia)
Now, go to:
Network -> Web Servers/ Proxiesand find nginx.
Press Space on it (it will get M flag), then go to its Configuration, enable any modules you need, then exit, exit, exit, and save the configuration at the end
make V=s package/nginx/compile
You might get an error here (if not, jump to the next point)::
cc/openwrt-v3.11.16/staging_dir/toolchain-arm_cortex-a9+vfpv3_gcc-4.8-linaro_musl-1.1.15_eabi/lib/libgcc_s.so.’: No such file or directory*
This is the moment I didn’t know how to get these libraries (there would be more of them) by “normal way”. At that point, ‘compile_fw omnia’ never finished successful for me.
So I came with idea that I will take it from openwrt-sdk-18 (the one we downloaded at the beginning).
I don’t know how safe and future-proof this approach is. Anyway let’s copy all libraries from the openwrt-sdk:
cp ../openwrt-sdk/staging_dir/toolchain-arm_cortex-a9+vfpv3_gcc-7.3.0_musl_eabi/lib/* staging_dir/toolchain-arm_cortex-a9+vfpv3_gcc-4.8-linaro_musl-1.1.15_eabi/lib/
make V=s package/nginx/compile
And here you might get another error (if not, jump to the next point):
configure: error: C compiler cannot create executables
See `config.log’ for more details
If we will look into build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.15_eabi/pcre-8.41/config.log
We will see:
configure:3823: checking whether the C compiler works
./configure: line 3847: arm-openwrt-linux-muslgnueabi-gcc: command not found
So again, I don’t know what is the proper way to deal with that, but I found out that if I will use a toolchain bin directory from the original openwrt-sdk it will work, so:
echo PATH=\$PATH:/root/cc/openwrt-sdk/staging_dir/toolchain-arm_cortex-a9+vfpv3_gcc-7.3.0_musl_eabi/bin > addPATH.sh
This will create a script addPATH.sh that will add our binaries to the $PATH every time you will need it in the future.
You can also add this path to your $PATH permanently, but I will not cover it here.
Now run it:
source addPATH.sh echo $PATH
The result should be similar to this:
- Now this magical path will be used as a source for binaries that the compiler is looking for.
Here is the point whereI consider cross-compilation environment ready and prepared.
A. Final compilation
Here you can probably add -jx parameter to increase the number of job threads, to make the compilation process faster
make -j5 package/nginx/compile
And as result (hopefully) you will find your compiled package(s) here:
ls -al bin/mvebu-musl/packages/packages/
B. To cleanly recompile Nginx again:
source addPATH.sh (optional - needed if you opened a new terminal and $PATH is not set) make package/nginx/clean make package/nginx/prepare make -j5 package/nginx/compile
C. Steps A and B should be the same or similar for other packages you will be compiling.
Sometimes binaries will end in different directories somewhere inside bin/mvebu-musl
Hopefully this was helpful. Enjoy.