Cross Compile - Howto?

cr0 · October 21, 2016, 7:08am

Hi,

I’m trying to cross compile software for my omnia, however, I’m struggling to do so. I set up the toolchain like described in https://wiki.openwrt.org/doc/devel/crosscompile. I use the openwrt git repository. The software in question is snapraid. The build machine is a 64bit ubuntu 16.04 vm.

Configuration is done using
./configure --build=x86_64-unkown-linux-gnu --host=arm-openwrt-linux

and compilation is done using
make CC=arm-openwrt-linux-gcc LD=arm-openwrt-linux-ld

However, the result is not runnable on my omnia:

root@turris:~# ./snapraid
-ash: /.snapraid: not found
root@turris:~# ldd ./snapraid 
	/lib/ld-musl-arm.so.1 (0x7f629000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb6f3f000)
	libc.so => /lib/ld-musl-arm.so.1 (0x7f629000)

I suppose that’s because I’m doing something wrong during compilation since omnia does not provide a /lib/ld-musl.arm.so.1 but a ld-musl-armhf.so.1.
How do I adjust compilation to use the later lib? Is there any howto how to compile software for my omnia?

Thanks for help.

achim71 · October 21, 2016, 9:50am

You can use the OpenwrtSDK from here https://api.turris.cz/openwrt-repo/omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64.tar.bz2. And follow these docs https://wiki.openwrt.org/doc/howto/obtain.firmware.sdk .

maurer · October 23, 2016, 5:04am

looks like zlib is broken - so a show stopper for everything

[details=hidden log]. /data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/include/shell.sh; gzip -dc /data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/dl/zlib-1.2.8.tar.gz | tar -C /data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/zlib-1.2.8/… -xf -
touch /data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/zlib-1.2.8/.prepared_6288865758a48b2549140fe9a3771fff
rm -f /data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/stamp/.zlib_installed
Makefile:123: recipe for target ‘/data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/zlib-1.2.8/.configured_yyyyn’ failed
make[2]: [/data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/zlib-1.2.8/.configured_yyyyn] Error 123 (ignored)
(cd /data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/zlib-1.2.8; AR=“arm-openwrt-linux-muslgnueabi-gcc-ar” AS=“ccache_cc -c -O2 -pipe -march=armv7-a -mtune=cortex-a9 -mfpu=vfpv3-d16 -fno-caller-saves -fhonour-copts -Wno-error=unused-but-set-variable -mfloat-abi=hard -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-z,now -Wl,-z,relro” LD=arm-openwrt-linux-muslgnueabi-ld NM=“arm-openwrt-linux-muslgnueabi-gcc-nm” CC=“ccache_cc” GCC=“ccache_cc” CXX=“ccache_cxx” RANLIB=“arm-openwrt-linux-muslgnueabi-gcc-ranlib” STRIP=arm-openwrt-linux-muslgnueabi-strip OBJCOPY=arm-openwrt-linux-muslgnueabi-objcopy OBJDUMP=arm-openwrt-linux-muslgnueabi-objdump SIZE=arm-openwrt-linux-muslgnueabi-size LDSHARED=“ccache_cc -shared -Wl,-soname,libz.so.1” CFLAGS="-O2 -pipe -march=armv7-a -mtune=cortex-a9 -mfpu=vfpv3-d16 -fno-caller-saves -fhonour-copts -Wno-error=unused-but-set-variable -mfloat-abi=hard -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-z,now -Wl,-z,relro -fpic" ./configure --prefix=/usr --shared --uname=Linux );
Compiler error reporting is too harsh for ./configure (perhaps remove -Werror).
** ./configure aborting.
Makefile:123: recipe for target ‘/data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/zlib-1.2.8/.configured_yyyyn’ failed
make[2]: *** [/data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/zlib-1.2.8/.configured_yyyyn] Error 1
make[2]: Leaving directory '/data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/feeds/base/package/libs/zlib’
package/Makefile:193: recipe for target ‘package/feeds/base/zlib/compile’ failed
make[1]: *** [package/feeds/base/zlib/compile] Error 2
make[1]: Leaving directory ‘/data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64’
/data/tmp/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/include/toplevel.mk:174: recipe for target ‘package/tmux/compile’ failed
make: *** [package/tmux/compile] Error 2
[/details]

Can someone help?

achim71 · October 23, 2016, 8:42am

The SDK requires ccache. Error messages are more obvious with other packages.

cr0 · October 24, 2016, 6:16pm

Great, I was able to compile the app successfully

Follow up: I’m trying to compile an app which uses libfuse. Seems like the toolchain does not provide this library?

$~/turris-omnia/mergerfs-2.16.1$ make CXX=arm-openwrt-linux-g++ LD=arm-openwrt-linux-ld
arm-openwrt-linux-g++ -g -Wall -O2 -Wno-unused-result -D_FILE_OFFSET_BITS=64 -I/usr/include/fuse   -DFUSE_USE_VERSION=29 -MMD -DFLAG_NOPATH=1 -DFALLOCATE=1 -DFLOCK=1 -DREAD_BUF=1 -DWRITE_BUF=1 -DUGID_USE_RWLOCK=0 obj/access.o obj/category.o obj/chmod.o obj/chown.o obj/clone.o obj/config.o obj/create.o obj/destroy.o obj/fallocate.o obj/fgetattr.o obj/flush.o obj/fs_attr.o obj/fs_clonefile.o obj/fs_clonepath.o obj/fs.o obj/fs_fadvise.o obj/fs_fallocate.o obj/fs_movefile.o obj/fs_path.o obj/fs_sendfile.o obj/fs_time.o obj/fs_xattr.o obj/fsync.o obj/ftruncate.o obj/fusefunc.o obj/getattr.o obj/getxattr.o obj/gidcache.o obj/init.o obj/ioctl.o obj/link.o obj/listxattr.o obj/mergerfs.o obj/mkdir.o obj/mknod.o obj/num.o obj/open.o obj/opendir.o obj/option_parser.o obj/policy_all.o obj/policy.o obj/policy_epall.o obj/policy_epff.o obj/policy_eplfs.o obj/policy_eplus.o obj/policy_epmfs.o obj/policy_eprand.o obj/policy_erofs.o obj/policy_ff.o obj/policy_invalid.o obj/policy_lfs.o obj/policy_lus.o obj/policy_mfs.o obj/policy_newest.o obj/policy_rand.o obj/read_buf.o obj/read.o obj/readdir.o obj/readlink.o obj/release.o obj/releasedir.o obj/removexattr.o obj/rename.o obj/resources.o obj/rmdir.o obj/setxattr.o obj/statfs.o obj/str.o obj/symlink.o obj/truncate.o obj/ugid.o obj/unlink.o obj/utimens.o obj/write_buf.o obj/write.o -o mergerfs -pthread -lfuse
/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/toolchain-arm_cortex-a9+vfpv3_gcc-4.8-linaro_musl-1.1.11_eabi/bin/../lib/gcc/arm-openwrt-linux-muslgnueabi/4.8.3/../../../../arm-openwrt-linux-muslgnueabi/bin/ld: cannot find -lfuse
collect2: error: ld returned 1 exit status
make: *** [mergerfs] Error 1

How can I fix this?

achim71 · October 24, 2016, 6:23pm

Libfuse is part of the fuse package

Is this not part of the SDK?

cr0 · October 24, 2016, 6:30pm

Can you tell how to verify if it’s part of the SDK? I cannot find a fuse.h file within the toolchain’s include dir.

achim71 · October 24, 2016, 6:41pm

I do not use the SDK here. Take a look at the ntfs-3g package. It uses libfuse. https://github.com/CZ-NIC/turris-os-packages/blob/test/utils/ntfs-3g/Makefile.
With the full build environment i’d use

make package/utils/fuse/compile
make package/utils/fuse/install

This builds and installs fuse headers and libraries into staging_dir/target.

cr0 · October 24, 2016, 7:20pm

Is the full build env the one which I can get from https://github.com/CZ-NIC/turris-os?
I cloned that one and followed the instructions (./scripts/feeds update -a & ./scripts/feeds install -a).

Then I ran make package/utils/fuse/compile but it fails with:

$ make package/utils/fuse/compile
Checking 'working-make'... ok.
Checking 'case-sensitive-fs'... ok.
Checking 'gcc'... ok.
Checking 'working-gcc'... ok.
Checking 'g++'... ok.
Checking 'working-g++'... ok.
Checking 'ncurses'... ok.
Checking 'zlib'... ok.
Checking 'libssl'... ok.
Checking 'tar'... ok.
Checking 'find'... ok.
Checking 'bash'... ok.
Checking 'patch'... ok.
Checking 'diff'... ok.
Checking 'cp'... ok.
Checking 'seq'... ok.
Checking 'awk'... ok.
Checking 'grep'... ok.
Checking 'getopt'... ok.
Checking 'stat'... ok.
Checking 'md5sum'... ok.
Checking 'unzip'... ok.
Checking 'bzip2'... ok.
Checking 'wget'... ok.
Checking 'perl'... ok.
Checking 'python'... ok.
Checking 'svn'... ok.
Checking 'git'... ok.
Checking 'file'... ok.
Checking 'openssl'... ok.
Checking 'ldconfig-stub'... ok.
Collecting package info: doneing...eds/turrispackages/xzhaperg_adjustsynces-gadgets
Collecting target info: doneing...ux/xburstew
make[2]: Entering directory `/home/chr/turris-omnia/turris-os/scripts/config'
conf.c: In function 'check_stdin':
conf.c:77:3: warning: format not a string literal and no format arguments [-Wformat-security]
   printf(_("aborted!\n\n"));
   ^
conf.c:78:3: warning: format not a string literal and no format arguments [-Wformat-security]
   printf(_("Console input/output is redirected. "));
   ^
conf.c:79:3: warning: format not a string literal and no format arguments [-Wformat-security]
   printf(_("Run 'make oldconfig' to update configuration.\n\n"));
   ^
conf.c: In function 'conf_askvalue':
conf.c:89:3: warning: format not a string literal and no format arguments [-Wformat-security]
   printf(_("(NEW) "));
   ^
conf.c: In function 'conf_choice':
conf.c:290:5: warning: format not a string literal and no format arguments [-Wformat-security]
     printf(_(" (NEW)"));
     ^
conf.c: In function 'check_conf':
conf.c:438:6: warning: format not a string literal and no format arguments [-Wformat-security]
      printf(_("*\n* Restart config...\n*\n"));
      ^
conf.c: In function 'main':
conf.c:615:6: warning: format not a string literal and no format arguments [-Wformat-security]
      _("\n*** The configuration requires explicit update.\n\n"));
      ^
conf.c:669:4: warning: format not a string literal and no format arguments [-Wformat-security]
    fprintf(stderr, _("\n*** Error during writing of the configuration.\n\n"));
    ^
conf.c:673:4: warning: format not a string literal and no format arguments [-Wformat-security]
    fprintf(stderr, _("\n*** Error during update of the configuration.\n\n"));
    ^
conf.c:684:4: warning: format not a string literal and no format arguments [-Wformat-security]
    fprintf(stderr, _("\n*** Error during writing of the configuration.\n\n"));
    ^
make[2]: Leaving directory `/home/chr/turris-omnia/turris-os/scripts/config'
make[2]: Entering directory `/home/chr/turris-omnia/turris-os'
make[3]: Entering directory `/home/chr/turris-omnia/turris-os/scripts/config'
make[3]: Leaving directory `/home/chr/turris-omnia/turris-os/scripts/config'
tmp/.config-package.in:51955:error: recursive dependency detected!
tmp/.config-package.in:51955:   symbol PACKAGE_avahi-dbus-daemon is selected by PACKAGE_libavahi-client
tmp/.config-package.in:26142:   symbol PACKAGE_libavahi-client is selected by PACKAGE_libcups
tmp/.config-package.in:24250:   symbol PACKAGE_libcups is selected by PACKAGE_libcupscgi
tmp/.config-package.in:24269:   symbol PACKAGE_libcupscgi is selected by PACKAGE_cups
tmp/.config-package.in:54524:   symbol PACKAGE_cups is selected by PACKAGE_splix
tmp/.config-package.in:72949:   symbol PACKAGE_splix depends on PACKAGE_libavahi-dbus-support
tmp/.config-package.in:26195:   symbol PACKAGE_libavahi-dbus-support is selected by PACKAGE_avahi-dbus-daemon


*** End of the configuration.
*** Execute 'make' to start the build or try 'make help'.

make[2]: Leaving directory `/home/chr/turris-omnia/turris-os'
 make[1] package/utils/fuse/compile
 make[2] -C package/libs/toolchain compile
make -r package/utils/fuse/compile: build failed. Please re-run make with -j1 V=s to see what's going on
make: *** [package/utils/fuse/compile] Error 1

Thank you very much for your help.

achim71 · October 24, 2016, 7:35pm

Are you writing an pacakge for your app? Your Makefile should contain an line like

DEPENDS+= +libfuse

With this the required fuse package should be build when running make in the sdk. No need for the full build environment in your case.

Update: You can find the required includes in the sdk in staging_dir/target/usr/include not staging_dir/toolchain.

cr0 · October 25, 2016, 6:55pm

Thank you for your answer.

So you’re saying that I have to create a package for the application using this instructions: https://wiki.openwrt.org/doc/devel/packages?

Maybe you want to have a look at the current Makefile for mergerfs: https://github.com/trapexit/mergerfs/blob/master/Makefile Adding a DEPENDS+= +libfuse still does not fix the Problem. I also tried to manually add -I$staging_dir/target/usr/include/fuse to include the fuse package, but this did not work, too. It seems that the Makefile uses pkg-config a lot to extract compiler Arguments.

achim71 · October 25, 2016, 7:53pm

You must create an foldr for your package eg mergerfs and and Makefile for the package like it is described in the openwrt docs. This Makefile needs the DEPENDS+=+libfuse line. This Makefile is your package configuration file.
From within this Makefile’s Build/Compile function the Makefile from mergerfs (the one you linked) is invoked.
Looking at the fuse Packages Makefile it seems pkg-config files is taken care of. https://github.com/CZ-NIC/turris-os/blob/test/package/utils/fuse/Makefile

cr0 · October 25, 2016, 8:59pm

Thank you a lot for explaining.

I now created a Makefile for mergerfs which I marked as M in make menuconfig. However, when I try to compile it using make package/mergerfs/compile V=99 it crashes because package/Feeds/base/Linux/compile throws an error:

make[2]: Entering directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/feeds/base/package/kernel/linux'
mkdir -p /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/bin/mvebu-musl/packages /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/linux-mvebu/packages/ipkg-mvebu/kernel/CONTROL /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo
install -d -m0755 /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/linux-mvebu/packages/ipkg-mvebu/kernel/boot
cp -fpR /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/linux-mvebu/linux-4.4.13/arch/arm/boot/zImage /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/linux-mvebu/packages/ipkg-mvebu/kernel/boot/zImage-4.4.13-1-05df79f63527051ea0071350f86faf76-7
cp: cannot stat '/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/linux-mvebu/linux-4.4.13/arch/arm/boot/zImage': No such file or directory
make[2]: *** [/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/bin/mvebu-musl/packages/base/kernel_4.4.13-1-05df79f63527051ea0071350f86faf76-7_mvebu.ipk] Error 1
make[2]: Leaving directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/feeds/base/package/kernel/linux'
make[1]: *** [package/feeds/base/linux/compile] Error 2
make[1]: Leaving directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64'
make: *** [package/mergerfs/compile] Fehler 2
chr@DESKTOP-DVL8FLQ:~/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64$

Looks like zImage cannot be found (full log)

You can find the Makefile here: https://gist.github.com/cr0/b0fb149b2a0adcb3846a74f16185fc0e

achim71 · October 26, 2016, 9:46am

Create an folder package/mergerfs and put the Makefile in that folder.
Try below Makefile, it still has bugs but it builds the maregerfs binary here. Usually I only tweak broken packages.
Use

make package/mergerfs/compile V=s

in the SDK’s root folder

# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
# $Id$
include $(TOPDIR)/rules.mk

PKG_NAME:=mergerfs
PKG_VERSION:=2.16.1
PKG_RELEASE:=1
PKG_SOURCE:=$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=https://github.com/trapexit/mergerfs/archive/

PKG_MD5SUM:=1ae6e3ca40d23e70fb2719135ba7c961
PKG_CAT:=zcat

include $(INCLUDE_DIR)/package.mk

define Package/mergerfs
  SECTION:=utils
  CATEGORY:=Filesystem
  TITLE:=A FUSE based union filesystem
  URL:=https://github.com/trapexit/mergerfs
  DEPENDS+= +libfuse +libpthread
endef

define Package/mergerfs/description
  mergerfs is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. It is similar to mhddfs, unionfs, and aufs.
endef

define Build/Configure
endef

define Build/Compile
        $(MAKE) -C $(PKG_BUILD_DIR) \
        CC="$(TARGET_CC)" \
        CXX="$(TARGET_CXX)" \
        OPTS="-O2 $(TARGET_CFLAGS) -I$(STAGING_DIR)/usr/include" \
        LDFLAGS="${TARGET_LDFLAGS} -lfuse -lpthread" \
        XATTR_AVAILABLE=0
endef

define Package/mergerfs/install
        $(INSTALL_DIR) $(1)/bin
        $(CP) $(PKG_BUILD_DIR)/mergerfs $(1)/bin/mount.mergerfs
endef

$(eval $(call BuildPackage,mergerfs))

cr0 · October 26, 2016, 7:55pm

Again, thank you very much for your help, however, the build still fails with your Makefile. Make is still unable to find zImage.

#
# configuration written to .config
#
make[1]: Entering directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64'
make[2]: Entering directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/feeds/base/package/libs/toolchain'
if [ -f /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install.clean ]; then rm -f /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install.clean; fi; echo "libc" >> /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install
if [ -f /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install.clean ]; then rm -f /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install.clean; fi; echo "libgcc" >> /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install
if [ -f /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install.clean ]; then rm -f /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install.clean; fi; echo "libpthread" >> /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install
if [ -f /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install.clean ]; then rm -f /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install.clean; fi; echo "librt" >> /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo/toolchain.default.install
make[2]: Leaving directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/feeds/base/package/libs/toolchain'
make[2]: Entering directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/feeds/base/package/firmware/linux-firmware'
make[2]: Leaving directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/feeds/base/package/firmware/linux-firmware'
make[2]: Entering directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/feeds/base/package/kernel/linux'
mkdir -p /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/bin/mvebu-musl/packages /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/linux-mvebu/packages/ipkg-mvebu/kernel/CONTROL /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/pkginfo
install -d -m0755 /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/linux-mvebu/packages/ipkg-mvebu/kernel/boot
cp -fpR /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/linux-mvebu/linux-4.4.13/arch/arm/boot/zImage /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/linux-mvebu/packages/ipkg-mvebu/kernel/boot/zImage-4.4.13-1-05df79f63527051ea0071350f86faf76-7
cp: cannot stat '/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/build_dir/target-arm_cortex-a9+vfpv3_musl-1.1.11_eabi/linux-mvebu/linux-4.4.13/arch/arm/boot/zImage': No such file or directory
make[2]: *** [/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/bin/mvebu-musl/packages/base/kernel_4.4.13-1-05df79f63527051ea0071350f86faf76-7_mvebu.ipk] Error 1
make[2]: Leaving directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/feeds/base/package/kernel/linux'
make[1]: *** [package/feeds/base/linux/compile] Error 2
make[1]: Leaving directory `/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64'
make: *** [package/mergerfs/compile] Error 2

What I did:

I downloaded https://api.turris.cz/openwrt-repo/omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64.tar.bz2, added /usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/bin:/home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/toolchain-arm_cortex-a9+vfpv3_gcc-4.8-linaro_musl-1.1.11_eabi/bin to $PATH and set $STAGING_DIR to /home/chr/turris-omnia/OpenWrt-SDK-mvebu_gcc-4.8-linaro_musl-1.1.11_eabi.Linux-x86_64/staging_dir/toolchain-arm_cortex-a9+vfpv3_gcc-4.8-linaro_musl-1.1.11_eabi/
./scripts/feeds update -a && ./scripts/feeds install -a
Create the package like you explained
make menuconfig
ln -s ../feeds/base/package/utils package/utils because of feeds/base/package/utils/busybox/config/Config.in:818: glob failed: No files found "package/utils/busybox/config/libbb/Config.in" when execing make menuconfig
make menuconfig
make package/mergerfs/compile V=s

achim71 · October 26, 2016, 8:05pm

I extracted the SDK, created the mergerfs folder, added the Makefile and ran
make package/mergerfs/compile V=s
It compiles the mergerfs binary but afterwards fails with an error about an missing libstdc++ dependency, guess it is an library configuration thing.

cr0 · October 26, 2016, 8:28pm

Great, now I got a working binary, but I also get the libstdc++ error, but who cares
Thank you again for your great support.

what i did:

removed the openwrt stuff from $PATH and $STAGING_DIR
did not run make menuconfig and the feed scripts
switched from bash on ubuntu on wndows to a ubuntu vm 16.04

achim71 · October 26, 2016, 8:55pm

This Makefile builds an package with the libstdcpp dependency. It installs the binary as /sbin/mount.mergerfs. Looking at the docs having xattr support would be nice.

# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
# $Id$
include $(TOPDIR)/rules.mk

PKG_NAME:=mergerfs
PKG_VERSION:=2.16.1
PKG_RELEASE:=1
PKG_SOURCE:=$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=https://github.com/trapexit/mergerfs/archive/

PKG_MD5SUM:=1ae6e3ca40d23e70fb2719135ba7c961

PKG_BUILD_PARALLEL:=1
PKG_INSTALL:=1

include $(INCLUDE_DIR)/package.mk

define Package/mergerfs
  SECTION:=utils
  CATEGORY:=Filesystem
  TITLE:=A FUSE based union filesystem
  URL:=https://github.com/trapexit/mergerfs
  DEPENDS+= +libfuse +libpthread +libstdcpp
endef

define Package/mergerfs/description
  mergerfs is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. It is similar to mhddfs, unionfs, and aufs.
endef

define Build/Configure
endef

define Build/Compile
        $(MAKE) -C $(PKG_BUILD_DIR) \
        CC="$(TARGET_CC)" \
        CXX="$(TARGET_CXX)" \
        OPTS="-O2 $(TARGET_CFLAGS) -I$(STAGING_DIR)/usr/include" \
        LDFLAGS="${TARGET_LDFLAGS} -lfuse -lpthread" \
        XATTR_AVAILABLE=0
endef

define Package/mergerfs/install
        $(INSTALL_DIR) $(1)/sbin
        $(CP) $(PKG_BUILD_DIR)/mergerfs $(1)/sbin/mount.mergerfs
endef

$(eval $(call BuildPackage,mergerfs))

achim71 · October 26, 2016, 9:09pm

Slightly modified Makefile using uclibc++ instead of libstdc++.

# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
# $Id$
include $(TOPDIR)/rules.mk

PKG_NAME:=mergerfs
PKG_VERSION:=2.16.1
PKG_RELEASE:=1
PKG_SOURCE:=$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=https://github.com/trapexit/mergerfs/archive/

PKG_MD5SUM:=1ae6e3ca40d23e70fb2719135ba7c961

PKG_BUILD_PARALLEL:=1
PKG_INSTALL:=1

include $(INCLUDE_DIR)/package.mk

define Package/mergerfs
  SECTION:=utils
  CATEGORY:=Filesystem
  TITLE:=A FUSE based union filesystem
  URL:=https://github.com/trapexit/mergerfs
  DEPENDS+= +libfuse +libpthread +uclibcxx
endef

define Package/mergerfs/description
  mergerfs is a union filesystem geared towards simplifying storage and management of files across numerous commodity storage devices. It is similar to mhddfs, unionfs, and aufs.
endef

define Build/Configure
endef

define Build/Compile
        $(MAKE) -C $(PKG_BUILD_DIR) \
        CC="$(TARGET_CC)" \
        CXX="g++-uc+std" \
        OPTS="-O2 $(TARGET_CFLAGS) -I$(STAGING_DIR)/usr/include" \
        LDFLAGS="${TARGET_LDFLAGS} -lfuse -lpthread" \
        XATTR_AVAILABLE=0
endef

define Package/mergerfs/install
        $(INSTALL_DIR) $(1)/sbin
        $(CP) $(PKG_BUILD_DIR)/mergerfs $(1)/sbin/mount.mergerfs
endef

$(eval $(call BuildPackage,mergerfs))

maurer · October 31, 2016, 12:57pm

Hi ,
i’ve getting errors when running ‘make menuconfig’

debian@multi-cpu:~/turris-os-sdk$ make menuconfig
Collecting package info: done
tmp/.config-package.in:8:warning: ignoring type redefinition of 'PACKAGE_base-files' from 'boolean' to 'tristate'
tmp/.config-package.in:25:warning: ignoring type redefinition of 'PACKAGE_block-mount' from 'boolean' to 'tristate'
tmp/.config-package.in:62:warning: ignoring type redefinition of 'PACKAGE_busybox' from 'boolean' to 'tristate'
feeds/base/package/utils/busybox/config/Config.in:818: glob failed: No files found "package/utils/busybox/config/libbb/Config.in"
/home/debian/turris-os-sdk/include/toplevel.mk:116: recipe for target 'menuconfig' failed
make: *** [menuconfig] Error 1