Docker on Turris Omnia

Turris Omnia
21

It is still not implemented.
I want to run daemon written in golang and it exits with madvise: function not implemented

Can you guys - CZ NIC - please compile next kernel with CONFIG_ADVISE_SYSCALLS option ?

Thank you very much for an update on this.

22

Any updates from CZ-NIC on this issue ?
Any workaround for this?

I can’t run applications in the lxc container that require madvise.

Thank you very much in advance for reply&help.

23

You would be better off emailing tech support asking for it than posting here. Although people from the team respond from time to time, writing support will ensure your query will reach the right people.

24

Here we go! https://github.com/CZ-NIC/turris-os/commit/5dff711e3a3289c04d1c02c1ab057d6b3d9864e0

I appreciate that, guys!

1 Like
25

So can we run Docker now? Do we need to compile some packages ourselves?

26

This is where I stuck at the moment:
(I am trying to run docker inside lxc-containered ubuntu)

➜  ~ dockerd
INFO[0000] libcontainerd: new containerd process, pid: 1121
WARN[0000] containerd: low RLIMIT_NOFILE changing to max  current=1024 max=4096
INFO[0001] [graphdriver] using prior storage driver: overlay2
INFO[0001] Graph migration to content-addressability took 0.00 seconds
WARN[0001] Your kernel does not support swap memory limit
WARN[0001] Your kernel does not support kernel memory limit
WARN[0001] Your kernel does not support cgroup cpu shares
WARN[0001] Your kernel does not support cgroup cfs period
WARN[0001] Your kernel does not support cgroup cfs quotas
WARN[0001] Your kernel does not support cgroup rt period
WARN[0001] Your kernel does not support cgroup rt runtime
WARN[0001] Your kernel does not support cgroup blkio weight
WARN[0001] Your kernel does not support cgroup blkio weight_device
WARN[0001] Your kernel does not support cgroup blkio throttle.read_bps_device
WARN[0001] Your kernel does not support cgroup blkio throttle.write_bps_device
WARN[0001] Your kernel does not support cgroup blkio throttle.read_iops_device
WARN[0001] Your kernel does not support cgroup blkio throttle.write_iops_device
WARN[0001] mountpoint for pids not found
INFO[0001] Loading containers: start.
WARN[0001] Running modprobe bridge br_netfilter failed with message: modprobe: ERROR: ../libkmod/libkmod.c:514 lookup_builtin_file() could not open builtin file '/lib/modules/4.4.87-cb5e816fa6b1a6b5342df69755869d71-2/modules.builtin.bin'
modprobe: WARNING: Module bridge not found in directory /lib/modules/4.4.87-cb5e816fa6b1a6b5342df69755869d71-2
modprobe: ERROR: ../libkmod/libkmod.c:514 lookup_builtin_file() could not open builtin file '/lib/modules/4.4.87-cb5e816fa6b1a6b5342df69755869d71-2/modules.builtin.bin'
modprobe: WARNING: Module br_netfilter not found in directory /lib/modules/4.4.87-cb5e816fa6b1a6b5342df69755869d71-2
, error: exit status 1
INFO[0001] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address
Error starting daemon: Error initializing network controller: Error creating default "bridge" network: Failed to program NAT chain: Failed to inject DOCKER in PREROUTING chain: iptables failed: iptables --wait -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER: iptables: No chain/target/match by that name.
 (exit status 1)
27

LXC and Docker is like škoda auto and volkswagen. Both are cars, just different. Docker is similiar to lxc but is used more for virtualization. you can automate it way better. So no point in running docker inside lxc. It is like having car inside another and going somewhere.

28

You are not correct if you think that I don’t get the difference between lxc/lxd and docker. If you have an instruction how to run docker on bare TurrisOS, you are welcome to share the knowledge

29

Okay, sorry for that.
Good luck with LXC. But I thought that module is not in 3.8 Turris but will be only tested for now.
Is it already in turris_stable?

30

What ‘module’ are you talking about? Lxc? It was there from the beginning. Docker? I am not aware of any plans for including it into TurrisOS

31

This CONFIG_ADVISE_SYSCALLS kernel module

1 Like
32

It’s not a kernel module but only a config parameter. It was changed on ‘stable’ branch as well: https://github.com/CZ-NIC/turris-os/commit/8b3d5dae3d05d590a3a114cbdd790eaf87c79423

And regarding the topic, docker can start with --iptables=0 and I’ve even could start some docker containers, but network obviously would not be working. I am still looking for a solution to configure the network for docker inside of LXC container.

33

di you use expose [number of port]?
do you know how to use it?

1 Like
34

have you tried net=host ?

btw do you have iptables installed inside the lxc?

35

Hi guys.

What is the status of docker on Turris Omnia? Does it work already? I would be also interested in it, and it would be nice to have a repo for it, so that we can install docker via opkg

36

Not an explicit answer to your specific (Docker) question, but LXC (the namespace technology underlying Docker) is fully supported!

37

Hey guys,

I manged to get docker running (more or less) on the Turris Omnia.

I used precompiled armhf binaries from the alpine linux distribution. To circumvent the networking problems described above (and a few more) you need to install following dependencies:

  • kmod-veth
  • kmod-ipt-extra
  • iptables-mod-extra

Here you can find a package compiled for the current released kernel version (4.4.91-e8cacce0ae0bf48eea19d58c2e860359-1): https://mega.nz/#!tOJgGTTB!kUsQX7oC3DKDB6FWYd1nTts4ih-nEt8grv7XSzaonuw

Limitations:

  • Ressource sharing: Since the required cgroups are not enabled in the stock kernel, the docker daemon has no way of limiting the consumption of certain resources for containers. So if you’re running resource hungry containers, they might starve others.
  • Swarm: There is (as far as I could find) no available IP_VS module and the kernel is compiled without support for it. If I remember correctly, this is required to set up the ingress network used by docker in swarm mode to route incoming packets. If you want your router to join a swarm, you will most probably need to compile it yourself (or ask me, I’ve got a working version for the current kernel but I am too lazy to look for it right now since I think that probably noone will want it)

For other geeks that want to take this further, my makefile that I used to build the package (I assume you can understand the folder structure and copy & paste the corresponding files where they belong): https://gist.github.com/DanielSel/e575e189720022eaffd61c2878a41508

@Turris Omnia Team: Would it be possible to enable (at least) the following kernel options at some point in the future?

  • CONFIG_MEMCG_SWAP_ENABLED
  • CONFIG_MEMCG_KMEM
  • CONFIG_BLK_DEV_THROTTLING
  • CONFIG_IOSCHED_CFQ
  • CONFIG_CFQ_GROUP_IOSCHED
  • CONFIG_CGROUP_PERF
  • CONFIG_CGROUP_HUGETLB
  • CONFIG_CGROUP_NET_PRIO
  • CONFIG_CFS_BANDWIDTH
  • CONFIG_FAIR_GROUP_SCHED
  • CONFIG_RT_GROUP_SCHED

Especially the cgroups would make docker more reliable. Thanks!

Have fun and enjoy the rest of your day!

11 Likes
38

@huibuh - This is great, thanks!!

-=ChaZ=-

39

You can send a PR to Turris repo enabling all these kernel modules. Take a look at this: https://github.com/CZ-NIC/turris-os/pull/60.

2 Likes
40

I know its old topic but the docker thing is still on. I was wondering if anybody got it running after some progress with TOS4.x and stuff?

5 Likes