[TOS 6.0] Unpriviledged LXC containers

AreYouLoco · September 8, 2022, 2:25pm

Hi,

I am trying to recreate the configuration from:

But it started to fail on some net device. Anybody got it working? I was wondering about docker but thats overcomplicating things and currently its not possible to use Docker on more than one device (no docker swarm option)

Thanks!
AreYouLoco?

AreYouLoco · September 8, 2022, 2:28pm

root@ap:/srv/lxc# lxc-start -F bitcoin                          
lxc-start: bitcoin: utils.c: safe_mount: 1198 Operation not permitted - Faile
d to mount "(null)" onto "/usr/lib/lxc/rootfs/sys/devices/virtual/net"
lxc-start: bitcoin: conf.c: lxc_mount_auto_mounts: 801 Operation not permitte
d - Failed to mount "(null)" on "/usr/lib/lxc/rootfs/sys/devices/virtual/net"
 with flags 46
lxc-start: bitcoin: conf.c: lxc_setup: 4094 Failed to setup first automatic m
ounts
lxc-start: bitcoin: start.c: do_start: 1291 Failed to setup container "bitcoi
n"
lxc-start: bitcoin: sync.c: sync_wait: 36 An error occurred in another proces
s (expected sequence number 3)
lxc-start: bitcoin: start.c: __lxc_start: 2053 Failed to spawn container "bit
coin"
lxc-start: bitcoin: tools/lxc_start.c: main: 308 The container failed to star
t
lxc-start: bitcoin: tools/lxc_start.c: main: 314

AreYouLoco · September 8, 2022, 3:24pm

So I realized that the problem exists because I introduced VLANs to my network setup. And it should be the same on TOS 5.x. I tried to use options mentioned in Linux Containers - LXC - Manpages - lxc.container.conf.5

But they are not recognized by lxc on Omnia. Anybody is doing some VLANs with LXC?

cocturr · September 8, 2022, 3:30pm

Your LXC errors seem to relate to mounting volumes though. How did you come to the conclusion that vlans are the cause?

AreYouLoco · September 8, 2022, 3:33pm

Because its trying to mount in /virtual/net and the only thing that changed in my setup from TOS5.x is VLANs

EDIT: ~~Now I am sure its VLANs~~ because when I change the LXC config for particular container to type vlan and vlan id it spits out different error:

@ap:/srv/lxc# lxc-start -F bitcoin
lxc-start: bitcoin: network.c: netdev_configure_server_vlan: 1102 File exists
 - Failed to create vlan interface "vlan1337-0" on "br-lan"
lxc-start: bitcoin: network.c: lxc_create_network_priv: 3419 File exists - Fa
iled to create network device
lxc-start: bitcoin: start.c: lxc_spawn: 1826 Failed to create the network
lxc-start: bitcoin: start.c: __lxc_start: 2053 Failed to spawn container "bit
coin"
lxc-start: bitcoin: tools/lxc_start.c: main: 308 The container failed to star
t
lxc-start: bitcoin: tools/lxc_start.c: main: 314 Additional information can b
e obtained by setting the --logfile and --logpriority options
root@ap:/srv/lxc#

AreYouLoco · September 8, 2022, 3:36pm

I think I will ask on openwrt forum then as the problem is not Turris specific.

AreYouLoco · September 8, 2022, 5:16pm

Seems like I need newer lxc:

https://linuxcontainers.org/lxc/news/#vlan-support-on-veth-devices

Edit: Nope I managed to set containers on VLAN using LuCi and lxc 4.0.10 and it works!

AreYouLoco · September 9, 2022, 2:34pm

I am trying to compile it for HBL branch. Will see if it works. Its still compiling…

No it didnt work. It needs ninja/meson that is available only in openwrt 22.03. I asked for a backport we will see

AreYouLoco · September 9, 2022, 6:08pm

Regardless of what I said that it would be nice to have lxc 5.0.1 the problem goes away when I comment out two lines about lxc.idmap. Then the container starts.

So it seems like the unprivileged part is broken on TOS 6.0. For today I am fed up of trying to make it work

AreYouLoco · September 10, 2022, 2:55pm

@psiegl Hi do you have any idea why it fails on TOS 6.0? You are the OP of the Unprivileged LXC container on Turris OS 5.0.0

AreYouLoco · September 24, 2022, 10:04am

github.com/lxc/lxc

Upgrading from version 4.0.9 to 4.0.10 results in utils.c: safe_mount: 1198 Operation not permitted - Failed to mount "(null)" onto "/var/lib/lxc/rootfs/sys/devices/virtual/net"

opened 06:05PM - 04 Sep 21 UTC

closed 09:13PM - 12 Feb 22 UTC

tglaeser

# Required information * Distribution: Gentoo * Distribution version: roll…ing/latest * The output of * `lxc-start --version` `4.0.10` * `lxc-checkconfig` ``` LXC version 4.0.10 --- Namespaces --- Namespaces: enabled Utsname namespace: enabled Ipc namespace: enabled Pid namespace: enabled User namespace: enabled Network namespace: enabled --- Control groups --- Cgroups: enabled Cgroup namespace: enabled Cgroup v1 mount points: /sys/fs/cgroup/openrc /sys/fs/cgroup/cpuset /sys/fs/cgroup/cpu /sys/fs/cgroup/cpuacct /sys/fs/cgroup/blkio /sys/fs/cgroup/memory /sys/fs/cgroup/devices /sys/fs/cgroup/freezer /sys/fs/cgroup/net_cls /sys/fs/cgroup/perf_event /sys/fs/cgroup/net_prio /sys/fs/cgroup/hugetlb /sys/fs/cgroup/pids /sys/fs/cgroup/rdma Cgroup v2 mount points: /sys/fs/cgroup/unified Cgroup v1 systemd controller: missing Cgroup v1 clone_children flag: enabled Cgroup device: enabled Cgroup sched: enabled Cgroup cpu account: enabled Cgroup memory controller: enabled Cgroup cpuset: enabled --- Misc --- Veth pair device: enabled, loaded Macvlan: enabled, not loaded Vlan: enabled, loaded Bridges: enabled, loaded Advanced netfilter: enabled, loaded CONFIG_NF_NAT_IPV4: missing CONFIG_NF_NAT_IPV6: missing CONFIG_IP_NF_TARGET_MASQUERADE: enabled, not loaded CONFIG_IP6_NF_TARGET_MASQUERADE: enabled, not loaded CONFIG_NETFILTER_XT_TARGET_CHECKSUM: enabled, not loaded CONFIG_NETFILTER_XT_MATCH_COMMENT: enabled, not loaded FUSE (for use with lxcfs): enabled, not loaded --- Checkpoint/Restore --- checkpoint restore: enabled CONFIG_FHANDLE: enabled CONFIG_EVENTFD: enabled CONFIG_EPOLL: enabled CONFIG_UNIX_DIAG: enabled CONFIG_INET_DIAG: enabled CONFIG_PACKET_DIAG: enabled CONFIG_NETLINK_DIAG: enabled File capabilities: Note : Before booting a new kernel, you can check its configuration usage : CONFIG=/path/to/config /usr/bin/lxc-checkconfig ``` * `uname -a` `Linux ###### 5.10.52-gentoo-x86_64 #3 SMP Sat Jul 31 17:53:47 EDT 2021 x86_64 Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz GenuineIntel GNU/Linux` * `cat /proc/self/cgroup` ``` 14:rdma:/ 13:pids:/ 12:hugetlb:/ 11:net_prio:/ 10:perf_event:/ 9:net_cls:/ 8:freezer:/ 7:devices:/ 6:memory:/ 5:blkio:/ 4:cpuacct:/ 3:cpu:/ 2:cpuset:/ 1:name=openrc:/sshd 0::/sshd ``` * `cat /proc/1/mounts` ``` proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,nosuid,size=10240k,nr_inodes=4090858,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,noexec,inode64 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,mode=755,inode64 0 0 /dev/sdb2 / btrfs rw,noatime,compress=lzo,ssd,space_cache,autodefrag,subvolid=5,subvol=/ 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 efivarfs /sys/firmware/efi/efivars efivarfs rw,nosuid,nodev,noexec,relatime 0 0 cgroup_root /sys/fs/cgroup tmpfs rw,nosuid,nodev,noexec,relatime,size=10240k,mode=755,inode64 0 0 openrc /sys/fs/cgroup/openrc cgroup rw,nosuid,nodev,noexec,relatime,release_agent=/lib/rc/sh/cgroup-release-agent.sh,name=openrc 0 0 none /sys/fs/cgroup/unified cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate 0 0 cpuset /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0 cpu /sys/fs/cgroup/cpu cgroup rw,nosuid,nodev,noexec,relatime,cpu 0 0 cpuacct /sys/fs/cgroup/cpuacct cgroup rw,nosuid,nodev,noexec,relatime,cpuacct 0 0 blkio /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0 memory /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0 devices /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0 freezer /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0 net_cls /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0 perf_event /sys/fs/cgroup/perf_event cgroup rw,nosuid,nodev,noexec,relatime,perf_event 0 0 net_prio /sys/fs/cgroup/net_prio cgroup rw,nosuid,nodev,noexec,relatime,net_prio 0 0 hugetlb /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 pids /sys/fs/cgroup/pids cgroup rw,nosuid,nodev,noexec,relatime,pids 0 0 rdma /sys/fs/cgroup/rdma cgroup rw,nosuid,nodev,noexec,relatime,rdma 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 ... ``` # Issue description ``` $ lxc-start dev --foreground lxc-start: dev: utils.c: safe_mount: 1198 Operation not permitted - Failed to mount "(null)" onto "/var/lib/lxc/rootfs/sys/devices/virtual/net" lxc-start: dev: conf.c: lxc_mount_auto_mounts: 801 Operation not permitted - Failed to mount "(null)" on "/var/lib/lxc/rootfs/sys/devices/virtual/net" with flags 46 lxc-start: dev: conf.c: lxc_setup: 4094 Failed to setup first automatic mounts lxc-start: dev: start.c: do_start: 1291 Failed to setup container "dev" lxc-start: dev: sync.c: sync_wait: 36 An error occurred in another process (expected sequence number 3) lxc-start: dev: start.c: __lxc_start: 2053 Failed to spawn container "dev" lxc-start: dev: tools/lxc_start.c: main: 308 The container failed to start lxc-start: dev: tools/lxc_start.c: main: 313 Additional information can be obtained by setting the --logfile and --logpriority options ``` # Steps to reproduce 1. Upgrading Gentoo package `app-emulation/lxc` from version `4.0.9-r1` to `4.0.10` 2. `lxc-start dev --foreground` 3. See the console output # Information to attach - [ ] any relevant kernel output (`dmesg`) - [ ] container log (The <log> file from running `lxc-start -n <c> -l TRACE -o <logfile> `) - [ ] the containers configuration file

There is a workaround to that issue but it should be fixed in lxc 4.0.12. And I was wrong its not VLAN related.

system · September 27, 2022, 10:04am

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.