Turris OS 4.0 beta2 is out!

Dear Turris users,

We would like to inform you that we released Turris OS 4.0 - beta2. It is released for Turris MOX and Turris Omnia.

Changelog for this release:

  • New implementation of dev-detect which does not depend on Pakon (experimental)
  • Fixed default encryption method for passwd from package shadow. Reset your
    system user’s passwords (including root) if you set them by passwd.
  • LXC fixes for systemd based hosts
  • Foris: packages lists UI reworked
  • Foris: improved “no link” message in WAN tab
  • Netmetr: fixed initial setup
  • Nextcloud: dropped duplicate Referrer-Policy and updated to 16.0.1
  • Commit hash replaced with router name in banner
  • Suricata updated to version 4.0.7
  • kmod-usb2 is now part of base installation
  • libxslt: CVE-2019-11068
  • prosody: CVE-2018-10847
  • python-urllib3: CVE-2019-9740, CVE-2019-11324

When you were using our previous Beta1 version, you should be updated to this version automatically within a few hours.

If someone wants to give it try on Turris Omnia router, which is running Turris OS 3.11.4 or any other operating system, then we have prepared some short notes about how to do it.

From scratch, you just need to plug USB flash drive and put there rootfs, which you can download it here and by using the re-flash method, which is described in our documentation. If you want to have a backup, we suggest you create a snapshot using our tool - schnapps.

We appreciate any feedback for this release.

We hope that you will enjoy it!
Turris Team

2 Likes

Known bugs:

Turris Omnia specific

  • Second CPU ethernet port to switch chip is disabled, only one of two ethernet ports between CPU and switch is in use.
  • Old version of libmariadb when using Nextcloud from Turris OS 3.11.x.

Turris 1.x specific

  • Currently not working because of kernel issues. Please do not test this release on Turris 1.x

Update from 4.0 beta 1 went smoothly without any problem.

However I still do have the issue with nextcloud, reported in gitlab #398

Updated to 4B2. Except from the usual mess up with Foris (takes a reboot to fix), so far so good.

Capture
Version 100.0; really!!!

Thanks for the new Version! :slight_smile:
Update installed without Problems.

Buuuuttt: I have problems to start “old” lxc-containers from Turris OS 3:
lxc-start gives me the output in the logfile:
start - start.c:start:2028 - Exec format error - Failed to exec “/sbin/init”

a second container:
start - start.c:start:2028 - Permission denied - Failed to exec “/sbin/init”

(the first-container i make a permission-change to 777 before, to test the Things…)

Could you help me? (which informations are needed? should i open a new thread?)

You could set up first a new guest container to test/see if lxc works in general, which would be expected, or fails too.

For the old container specifics run from cli lxc-start containername -F -l debug, or if you want to save the console output lxc-start containername -F -l debug -o /path/to/log/file


depends on the debug output. if it is not a TOS issue but rather lxc there are https://discuss.linuxcontainers.org and Linux Containers - LXC - Manpages

Hi,

maybe another (known) limitation … :wink:
I’ve installed the fresh beta2 and noticed that you’ve updated lxc to 3.x which is cool. Unfortunately there is no longer the lxc-related ubus interface available - which makes the luci interface pretty useless. Is it planned to re-add ubus support in a later beta?

Thanks for the progress!

Edit: lxc ubus interface options in OpenWrt:

root@blackhole1:~# ubus -v list lxc
'lxc' @d680b770
	"start":{"name":"String","config":"String"}
	"reboot":{"name":"String","config":"String"}
	"shutdown":{"name":"String","config":"String","timeout":"Integer"}
	"stop":{"name":"String","config":"String"}
	"freeze":{"name":"String","config":"String"}
	"unfreeze":{"name":"String","config":"String"}
	"info":{"name":"String","config":"String"}
	"rename":{"name":"String","config":"String","newname":"String"}
	"create":{"name":"String","config":"String","template":"String","flags":"Integer","args":"Array"}
	"destroy":{"name":"String","config":"String"}
	"list":{}
Is all there in this beta as well and luci-app-lxc is working as expected, least on my node

1 Like

@anon50890781 thanks!

@Pepe Sorry for the noise - was too early for testing - a manual rpcd restart did the trick!

Thanks for your answer.
LXC works in General, i can start an container which is created by OS4b1 - 2nd Test: OS4b2 created container works too.

Now the old OS3 container:

lxc-start Asterisk 20190608100338.274 INFO     lsm - lsm/lsm.c:lsm_init:50 - LSM security driver nop
lxc-start Asterisk 20190608100338.274 DEBUG    terminal - terminal.c:lxc_terminal_peer_default:714 - Using terminal "/dev/tty" as proxy
lxc-start Asterisk 20190608100338.274 DEBUG    terminal - terminal.c:lxc_terminal_signal_init:192 - Created signal fd 9
lxc-start Asterisk 20190608100338.274 DEBUG    terminal - terminal.c:lxc_terminal_winsz:90 - Set window size to 170 columns and 33 rows
lxc-start Asterisk 20190608100338.274 INFO     start - start.c:lxc_init:897 - Container "Asterisk" is initialized
lxc-start Asterisk 20190608100338.277 INFO     network - network.c:instantiate_veth:147 - Retrieved mtu 1500 from br-lan
lxc-start Asterisk 20190608100338.295 INFO     network - network.c:instantiate_veth:175 - Attached "vethJDGPGE" to bridge "br-lan"
lxc-start Asterisk 20190608100338.302 DEBUG    network - network.c:instantiate_veth:201 - Instantiated veth "vethJDGPGE/vethN41BHQ", index is "40"
lxc-start Asterisk 20190608100338.302 DEBUG    cgfsng - cgroups/cgfsng.c:cg_legacy_handle_cpuset_hierarchy:620 - "cgroup.clone_children" was already set to "1"
lxc-start Asterisk 20190608100338.306 INFO     start - start.c:lxc_spawn:1688 - Cloned CLONE_NEWNS
lxc-start Asterisk 20190608100338.306 INFO     start - start.c:lxc_spawn:1688 - Cloned CLONE_NEWPID
lxc-start Asterisk 20190608100338.306 INFO     start - start.c:lxc_spawn:1688 - Cloned CLONE_NEWUTS
lxc-start Asterisk 20190608100338.306 INFO     start - start.c:lxc_spawn:1688 - Cloned CLONE_NEWIPC
lxc-start Asterisk 20190608100338.306 INFO     start - start.c:lxc_spawn:1688 - Cloned CLONE_NEWNET
lxc-start Asterisk 20190608100338.306 DEBUG    start - start.c:lxc_try_preserve_namespaces:196 - Preserved mnt namespace via fd 15
lxc-start Asterisk 20190608100338.306 DEBUG    start - start.c:lxc_try_preserve_namespaces:196 - Preserved pid namespace via fd 16
lxc-start Asterisk 20190608100338.306 DEBUG    start - start.c:lxc_try_preserve_namespaces:196 - Preserved uts namespace via fd 17
lxc-start Asterisk 20190608100338.306 DEBUG    start - start.c:lxc_try_preserve_namespaces:196 - Preserved ipc namespace via fd 18
lxc-start Asterisk 20190608100338.307 DEBUG    start - start.c:lxc_try_preserve_namespaces:196 - Preserved net namespace via fd 19
lxc-start Asterisk 20190608100338.308 DEBUG    start - start.c:lxc_spawn:1742 - Preserved net namespace via fd 10
lxc-start Asterisk 20190608100338.351 DEBUG    network - network.c:lxc_network_move_created_netdev_priv:2500 - Moved network device "vethN41BHQ"/"eth0" to network namespace of 19805
lxc-start Asterisk 20190608100338.351 INFO     start - start.c:do_start:1242 - Unshared CLONE_NEWCGROUP
lxc-start Asterisk 20190608100338.352 DEBUG    storage - storage/storage.c:get_storage_by_name:231 - Detected rootfs type "btrfs"
lxc-start Asterisk 20190608100338.352 DEBUG    conf - conf.c:lxc_mount_rootfs:1332 - Mounted rootfs "/srv/lxc/Asterisk/rootfs" onto "/usr/lib/lxc/rootfs" with options "(null)"
lxc-start Asterisk 20190608100338.352 INFO     conf - conf.c:setup_utsname:791 - Set hostname to "Asterisk"
lxc-start Asterisk 20190608100338.391 DEBUG    network - network.c:setup_hw_addr:2767 - Mac address "52:69:41:ea:74:a8" on "eth0" has been setup
lxc-start Asterisk 20190608100338.397 DEBUG    network - network.c:lxc_setup_netdev_in_child_namespaces:3032 - Network device "eth0" has been setup
lxc-start Asterisk 20190608100338.397 INFO     network - network.c:lxc_setup_network_in_child_namespaces:3053 - network has been setup
lxc-start Asterisk 20190608100338.397 INFO     conf - conf.c:mount_autodev:1118 - Preparing "/dev"
lxc-start Asterisk 20190608100338.397 INFO     conf - conf.c:mount_autodev:1165 - Prepared "/dev"
lxc-start Asterisk 20190608100338.397 INFO     conf - conf.c:lxc_fill_autodev:1209 - Populating "/dev"
lxc-start Asterisk 20190608100338.397 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/full"
lxc-start Asterisk 20190608100338.397 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/null"
lxc-start Asterisk 20190608100338.398 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/random"
lxc-start Asterisk 20190608100338.398 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/tty"
lxc-start Asterisk 20190608100338.398 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/urandom"
lxc-start Asterisk 20190608100338.398 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/zero"
lxc-start Asterisk 20190608100338.398 INFO     conf - conf.c:lxc_fill_autodev:1286 - Populated "/dev"
lxc-start Asterisk 20190608100338.398 DEBUG    conf - conf.c:lxc_setup_dev_console:1771 - Mounted pts device "/dev/pts/1" onto "/usr/lib/lxc/rootfs/dev/console"
lxc-start Asterisk 20190608100338.398 INFO     utils - utils.c:lxc_mount_proc_if_needed:1253 - Mounted /proc in container for security transition
lxc-start Asterisk 20190608100338.440 WARN     conf - conf.c:lxc_setup_devpts:1616 - Invalid argument - Failed to unmount old devpts instance
lxc-start Asterisk 20190608100338.441 DEBUG    conf - conf.c:lxc_setup_devpts:1653 - Mount new devpts instance with options "gid=5,newinstance,ptmxmode=0666,mode=0620,max=1024"
lxc-start Asterisk 20190608100338.441 DEBUG    conf - conf.c:lxc_setup_devpts:1672 - Created dummy "/dev/ptmx" file as bind mount target
lxc-start Asterisk 20190608100338.441 DEBUG    conf - conf.c:lxc_setup_devpts:1677 - Bind mounted "/dev/pts/ptmx" to "/dev/ptmx"
lxc-start Asterisk 20190608100338.441 DEBUG    conf - conf.c:lxc_allocate_ttys:989 - Created tty "/dev/pts/0" with master fd 11 and slave fd 14
lxc-start Asterisk 20190608100338.441 DEBUG    conf - conf.c:lxc_allocate_ttys:989 - Created tty "/dev/pts/1" with master fd 15 and slave fd 16
lxc-start Asterisk 20190608100338.441 DEBUG    conf - conf.c:lxc_allocate_ttys:989 - Created tty "/dev/pts/2" with master fd 17 and slave fd 18
lxc-start Asterisk 20190608100338.441 DEBUG    conf - conf.c:lxc_allocate_ttys:989 - Created tty "/dev/pts/3" with master fd 19 and slave fd 20
lxc-start Asterisk 20190608100338.442 INFO     conf - conf.c:lxc_allocate_ttys:1005 - Finished creating 4 tty devices
lxc-start Asterisk 20190608100338.442 DEBUG    conf - conf.c:lxc_setup_ttys:940 - Bind mounted "/dev/pts/0" onto "/dev/tty1"
lxc-start Asterisk 20190608100338.442 DEBUG    conf - conf.c:lxc_setup_ttys:940 - Bind mounted "/dev/pts/1" onto "/dev/tty2"
lxc-start Asterisk 20190608100338.442 DEBUG    conf - conf.c:lxc_setup_ttys:940 - Bind mounted "/dev/pts/2" onto "/dev/tty3"
lxc-start Asterisk 20190608100338.442 DEBUG    conf - conf.c:lxc_setup_ttys:940 - Bind mounted "/dev/pts/3" onto "/dev/tty4"
lxc-start Asterisk 20190608100338.442 INFO     conf - conf.c:lxc_setup_ttys:949 - Finished setting up 4 /dev/tty<N> device(s)
lxc-start Asterisk 20190608100338.442 INFO     conf - conf.c:setup_personality:1716 - Set personality to "0x8"
lxc-start Asterisk 20190608100338.442 DEBUG    conf - conf.c:setup_caps:2506 - Capabilities have been setup
lxc-start Asterisk 20190608100338.442 NOTICE   conf - conf.c:lxc_setup:3692 - The container "Asterisk" is set up
lxc-start Asterisk 20190608100338.470 DEBUG    start - start.c:lxc_spawn:1817 - Preserved cgroup namespace via fd 20
lxc-start Asterisk 20190608100338.470 NOTICE   utils - utils.c:lxc_setgroups:1400 - Dropped additional groups
lxc-start Asterisk 20190608100338.471 NOTICE   start - start.c:start:2025 - Exec'ing "/sbin/init"
lxc-start Asterisk 20190608100338.471 ERROR    start - start.c:start:2028 - Permission denied - Failed to exec "/sbin/init"
lxc-start Asterisk 20190608100338.471 ERROR    sync - sync.c:__sync_wait:62 - An error occurred in another process (expected sequence number 7)
lxc-start Asterisk 20190608100338.471 WARN     network - network.c:lxc_delete_network_priv:2589 - Operation not permitted - Failed to remove interface "eth0" with index 40
lxc-start Asterisk 20190608100338.471 DEBUG    network - network.c:lxc_delete_network:3180 - Deleted network devices
lxc-start Asterisk 20190608100338.472 ERROR    start - start.c:__lxc_start:1939 - Failed to spawn container "Asterisk"
lxc-start Asterisk 20190608100338.561 ERROR    lxc_start - tools/lxc_start.c:main:330 - The container failed to start
lxc-start Asterisk 20190608100338.561 ERROR    lxc_start - tools/lxc_start.c:main:336 - Additional information can be obtained by setting the --logfile and --logpriority options

The different lines in compare to a second-container, on which i setup all permissions to 777 are (ok, network-interface is different):

1st Block:
lxc-start Asterisk 20190608100338.352 DEBUG    storage - storage/storage.c:get_storage_by_name:231 - Detected rootfs type "btrfs"
compare it: 
lxc-start Test 20190608100213.862 DEBUG    storage - storage/storage.c:storage_query:253 - Detected rootfs type "dir"

2nd Block:
lxc-start Asterisk 20190608100338.471 ERROR    start - start.c:start:2028 - Permission denied - Failed to exec "/sbin/init"
compare it:
lxc-start Test 20190608100214.228 ERROR    start - start.c:start:2028 - Exec format error - Failed to exec "/sbin/init"

Just to avoid any misunderstanding, despite the different file mask both container are failing to start with the same Permission denied - Failed to exec “/sbin/init.?

And to clarify to which path this been applied

  • assuming /path/to/container?

If am not mistaken exec “/sbin/init” refers to the root of the guest and not the host. And former gets supposedly temporarily mounted to /usr/lib/lxc/rootfs/

It is used to temporary mount the rootfs of lxc in a private mount namespace only visible by the processes running in the container.

Not sure if that might be the underlying cause, assuming that that uid (root 0) is all the same at your node directories that are involved in the lxc running?


Since you apparently participated in LXC containers - how to restore them? - Turris OS 4.x - Turris forum it would be reckoned that you compared the default.config and container.config between a new container and the old ones.

Did you source the container images consistently from the same source, e.g. turris.cz or linuxcontainers.org?

Suppose it will not make a difference but try lxc.arch = linux32 in the container config.

Though it used to be applied for ubuntu 14.04 kernel you may try with the old container config

lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
lxc.mount.entry = mqueue dev/mqueue mqueue rw,relatime,create=dir,optional 0 0

@BeCube, sorry bro for the late reply in my other topic, was kind of busy at work and all.

Last night when i read about Beta2 being released i updated it, however LXC was out and not visible in the lxc in luci. Just now i started to look what was wrong with it.

As there was no lxc container visible in luci i turned to the terminal and i saw that there was a problem the container not being visible. I suspected that it was because of the config being one based on version 2. I created another new container and decided to compare the new config file that is being created by lxc 3.x with the old config file.

Old lxc config file

root@K-Router:/mnt/LXC/K-Router-LXC# cat config.old

#Template used to create this container: /usr/share/lxc/templates/lxc-download
#Parameters passed to the template: --dist ubuntu --release bionic --arch armhf --server images.linuxcontainers.org --no-validate
#For additional config options, please look at lxc.container.conf(5)

#Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
#(Be aware this has security implications)

#Distribution configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.arch = linux32

#Container specific configuration
lxc.rootfs.path = btrfs:/mnt/LXC/K-Router-LXC/rootfs
lxc.uts.name = K-Router-LXC

lxc.tty.max = 3

#Network configuration
lxc.net.0.type = veth
lxc.net.0.link = br-lan
lxc.net.0.flags = up
lxc.net.0.name = eth0
lxc.net.0.mtu = 1500
lxc.net.0.ipv4.address = 192.168.1.210/24
lxc.net.0.ipv4.gateway = 192.168.1.1
#lxc.net.0.script.up = /usr/share/lxc/hooks/tx-off
lxc.net.0.hwaddr = a2:61:29:5c:0f:48

lxc.start.auto = 1
lxc.start.delay = 3

New config file of a new container

root@K-Router:/mnt/LXC/test# cat config
#Template used to create this container: /usr/share/lxc/templates/lxc-download
#Parameters passed to the template:
#For additional config options, please look at lxc.container.conf(5)

#Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
#(Be aware this has security implications)

#Some workarounds
#Template to generate fixed MAC address

#Distribution configuration
lxc.arch = armv7l

#Container specific configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.hook.start-host = /usr/share/lxc/hooks/systemd-workaround
lxc.rootfs.path = dir:/mnt/LXC/test/rootfs
lxc.uts.name = test

#Network configuration
lxc.net.0.type = veth
lxc.net.0.link = br-lan
lxc.net.0.flags = up
lxc.net.0.name = eth0
lxc.net.0.hwaddr = b2:c2:e5:3a:46:8c

Combination of old and new config

#Container specific configuration
root@K-Router:/mnt/LXC/K-Router-LXC# cat config
#Template used to create this container: /usr/share/lxc/templates/lxc-download
#Parameters passed to the template: --dist ubuntu --release bionic --arch armhf --server images.linuxcontainers.org --no-validate
#For additional config options, please look at lxc.container.conf(5)

#Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
#(Be aware this has security implications)

#Distribution configuration
lxc.arch = armv7l

#Container specific configuration
lxc.include = /usr/share/lxc/config/common.conf
lxc.hook.start-host = /usr/share/lxc/hooks/systemd-workaround
lxc.rootfs.path = btrfs:/mnt/LXC/K-Router-LXC/rootfs
lxc.uts.name = K-Router-LXC

lxc.tty.max = 3

#Network configuration
lxc.net.0.type = veth
lxc.net.0.link = br-lan
lxc.net.0.flags = up
lxc.net.0.name = eth0
lxc.net.0.mtu = 1500
lxc.net.0.ipv4.address = 192.168.1.210/24
lxc.net.0.ipv4.gateway = 192.168.1.1
#lxc.net.0.script.up = /usr/share/lxc/hooks/tx-off
lxc.net.0.hwaddr = a2:61:29:5c:0f:48

lxc.start.auto = 1
lxc.start.delay = 3

So far the whole systemd issue seems like it being solved as the systemd services have started at boot of the lxc. Big thanks to the TOS team for fixing this issue by updating to lxc 3.0.3, it bothered me alot as i had to rely on init.d however not all apps had a init.d script.

One thing I noticed is that

lxc.include = /usr/share/lxc/config/userns.conf

appears to be absent from the default conf and subsequent container conf, unless I missed it somewhere. Not sure whether it would make a difference though but perhaps worth a try

As you stated that you can start containers in b1 and b2, then most probably it is something in the config just like @anon50890781 said. In my topic “LXC containers - how to restore them?” there i also pointed at the config being the root cause of not working.

Hi again & Thanks again :slight_smile: (to big_boss too)

For the “Test” container, I primitively set all permissions from the /srv/lxc/Test/ directory to 777 to see if the container will be started.
I did not change the second container.
The difference between the two containers are the error message:
“Permission Denied” to “Exec Format Error”

Sorry, i do not understand you

When i remember correctly, i created both containers from the WebFrontend.
I check the config and Change it to the new OS4b2-format, no difference.
I Play around with you three hints (lxc.arch+lxc.include + lxc.mount.entry) and hints from [Big_boss].

My actual config is (i played with the additional lines, which i commented out & activate - but no important difference):

Distribution configuration

lxc.arch = armv7l
lxc.arch = linux32

Container specific configuration

lxc.include = /usr/share/lxc/config/common.conf
lxc.hook.start-host = /usr/share/lxc/hooks/systemd-workaround
lxc.tty.max = 4
lxc.pty.max = 1024
lxc.rootfs.path = btrfs:/srv/lxc/Asterisk/rootfs
lxc.uts.name = Asterisk
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
lxc.mount.entry = mqueue dev/mqueue mqueue rw,relatime,create=dir,optional 0 0

Network configuration

lxc.net.0.type = veth
lxc.net.0.link = br-lan
lxc.net.0.flags = up
lxc.net.0.name = eth0
lxc.net.0.hwaddr = 52:69:41:ea:74:a8

lxc.include = /usr/share/lxc/config/userns.conf
lxc.start.auto = 1
lxc.start.delay = 3

results in the log-file:

lxc-start Asterisk 20190608132228.713 INFO     lsm - lsm/lsm.c:lsm_init:50 - LSM security driver nop
lxc-start Asterisk 20190608132228.714 DEBUG    terminal - terminal.c:lxc_terminal_peer_default:714 - Using terminal "/dev/tty" as proxy
lxc-start Asterisk 20190608132228.714 DEBUG    terminal - terminal.c:lxc_terminal_signal_init:192 - Created signal fd 9
lxc-start Asterisk 20190608132228.714 DEBUG    terminal - terminal.c:lxc_terminal_winsz:90 - Set window size to 170 columns and 33 rows
lxc-start Asterisk 20190608132228.714 INFO     start - start.c:lxc_init:897 - Container "Asterisk" is initialized
lxc-start Asterisk 20190608132228.717 INFO     network - network.c:instantiate_veth:147 - Retrieved mtu 1500 from br-lan
lxc-start Asterisk 20190608132228.735 INFO     network - network.c:instantiate_veth:175 - Attached "veth40SYEO" to bridge "br-lan"
lxc-start Asterisk 20190608132228.742 DEBUG    network - network.c:instantiate_veth:201 - Instantiated veth "veth40SYEO/veth2TOK81", index is "66"
lxc-start Asterisk 20190608132228.742 DEBUG    cgfsng - cgroups/cgfsng.c:cg_legacy_handle_cpuset_hierarchy:620 - "cgroup.clone_children" was already set to "1"
lxc-start Asterisk 20190608132228.745 INFO     start - start.c:lxc_spawn:1688 - Cloned CLONE_NEWNS
lxc-start Asterisk 20190608132228.746 INFO     start - start.c:lxc_spawn:1688 - Cloned CLONE_NEWPID
lxc-start Asterisk 20190608132228.746 INFO     start - start.c:lxc_spawn:1688 - Cloned CLONE_NEWUTS
lxc-start Asterisk 20190608132228.746 INFO     start - start.c:lxc_spawn:1688 - Cloned CLONE_NEWIPC
lxc-start Asterisk 20190608132228.746 INFO     start - start.c:lxc_spawn:1688 - Cloned CLONE_NEWNET
lxc-start Asterisk 20190608132228.746 DEBUG    start - start.c:lxc_try_preserve_namespaces:196 - Preserved mnt namespace via fd 15
lxc-start Asterisk 20190608132228.746 DEBUG    start - start.c:lxc_try_preserve_namespaces:196 - Preserved pid namespace via fd 16
lxc-start Asterisk 20190608132228.746 DEBUG    start - start.c:lxc_try_preserve_namespaces:196 - Preserved uts namespace via fd 17
lxc-start Asterisk 20190608132228.746 DEBUG    start - start.c:lxc_try_preserve_namespaces:196 - Preserved ipc namespace via fd 18
lxc-start Asterisk 20190608132228.746 DEBUG    start - start.c:lxc_try_preserve_namespaces:196 - Preserved net namespace via fd 19
lxc-start Asterisk 20190608132228.747 DEBUG    start - start.c:lxc_spawn:1742 - Preserved net namespace via fd 10
lxc-start Asterisk 20190608132228.781 DEBUG    network - network.c:lxc_network_move_created_netdev_priv:2500 - Moved network device "veth2TOK81"/"eth0" to network namespace of 9183
lxc-start Asterisk 20190608132228.781 INFO     start - start.c:do_start:1242 - Unshared CLONE_NEWCGROUP
lxc-start Asterisk 20190608132228.782 DEBUG    storage - storage/storage.c:get_storage_by_name:231 - Detected rootfs type "btrfs"
lxc-start Asterisk 20190608132228.782 DEBUG    conf - conf.c:lxc_mount_rootfs:1332 - Mounted rootfs "/srv/lxc/Asterisk/rootfs" onto "/usr/lib/lxc/rootfs" with options "(null)"
lxc-start Asterisk 20190608132228.782 INFO     conf - conf.c:setup_utsname:791 - Set hostname to "Asterisk"
lxc-start Asterisk 20190608132228.811 DEBUG    network - network.c:setup_hw_addr:2767 - Mac address "52:69:41:ea:74:a8" on "eth0" has been setup
lxc-start Asterisk 20190608132228.817 DEBUG    network - network.c:lxc_setup_netdev_in_child_namespaces:3032 - Network device "eth0" has been setup
lxc-start Asterisk 20190608132228.817 INFO     network - network.c:lxc_setup_network_in_child_namespaces:3053 - network has been setup
lxc-start Asterisk 20190608132228.817 INFO     conf - conf.c:mount_autodev:1118 - Preparing "/dev"
lxc-start Asterisk 20190608132228.843 INFO     conf - conf.c:mount_autodev:1165 - Prepared "/dev"
lxc-start Asterisk 20190608132228.843 INFO     conf - conf.c:lxc_mount_auto_mounts:721 - Mount source or target for "/usr/lib/lxc/rootfs/proc/sys/net" on "/usr/lib/lxc/rootfs/proc/tty" does not exist. Skipping
lxc-start Asterisk 20190608132228.844 ERROR    utils - utils.c:safe_mount:1179 - No such file or directory - Failed to mount "/usr/lib/lxc/rootfs/proc/tty" onto "/usr/lib/lxc/rootfs/proc/sys/net"
lxc-start Asterisk 20190608132228.844 INFO     conf - conf.c:lxc_mount_auto_mounts:721 - Mount source or target for "/usr/lib/lxc/rootfs/proc/tty" on "/usr/lib/lxc/rootfs/proc/sys/net" does not exist. Skipping
lxc-start Asterisk 20190608132228.845 INFO     conf - conf.c:lxc_fill_autodev:1209 - Populating "/dev"
lxc-start Asterisk 20190608132228.845 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/full"
lxc-start Asterisk 20190608132228.845 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/null"
lxc-start Asterisk 20190608132228.845 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/random"
lxc-start Asterisk 20190608132228.845 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/tty"
lxc-start Asterisk 20190608132228.845 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/urandom"
lxc-start Asterisk 20190608132228.845 DEBUG    conf - conf.c:lxc_fill_autodev:1224 - Created device node "/usr/lib/lxc/rootfs/dev/zero"
lxc-start Asterisk 20190608132228.845 INFO     conf - conf.c:lxc_fill_autodev:1286 - Populated "/dev"
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2027 - Remounting "/sys/fs/fuse/connections" on "/usr/lib/lxc/rootfs/sys/fs/fuse/connections" to respect bind or remount options
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2048 - Flags for "/sys/fs/fuse/connections" were 1070, required extra flags are 14
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2102 - Mounted "/sys/fs/fuse/connections" on "/usr/lib/lxc/rootfs/sys/fs/fuse/connections" with filesystem type "none"
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2027 - Remounting "/sys/kernel/debug" on "/usr/lib/lxc/rootfs/sys/kernel/debug" to respect bind or remount options
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2048 - Flags for "/sys/kernel/debug" were 1056, required extra flags are 0
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2058 - Mountflags already were 4096, skipping remount
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2102 - Mounted "/sys/kernel/debug" on "/usr/lib/lxc/rootfs/sys/kernel/debug" with filesystem type "none"
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2027 - Remounting "/sys/kernel/security" on "/usr/lib/lxc/rootfs/sys/kernel/security" to respect bind or remount options
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2048 - Flags for "/sys/kernel/security" were 1070, required extra flags are 14
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2102 - Mounted "/sys/kernel/security" on "/usr/lib/lxc/rootfs/sys/kernel/security" with filesystem type "none"
lxc-start Asterisk 20190608132228.846 INFO     conf - conf.c:mount_entry:2014 - No such file or directory - Failed to mount "/sys/fs/pstore" on "/usr/lib/lxc/rootfs/sys/fs/pstore" (optional)
lxc-start Asterisk 20190608132228.846 DEBUG    conf - conf.c:mount_entry:2102 - Mounted "mqueue" on "/usr/lib/lxc/rootfs/dev/mqueue" with filesystem type "mqueue"
lxc-start Asterisk 20190608132228.846 INFO     conf - conf.c:mount_file_entries:2333 - Finished setting up mounts
lxc-start Asterisk 20190608132228.847 DEBUG    conf - conf.c:lxc_setup_dev_console:1771 - Mounted pts device "/dev/pts/1" onto "/usr/lib/lxc/rootfs/dev/console"
lxc-start Asterisk 20190608132228.847 INFO     utils - utils.c:lxc_mount_proc_if_needed:1231 - I am 1, /proc/self points to "1"
lxc-start Asterisk 20190608132228.890 WARN     conf - conf.c:lxc_setup_devpts:1616 - Invalid argument - Failed to unmount old devpts instance
lxc-start Asterisk 20190608132228.891 DEBUG    conf - conf.c:lxc_setup_devpts:1653 - Mount new devpts instance with options "gid=5,newinstance,ptmxmode=0666,mode=0620,max=1024"
lxc-start Asterisk 20190608132228.891 DEBUG    conf - conf.c:lxc_setup_devpts:1672 - Created dummy "/dev/ptmx" file as bind mount target
lxc-start Asterisk 20190608132228.891 DEBUG    conf - conf.c:lxc_setup_devpts:1677 - Bind mounted "/dev/pts/ptmx" to "/dev/ptmx"
lxc-start Asterisk 20190608132228.891 DEBUG    conf - conf.c:lxc_allocate_ttys:989 - Created tty "/dev/pts/0" with master fd 11 and slave fd 14
lxc-start Asterisk 20190608132228.891 DEBUG    conf - conf.c:lxc_allocate_ttys:989 - Created tty "/dev/pts/1" with master fd 15 and slave fd 16
lxc-start Asterisk 20190608132228.891 DEBUG    conf - conf.c:lxc_allocate_ttys:989 - Created tty "/dev/pts/2" with master fd 17 and slave fd 18
lxc-start Asterisk 20190608132228.891 DEBUG    conf - conf.c:lxc_allocate_ttys:989 - Created tty "/dev/pts/3" with master fd 19 and slave fd 20
lxc-start Asterisk 20190608132228.891 INFO     conf - conf.c:lxc_allocate_ttys:1005 - Finished creating 4 tty devices
lxc-start Asterisk 20190608132228.891 DEBUG    conf - conf.c:lxc_setup_ttys:940 - Bind mounted "/dev/pts/0" onto "/dev/tty1"
lxc-start Asterisk 20190608132228.892 DEBUG    conf - conf.c:lxc_setup_ttys:940 - Bind mounted "/dev/pts/1" onto "/dev/tty2"
lxc-start Asterisk 20190608132228.892 DEBUG    conf - conf.c:lxc_setup_ttys:940 - Bind mounted "/dev/pts/2" onto "/dev/tty3"
lxc-start Asterisk 20190608132228.892 DEBUG    conf - conf.c:lxc_setup_ttys:940 - Bind mounted "/dev/pts/3" onto "/dev/tty4"
lxc-start Asterisk 20190608132228.892 INFO     conf - conf.c:lxc_setup_ttys:949 - Finished setting up 4 /dev/tty<N> device(s)
lxc-start Asterisk 20190608132228.892 INFO     conf - conf.c:setup_personality:1716 - Set personality to "0x8"
lxc-start Asterisk 20190608132228.892 DEBUG    conf - conf.c:setup_caps:2506 - Capabilities have been setup
lxc-start Asterisk 20190608132228.892 NOTICE   conf - conf.c:lxc_setup:3692 - The container "Asterisk" is set up
lxc-start Asterisk 20190608132228.892 DEBUG    start - start.c:lxc_spawn:1817 - Preserved cgroup namespace via fd 20
lxc-start Asterisk 20190608132228.892 INFO     conf - conf.c:run_script_argv:356 - Executing script "/usr/share/lxc/hooks/systemd-workaround" for container "Asterisk", config section "lxc"
lxc-start Asterisk 20190608132228.895 NOTICE   utils - utils.c:lxc_setgroups:1400 - Dropped additional groups
lxc-start Asterisk 20190608132228.895 NOTICE   start - start.c:start:2025 - Exec'ing "/sbin/init"
lxc-start Asterisk 20190608132228.895 ERROR    start - start.c:start:2028 - Permission denied - Failed to exec "/sbin/init"
lxc-start Asterisk 20190608132228.896 ERROR    sync - sync.c:__sync_wait:62 - An error occurred in another process (expected sequence number 7)
lxc-start Asterisk 20190608132228.896 WARN     network - network.c:lxc_delete_network_priv:2589 - Operation not permitted - Failed to remove interface "eth0" with index 66
lxc-start Asterisk 20190608132228.896 DEBUG    network - network.c:lxc_delete_network:3180 - Deleted network devices
lxc-start Asterisk 20190608132228.896 ERROR    start - start.c:__lxc_start:1939 - Failed to spawn container "Asterisk"
lxc-start Asterisk 20190608132228.990 ERROR    lxc_start - tools/lxc_start.c:main:330 - The container failed to start
lxc-start Asterisk 20190608132228.991 ERROR    lxc_start - tools/lxc_start.c:main:336 - Additional information can be obtained by setting the --logfile and --logpriority options

the second lxc-container (that one with the 777 Manipulation) has a similar log (difference “Permission Denied” to “Exec Format Error”)

I assume that this is your new config right?

Do remove THE second lxc.include.

EDIT: I also ASSUME it is a BTRFS file system right? The external drive where the LXC is stored on?

Can you post the output of this.

cat /etc/config/fstab

As i already have asked you some of the things, can you also put the output of this command.

ls -lisa /srv/lxc

It will look something like this.

Output

root@K-Router:/mnt# ls -lisa LXC/
256 16 drwxr-xr-x 1 root root 150 Jun 8 14:59 .
1286 0 drwxr-xr-x 1 root root 44 May 3 20:30 …
261 0 drwxrwx— 1 root root 102 Jun 8 14:04 K-Router-LXC
127695 0 drwxr-xr-x 1 root root 22 Oct 2 2018 backup
857367 0 drwxrwx— 1 root root 0 Mar 10 09:47 just_want_to_list_available_lxc_templates
18609 8 -rw------- 1 root root 5731 Mar 16 13:46 lxc-monitord.log

Yes, it’s my actual config.
No, the remove changes Nothing (Exec Format Error & Permission error)

Yes, internal SSD, Setup with “Storage” in Foris.

Output

config global
option auto_swap ‘1’
option auto_mount ‘0’
option delay_root ‘5’
option check_fs ‘0’
option anon_swap ‘0’
option anon_mount ‘0’

config swap
option uuid ‘8b82e242-4de8-4767-91bb-c18e4a6c43ab’
option enabled ‘0’

config mount
option target ‘/mnt/sda2’
option uuid ‘ca433ce6-59fd-4d09-8ae0-6461542f1efd’
option enabled ‘0’

config mount
option target ‘/mnt/sdb1’
option uuid ‘0000001800000048’
option enabled ‘0’

config mount
option enabled ‘1’
option uuid ‘AAD0B38FD0B35FED’
option target ‘/mnt/sdb1’

Output

327755 0 drwxr-xr-x 1 root root 144 Jun 8 15:59 .
256 0 drwxr-xr-x 1 root root 816 May 13 11:46 …
370672 0 drwxrwx— 1 root root 130 May 12 19:08 Asterisk
328467 0 drwxrwx— 1 root root 24 May 30 20:55 Test

What if you used the config of the test container for your actual container, ONLY changed a few settings, for example, the name (uts), rootfs. With the rootfs, also NOT changing the dir to btfrs. just ONLY the names where the directory is.

i changed this two lines:
lxc.rootfs.path = btrfs:/srv/lxc/CosmicTest2/rootfs
lxc.uts.name = CosmicTest2

The Cosmic-Container starts + lxc-attach is possible

Addtional, Maybe it’s interesting:

root@Fritz:/srv/lxc# ls -lisa /srv/lxc/Asterisk/rootfs/sbin/init
 374164    192 -r--------    1 root     root        194844 Jul 18  2014 /srv/lxc/Asterisk/rootfs/sbin/init
root@Fritz:/srv/lxc# ls -lisa /srv/lxc/CosmicTest2/rootfs/sbin/init
  17345      4 lrwxrwxrwx    1 root     root            20 Apr  4 13:29 /srv/lxc/CosmicTest2/rootfs/sbin/init -> /lib/systemd/systemd