How to use the cake queue management system on the Turris Omnia

howto

#1

Cake is a comprehensive queue management system, implemented as a queue discipline (qdisc) for the Linux kernel. It is designed to replace and improve upon the complex hierarchy of simple qdiscs presently required to effectively tackle the bufferbloat problem at the network edge.

Bufferbloat is the undesirable latency that comes from a router or other network equipment buffering too much data. It is a huge drag on Internet performance created, ironically, by previous attempts to make it work better. The one-sentence summary is “Bloated buffers lead to network-crippling latency spikes.”

Alright, sounds good! But, how do we use cake on the Turris Omnia?

How to install SQM QoS with cake:

opkg update && opkg install sqm-scripts luci-app-sqm kmod-sched-cake

If you do not want to use luci, remove luci-app-sqm from the command above. For configuring SQM QoS manually see below.

Configure using luci (web interface):

Now in the webinterface, under the network tab, a new item, SQM QoS should be there. If not log out, delete browser cache and log in. You could also change browser.

- How to configure SQM QoS to use cake:
Lots of knobs and input fields here but configuring SQM QoS to use cake is actually a piece of cake. SQM QoS has three tabs: Basic Settings, Queue Discipline and Link Layer Adaptation. We need all three of them in order to correctly setup cake.
- Basic Settings settings:

  • Tick Enable this SQM instance
  • Select eth1 under Interface name
  • Set Download speed and Upload speed to 85-95% of your rated line speed. (Note that the speed is in kbit/s, eg. a 150/15 Mbit/s = 150000/15000 kbit/s.)

- Queue Discipline settings:

  • Select cake under Queuing disciplines usable on this system. (If there is no cake (:sob:) among the available options either reboot or run the following command in terminal using ssh: “/usr/lib/sqm/update-available-qdiscs”.)
  • Select piece_of_cake.qos under Queue setup script.
  • check the box named:
    “Show and Use Advanced Configuration. Advanced options will only be used as long as this box is checked.”
  • Then check the box named:
    “Show and Use Dangerous Configuration. Dangerous options will only be used as long as this box is checked.”
  • Now enter the following (without the quotes) into the “Advanced option string to pass to the ingress queueing disciplines; no error checking, use very carefully.” field:
    “nat dual-dsthost”
  • and to to the “Advanced option string to pass to the egress queueing disciplines; no error checking, use very carefully.
    ” field “nat dual-srchost”

- Link Layer Adaptation settings:

  • If you have a cable connection you don’t need to do anything here. However, if you have ADSL you will need to select ATM. If you have VDSL you will need to select Ethernet.
  • If you have an ADSL or VDSL line you will need to specify a Per Packet Overhead value. For ADSL, use 44. For VDSL, use 8. (Note, overhead can vary! If you want to measure it you can use this project)

###Configure using an editor:

  • Point your editor of choice to /etc/config/sqm
  • Edit the file to make it look like this:
    config queue 'eth1’
    option enabled ‘1’ (0 = disabled; 1 = enabled)
    option interface ‘eth1’ (WAN interface, should be eth1)
    option download '85-95% of your download speed in kbit/s’
    option upload '85-95% of your upload speed in kbit/s’
    option debug_logging ‘0’ (disable = 0; enable = 1 debugging)
    option verbosity ‘5’ (Verbosity of SQM’s output into the system log. 5 = info)
    option qdisc ‘cake’ (since we want cake, this should be cake)
    option script ‘piece_of_cake.qos’ (self explanatory)
    option qdisc_advanced ‘1’ (disable = 0; enable =1 advanced qdisc configuration)
    option squash_dscp ‘1’ (0 = disable; 1 = enable squash DSCP on inbound (ingress) packets)
    option squash_ingress ‘1’ (0 = regard; 1 = ignore DSCP on ingress)
    option ingress_ecn ‘ECN’ (ECN or NOECN, explicit congestion notification status on inbound packets (ingress))
    option egress_ecn ‘NOECN’ (ECN or NOECN, explicit congestion notification status on outbound packets (regress))
    option qdisc_really_really_advanced ‘1’ (disable = 0; enable =1 really advanced qdisc configuration)
    option iqdisc_opts ‘nat dual-dsthost’ (advanced option string to pass to the ingress queueing disciplines; no error checking, use very carefully. Setting nat dual-dsthost makes cake nat aware. Only set this when the set interface is eth1)
    option eqdisc_opts ‘nat dual-srchost’ (advanced option string to pass to the regress queueing disciplines; no error checking, use very carefully. Setting nat dual-srchost makes cake nat aware.Only set this when the set interface is eth1)
    option linklayer ‘atm’ (“none” (cable), “ethernet” (vdsl), or “atm” (adsl).
    option overhead ‘0/8/44’ (per packet overhead in bytes)

Example sqm conf file:

config queue 'eth1’
option enabled '1’
option interface 'eth0.2’
option download '9000’
option upload '900’
option debug_logging '0’
option verbosity '5’
option qdisc 'cake’
option script 'piece_of_cake.qos’
option qdisc_advanced '1’
option squash_dscp '1’
option squash_dscp '1’
option squash_ingress '1’
option ingress_ecn 'ECN’
option egress_ecn 'NOECN’
option qdisc_really_really_advanced '1’
option iqdisc_opts 'nat dual-dsthost’
option eqdisc_opts 'nat dual-srchost’
option linklayer 'atm’
option overhead ‘44’

Further reading:


Using SQM . can i set bandwidth to each ethernet port on turris?
Way to have turris.omnia run like stock LEDE?
Internet radio stutters when running a speedtest
Turris Omnia possible routing issues
Stuck trying to get qos-scripts working on the router
Stuck trying to get qos-scripts working on the router
Mysterious Updater failed message, followed by bad connection
Post here about the "upcoming update"
#2

Oh great advise! May I add two more ideas to make cake really fly?
On the “Queue Discipline” tab of the SQM luci app, check the box named:
"Show and Use Advanced Configuration. Advanced options will only be used as long as this box is checked."
Then check the box named:
“Show and Use Dangerous Configuration. Dangerous options will only be used as long as this box is checked.”

Now enter the following (without the quotes) into the “Advanced option string to pass to the ingress queueing disciplines; no error checking, use very carefully.” field:
“nat dual-dsthost"
and to to the “Advanced option string to pass to the egress queueing disciplines; no error checking, use very carefully.
” field
"nat dual-srchost”

And make sure to empty these fields again before switching to another qdisc as otherwise sqm will not work (until you empty these fields again).

Final Note: the dual-xxxhost options are directional, so the above works if cake is instantiate on an outwad facing WAN interface, if however cake is instantiated on an inward facing LAN interface (which is also a valid configuration) the direction of the sqm download and upload fields (or generally the whole notion of ingress and egress) flips in relation to the internet. Again this is quite logical as ingress and egress are properties of any network interface and always depend on what an interface is connected to. But in the cake-on-LAN case put dual-dsthost into the “Advanced option string to pass to the egress…” field and dual-srchost into the ingress one. Im a not sure whether the “nat” option will actually do something useful in the on-LAN case though.

Best Regards


#3

You learn something new every day. Thanks for the excellent addition!
Tested this on 2 routers and the results are even better :smiley:

For the people who like to read this is an interesting conversation about de-natting (also including @moeller0)


#4

As a long time fq_codel user, I’d love to give cake a try, but on 3.5.1 I’m running into a weird problem.

After I install kmod-sched-cake, it upgrades the kernel to 4.4.39-1-80079e1c1e5f9ca7ad734044462a761a-4 and upon reboot i’m in a reboot loop. It looks like kernel modules aren’t there or can’t be loaded. Luckily, schnapps can take me back to the state I was in before.

Has anyone else run into this?


#5

That is strange, my TO has this kernel version: 4.4.39-80079e1c1e5f9ca7ad734044462a761a-3
Are you on a different branch?


#6

I’m on stable and just upgraded to to the latest stable yesterday morning.


#7

Probably a stupid question but is your TO is on 3.5.1 just like mine? And, if so, why do we have different kernel versions? That simply makes no sense.
Although, I found this commit which might explain it. Feeds were updated, thus maybe the requirements and thus you have a different kernel installed. In the test branch we are already up to 4.4.44 if I am not mistaken.
Tonight, when I have the time, I will try removing and installing cake to see if I can replicate your issue.


#8

Yep, that was one of the first things I double checked. I upgraded and then went for the cake try back to back.

# cat /etc/turris-version
3.5.1

#9

I must have caught things mid-update yesterday. Now if I do a

opkg list-upgradable

I’m seeing all the kernel mods wanting to go to -4, and that was not the case yesterday. I’ll do an update and see what happens.

Edit:
Update: updater didn’t find the new packages, so I went YOLO and went via opkg. So far so good.


#10

Good to hear! (+ it saves me time :sweat_smile:).
Hopefully no weird breakage in the future.


#11

Do I have to keep the additional settings coming with the advanced or can they be deleted?

    option qdisc_advanced '1'
    option squash_dscp '1'  <-- this
    option squash_ingress '1'  <-- this
    option ingress_ecn 'ECN'  <-- this
    option egress_ecn 'NOECN'  <-- and this
    option qdisc_really_really_advanced '1'
    option iqdisc_opts 'nat dual-dsthost'
    option eqdisc_opts 'nat dual-srchost'

#12

Keep those, unless you do not want them, they are considered the defaults… BUT really make sure to empty iqdisc_opts and eqdisc_opts before switching back to a non-cake qdisc, as otherwise sqm-scripts will fail to start-up and function properly, there is a reason why these are in the dangerous section of the GUI (one goal was to make the common thing easy and the more elaborate at least possible, and especially to allow to test and do things not considered at the time sqm-scripts was created).

Then again, these are defaults you might want to change:
SQUASH_DSCP this basically will remap incoming packet’s DSCP to the default 0
SQUASK_INGRESS this will ignore DSCP markings on incoming packets
These two are separate as the iptables based DSCP remapping unfortunately only happens after the ifb device, so sqm-scripts ingress shaper will see these markings no matter what.
INGRESS_ECN this will use ecn markings on incoming packets to signal explicit congestion notification using flows to slow down instead of throwing the packet away. EGRESS_ECN will do the same for outgoing packets. Since packets on the downstream typically already have passed the true bottleneck default to ECN marking as dropping those will neither signal congestion faster (but actually a tiny bit slower) nor will it speed up data delivery. On egress the default is noecn, as typically egress bandwidth is much lower so not sending a packet allows to keep the sent packet’s latency lower and since the packet still might be dropped along the path it is not really that certain that the ECN marking will actually help the sender react faster to the congestion event. Current opinion seems to be that with reasonable fast egress links (>=10Mbps?) ecn could also be used as default on egress, but I digress.


#13

TL;DR: These are the default values and it is advisable for non-experts to keep them like that.

What do these values mean? (source)

  • ingress_ecn: either “ECN” or “NOECN”.
    Explicit congestion notification (ECN) status on inbound packets (ingress)
  • egress_ecn: either “ECN” or “NOECN”.
    Explicit congestion notification (ECN) status on outbound packets (egress)
  • squash_dscp: either “0” or “1”.
    Squash DSCP on inbound packets (ingress). 1 = squash, 0 = do not squash
  • squash_ingress: either “0” or “1”.
    Ignore DSCP on ingress. 1 = ignore, 0 = allow

What is ECN? (more reading)

Explicit Congestion Notification (ECN) is an extension to the Internet Protocol and to the Transmission Control Protocol and is defined in RFC 3168 (2001). ECN allows end-to-end notification of network congestion without dropping packets. ECN is an optional feature that may be used between two ECN-enabled endpoints when the underlying network infrastructure also supports it.

What is DSCP? (more reading)

Differentiated services or DiffServ is a computer networking architecture that specifies a simple, scalable and coarse-grained mechanism for classifying and managing network traffic and providing quality of service (QoS) on modern IP networks. DiffServ can, for example, be used to provide low-latency to critical network traffic such as voice or streaming media while providing simple best-effort service to non-critical services such as web traffic or file transfers.


#14

I don’t see “cake” under Queuing disciplines. Only:

fq_codel
pie
codel
sfq

Is there some sort of refresh I need to do so luci-app-sqm can see “cake” as an option?


#15

you need to install kmod-sched-cake and then either reboot or delete the /var/run/sqm/available_qdiscs directorry and its contents. But if you just added kmod-sched-cake running “/usr/bin/sqm/update-available-qdiscs” should also do the trick, if not just reboot the router… (testing the different qdiscs has enough potential side-effects that it can not run per-default on every invocation of luci-app-sqm, so this is a decent compromise so that people who have no active sqm instance should not feel any side effects, but I digress)

Best Regards


#16

Ah, it’s “/usr/lib/sqm/update-available-qdiscs”.

Thanks.


#17

Ooops, sorry for the incorrect path, but you luckily understood the gist of it :wink:

I hope it worked like you expected…


#18

Updated the how-to with what to do when there is no cake among the available queuing disciplines even though kmod-sched-cake is installed. (And, of course, I could not resist the pun)


#19

There is also the possibility if not seeing the cake discipline in Luci even when it is installed.Maybe you put a line on editing the config file.


#20

How to refresh available qdiscs is in there :wink:
Added 50% of how to configure SQM QoS with cake but did not have the time to completely finish it. I will update it as soon as I can.