SQM on turris flent benchmarks?

Will do tonight.

No, with only a mild 100/36 asymmetry and a nominally non-bursty link I leave ACKs as they are.

Can do tonight once everybody else stopped using the network.

1 Like

No SQM:

moeller@work-horse:~/CODE/flent$ ./run-flent -x --ipv4 -l 120 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net rtt_fair4be --remote-metadata=root@192.168.42.1 --test-parameter=cpu_stats_hosts=root@192.168.42.1 --step-size=.05 --socket-stats -D . -t ingress_IPv4_NoSQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_netperf-eu.bufferbloat.net --log-file
Starting Flent 2.0.1-git-65cd64f using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 130 seconds.
Data file written to ./rtt_fair4be-2022-04-19T231920.248480.ingress_IPv4_NoSQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS_5_3_7-pppoe-wan-eth2_7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDS.flent.gz

Summary of rtt_fair4be test run from 2022-04-19 21:19:20.248480
  Title: 'ingress_IPv4_NoSQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_netperf-eu.bufferbloat.net'

                                                                          avg       median       99th %          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :        36.60        38.20        53.82 ms             2931
 Ping (ms) ICMP2 london.starlink.taht.net                      :        47.60        49.20        65.02 ms             2931
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :       190.00       191.00       207.00 ms             2931
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :       191.00       192.00       209.00 ms             2931
 Ping (ms) avg                                                 :       116.30          N/A          N/A ms             2931
 TCP download BE1 de.starlink.taht.net                         :        44.94        48.09        65.22 Mbits/s        2931
 TCP download BE2 london.starlink.taht.net                     :        25.23        27.17        39.78 Mbits/s        2931
 TCP download BE3 singapore.starlink.taht.net                  :        19.34        16.23        69.87 Mbits/s        2931
 TCP download BE4 fremont.starlink.taht.net                    :        13.39        11.72        41.27 Mbits/s        2931
 TCP download avg                                              :        25.73          N/A          N/A Mbits/s        2931
 TCP download fairness                                         :         0.82          N/A          N/A Mbits/s        2931
 TCP download sum                                              :       102.90          N/A          N/A Mbits/s        2931
 TCP upload BE1 de.starlink.taht.net                           :        14.08        14.46        20.12 Mbits/s        2931
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :        48.29        49.00        75.00                1695
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :        13.79        13.80        19.03                1694
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :        17.74        17.36        33.62                1695
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :        39.24        40.31        54.74                1692
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :         0.93         0.74         4.86                1692
 TCP upload BE2 london.starlink.taht.net                       :         9.49         9.66        15.08 Mbits/s        2931
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :        42.11        40.00        77.00                1697
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         9.32         9.19        14.39                1696
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :        11.90        11.59        21.32                1696
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :        50.27        51.27        65.51                1693
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :         1.25         1.01         4.98                1693
 TCP upload BE3 singapore.starlink.taht.net                    :         2.50         2.58         5.48 Mbits/s        2931
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :        44.55        40.00        78.00                1720
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         2.49         2.23         4.22                1715
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         3.19         2.79         6.39                1718
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       201.28       202.55       219.12                1714
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :         3.43         2.87        10.46                1714
 TCP upload BE4 fremont.starlink.taht.net                      :         2.30         2.33         5.38 Mbits/s        2931
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :        44.15        41.00        82.00                1727
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         2.29         2.09         4.14                1721
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         2.95         2.62         6.16                1724
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :       217.27       218.09       237.47                1722
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :         3.66         3.02        10.61                1722
 TCP upload avg                                                :         7.09          N/A          N/A Mbits/s        2931
 TCP upload fairness                                           :         0.67          N/A          N/A Mbits/s        2931
 TCP upload sum                                                :        28.37          N/A          N/A Mbits/s        2931
 cpu_stats_root@192.168.42.1::load                             :         0.26         0.25         0.67                2292

Note what My ISP does to my latency under load, and how much worst this looks compared to the competent AQM/traffic shaper that is cake.

2 Likes

Base on that last result… your ISP is actually doing a pretty good job at defaults. Most end-users (not gamers) would be perfectly content with only the 60ms latency and jitter you are showing, as it’s kilometers better than most dsl and cable. But since we both have OCD and can do much better for all uses and users and all workloads, we sally forth, with cake.

Thx VERY much for testing this on the turris for me. I’m still hoping other turris users running at > 500Mbit can chip in, which was my original question!

Also thx for validating my new cloud!

1 Like

in terms of moving variables around, could you re-test with cake 100mbit with the default memlimit? I’m pretty sure at this point that its too low from various other tests, but…

1 Like

Yes they have come a long way… this is a VDSL2 link with bidirectional vectoring and G.INP retransmissions. But I am using my own modem and router so can not fully appreciate the experience of the bulk of my ISPs customers who likely use the all in one modem router offered by the ISP.

Again, yes. Since it is not that hard to overload/saturate my link I really appreciate the IP and flow isolation cake’s scheduler offers…

+1; over here the plan now is to convert the bulk of the links to FTTH by 2030, though that is more aspirational than reliable. So I guess if nobody beats me to it, I might be able to post such a result within the next 8 years :wink:

My pleasure, thanks for sharing.

I can and will do tonight, but when I had a look at cake’s reported memory usage (by manually calling tc -s qdisc on a second terminal during the test) it did not seem to get close to the default 4MB.

1 Like

Reference memlimit 32mb 105/36

run-flent -x --ipv4 -l 60 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net rtt_fair4be --remote-metadata=root@192.168.42.1 --test-parameter=cpu_stats_hosts=root@192.168.42.1 --step-size=.05 --socket-stats -D . -t memlimit32mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_xxx.starlink.taht.net --log-file

Starting Flent 2.0.1-git-65cd64f using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-21T203810.994215.memlimit32mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS_5_3_7-pppoe-wan-eth2_7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-.flent.gz

Summary of rtt_fair4be test run from 2022-04-21 18:38:10.994215
  Title: 'memlimit32mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_xxx.starlink.taht.net'

                                                                          avg       median       99th %          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :        19.30        19.10        26.61 ms             1604
 Ping (ms) ICMP2 london.starlink.taht.net                      :        31.60        31.40        37.70 ms             1604
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :       172.00       172.00       179.00 ms             1604
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :       183.00       182.00       189.00 ms             1604
 Ping (ms) avg                                                 :       101.47          N/A          N/A ms             1604
 TCP download BE1 de.starlink.taht.net                         :        37.21        36.22        56.90 Mbits/s        1604
 TCP download BE2 london.starlink.taht.net                     :        34.12        34.30        46.01 Mbits/s        1604
 TCP download BE3 singapore.starlink.taht.net                  :         3.95         4.29         7.09 Mbits/s        1604
 TCP download BE4 fremont.starlink.taht.net                    :        19.67        21.34        34.80 Mbits/s        1604
 TCP download avg                                              :        23.74          N/A          N/A Mbits/s        1604
 TCP download fairness                                         :         0.76          N/A          N/A Mbits/s        1604
 TCP download sum                                              :        94.95          N/A          N/A Mbits/s        1604
 TCP upload BE1 de.starlink.taht.net                           :         9.73         9.59        22.44 Mbits/s        1604
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :        22.61        22.00        39.00                 854
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         9.30         8.70        15.50                 853
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :        12.06        11.39        20.40                 853
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :        26.31        25.80        36.13                 851
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :         1.47         1.27         4.38                 851
 TCP upload BE2 london.starlink.taht.net                       :         9.30         9.11        16.58 Mbits/s        1604
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :        30.45        29.00        53.00                 852
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         8.98         8.44        14.59                 851
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :        11.70        10.91        20.25                 851
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :        37.27        36.49        46.68                 848
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :         1.26         1.03         4.46                 848
 TCP upload BE3 singapore.starlink.taht.net                    :         5.74         6.37        10.10 Mbits/s        1604
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :        92.40        97.00       140.42                 859
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         5.73         6.02         8.41                 854
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         7.00         7.32        10.68                 857
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       184.02       183.31       200.00                 855
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :         2.11         1.22        12.11                 855
 TCP upload BE4 fremont.starlink.taht.net                      :         5.90         6.73         9.52 Mbits/s        1604
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :        95.01       103.00       137.00                 857
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         5.89         6.46         8.49                 852
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         7.36         7.97        14.42                 854
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :       184.54       183.87       197.84                 851
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :         1.84         1.10        11.00                 851
 TCP upload avg                                                :         7.67          N/A          N/A Mbits/s        1604
 TCP upload fairness                                           :         0.94          N/A          N/A Mbits/s        1604
 TCP upload sum                                                :        30.67          N/A          N/A Mbits/s        1604
 cpu_stats_root@192.168.42.1::load                             :         0.36         0.33         0.79                1248

And now with the OpenWrt default of 4mb per direction:

user@work-horse:~/CODE/flent$ ./run-flent -x --ipv4 -l 60 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net rtt_fair4be --remote-metadata=root@192.168.42.1 --test-parameter=cpu_stats_hosts=root@192.168.42.1 --step-size=.05 --socket-stats -D . -t memlimit4mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_xxx.starlink.taht.net --log-file
Starting Flent 2.0.1-git-65cd64f using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-21T205112.644552.memlimit4mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS_5_3_7-pppoe-wan-eth2_7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-V.flent.gz

Summary of rtt_fair4be test run from 2022-04-21 18:51:12.644552
  Title: 'memlimit4mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_xxx.starlink.taht.net'

                                                                          avg       median       99th %          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :        19.30        18.90        25.60 ms             1631
 Ping (ms) ICMP2 london.starlink.taht.net                      :        30.90        30.60        37.60 ms             1631
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :       172.00       172.00       178.00 ms             1631
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :       183.00       182.00       189.00 ms             1631
 Ping (ms) avg                                                 :       101.30          N/A          N/A ms             1631
 TCP download BE1 de.starlink.taht.net                         :        37.17        36.36        55.30 Mbits/s        1631
 TCP download BE2 london.starlink.taht.net                     :        33.21        33.68        48.78 Mbits/s        1631
 TCP download BE3 singapore.starlink.taht.net                  :         4.13         4.30         7.42 Mbits/s        1631
 TCP download BE4 fremont.starlink.taht.net                    :        19.49        19.94        42.91 Mbits/s        1631
 TCP download avg                                              :        23.50          N/A          N/A Mbits/s        1631
 TCP download fairness                                         :         0.77          N/A          N/A Mbits/s        1631
 TCP download sum                                              :        94.00          N/A          N/A Mbits/s        1631
 TCP upload BE1 de.starlink.taht.net                           :         9.68         9.23        18.78 Mbits/s        1631
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :        22.50        21.00        39.73                 828
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :         9.23         8.69        15.79                 828
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :        11.92        11.31        21.14                 828
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :        26.36        25.79        36.15                 826
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :         1.38         1.16         4.69                 826
 TCP upload BE2 london.starlink.taht.net                       :         9.32         8.98        19.14 Mbits/s        1631
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :        30.38        29.00        49.00                 831
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :         9.02         8.54        14.58                 830
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :        11.62        10.91        19.67                 830
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :        37.16        36.64        47.22                 829
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :         1.23         0.99         4.54                 829
 TCP upload BE3 singapore.starlink.taht.net                    :         5.66         6.28         9.91 Mbits/s        1631
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :        94.52        99.00       146.57                 844
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         5.65         5.92         8.33                 839
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         6.87         7.16        10.40                 842
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       191.22       190.57       205.30                 838
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :         2.09         1.20        14.72                 838
 TCP upload BE4 fremont.starlink.taht.net                      :         6.18         6.95        10.18 Mbits/s        1631
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :        98.33       107.00       143.62                 839
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :         6.16         6.78         8.71                 834
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :         7.73         8.32        15.17                 836
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :       182.40       181.54       197.98                 833
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :         1.85         1.10        12.08                 833
 TCP upload avg                                                :         7.71          N/A          N/A Mbits/s        1631
 TCP upload fairness                                           :         0.95          N/A          N/A Mbits/s        1631
 TCP upload sum                                                :        30.84          N/A          N/A Mbits/s        1631
 cpu_stats_root@192.168.42.1::load                             :         0.39         0.40         0.90                1275

1 Like

In Download direction I see some degradation, upload however was/is a bit crummy tonight… I guess I will increase the ingress memlimit for the future… thanks.

P.S. Since the question came up, all tests via an ubuntu 20LTS host with on-board braodcom gigabit ethernet NIC. That host was controlled via ssh from a different wired computer.

1 Like

Well, darn it, singapore shouldn’t look like that. I’ll poke into it. thx for (inadvertently) testing out my cluster at these rtts!

1 Like

Still wanna test 1Gbps+, @Dave_Taht ?

very much so! Everything except singapore seem capable of it, but in part by spreading the load worldwide, we get better data and stress out the servers less. I have other boxes up (atlanta, mumbai) that are worth testing from europe, also.

1 Like

Cool. So I hope to get my colleagues to test it next week.

I don’t know much about the turris. Does the ethernet driver have BQL and fq_codel on? It’s the default in openwrt, but a lot of devices now-adays do offloads. ?

 tc -s qdisc show

So an rtt_fair result like this would be useful on a gbit-symmetric link for reference, then one with sqm on, taking a look at the cpu usage for both. IF you have working BQL, cake at line rate (no bandwidth parameter) would be interesting as well. After that tracing some cpu bottlenecks would start to be useful.

I’m pretty sure from looking at this data (so far) that the default openwrt memlimit for fq_codel and cake is too low for these rates - the linux default of 32MB we tested to 10Gbit on other networks, but we didn’t look hard enough at the long-rtt cases at the openwrt patched default of 4Mb.

1 Like

This is new tools for me and I do not have as deep knowledge as you guys but I made a couple of test runs on my Turris MOX to night.
I’m located in sweden and have a 1G uplink from MOX

labuser@lab1:~/flent$ flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H netperf-eu.bufferbloat.net -t rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX rtt_fair4be
Started Flent 1.3.2 using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-21T213249.605471.rtt_fair4be-SQM_fq_codel_simple_qos-upload850000_download850000_1Gfibre_MOX.flent.gz.
Summary of rtt_fair4be test run from 2022-04-21 19:32:49.605471
  Title: 'rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX'
                                                                          avg       median          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :        36.69        35.90 ms             1398
 Ping (ms) ICMP2 london.starlink.taht.net                      :        37.17        36.40 ms             1398
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :       258.17       257.00 ms             1398
 Ping (ms) ICMP4 netperf-eu.bufferbloat.net                    :         7.56         6.77 ms             1398
 Ping (ms) avg                                                 :        84.90          N/A ms             1398
 TCP download BE1 de.starlink.taht.net                         :        16.99        16.62 Mbits/s        1398
 TCP download BE2 london.starlink.taht.net                     :        16.03        11.66 Mbits/s        1398
 TCP download BE3 singapore.starlink.taht.net                  :        16.57         5.01 Mbits/s        1398
 TCP download BE4 netperf-eu.bufferbloat.net                   :       211.36       203.50 Mbits/s        1398
 TCP download avg                                              :        65.24          N/A Mbits/s        1398
 TCP download fairness                                         :         0.37          N/A Mbits/s        1398
 TCP download sum                                              :       260.95          N/A Mbits/s        1398
 TCP upload BE1 de.starlink.taht.net                           :        62.83        68.66 Mbits/s        1398
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :       205.25       190.00                 580
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :        62.35        57.85                 579
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :        78.38        72.13                 580
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :        36.91        36.38                 580
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :         0.78         0.45                 580
 TCP upload BE2 london.starlink.taht.net                       :        87.41        79.88 Mbits/s        1398
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :       292.67       245.50                 582
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :        87.65        75.07                 581
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :       109.57        92.84                 582
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :        37.51        36.85                 580
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :         0.67         0.38                 580
 TCP upload BE3 singapore.starlink.taht.net                    :        12.96        13.21 Mbits/s        1398
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :       293.20       276.00                 597
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :        13.01        12.28                 592
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :        16.73        15.10                 594
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       258.57       257.99                 592
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :         1.89         0.65                 592
 TCP upload BE4 netperf-eu.bufferbloat.net                     :       392.23       395.09 Mbits/s        1398
 TCP upload BE4 netperf-eu.bufferbloat.net::tcp_cwnd           :       359.41       357.00                 575
 TCP upload BE4 netperf-eu.bufferbloat.net::tcp_delivery_rate  :       387.82       386.62                 575
 TCP upload BE4 netperf-eu.bufferbloat.net::tcp_pacing_rate    :       511.19       513.45                 575
 TCP upload BE4 netperf-eu.bufferbloat.net::tcp_rtt            :        10.04         9.21                 575
 TCP upload BE4 netperf-eu.bufferbloat.net::tcp_rtt_var        :         0.31         0.21                 575
 TCP upload avg                                                :       138.86          N/A Mbits/s        1398
 TCP upload fairness                                           :         0.47          N/A Mbits/s        1398
 TCP upload sum                                                :       555.43          N/A Mbits/s        1398

1 Like

thank you very much for the new data. I figured things would deteriorate at higher rates, but not by that much. What happens with no sqm on at all?

  1. could you use fremont.starlink.taht.net rather than netperf-eu?

  2. I wish there was a way to post the flent.gz files also. As much as I like and prefer to look at the long form plot, the summary plots we get from flent-gui, especially the cdfs and totals plots are often useful.

1 Like

Glad I can help.
As your host set did not deliver anywhere near the throughput of >500 so that was why I used the netperf-eu host.
But I will rerun with your host set shortly.

1 Like

EDIT:not relevant test as I ran over wifi (not turris wifi, external AP)

labuser@lab1:~/flent$ flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX rtt_fair4be
Started Flent 1.3.2 using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-21T221617.611439.rtt_fair4be-SQM_fq_codel_simple_qos-upload850000_download850000_1Gfibre_MOX.flent.gz.

Summary of rtt_fair4be test run from 2022-04-21 20:16:17.611439
  Title: 'rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX'

                                                                          avg       median          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :       114.46       101.00 ms             1400
 Ping (ms) ICMP2 london.starlink.taht.net                      :       115.47       101.00 ms             1400
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :       336.37       322.00 ms             1400
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :       235.67       218.00 ms             1400
 Ping (ms) avg                                                 :       200.49          N/A ms             1400
 TCP download BE1 de.starlink.taht.net                         :        36.41        36.44 Mbits/s        1400
 TCP download BE2 london.starlink.taht.net                     :        19.74        19.26 Mbits/s        1400
 TCP download BE3 singapore.starlink.taht.net                  :        17.34        19.37 Mbits/s        1400
 TCP download BE4 fremont.starlink.taht.net                    :        15.33        18.21 Mbits/s        1400
 TCP download avg                                              :        22.20          N/A Mbits/s        1400
 TCP download fairness                                         :         0.88          N/A Mbits/s        1400
 TCP download sum                                              :        88.82          N/A Mbits/s        1400
 TCP upload BE1 de.starlink.taht.net                           :        54.35        55.88 Mbits/s        1400
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :       696.33       410.00                 543
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :        53.62        44.62                 542
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :        70.40        58.18                 542
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :       125.65       113.31                 542
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :         2.48         2.06                 542
 TCP upload BE2 london.starlink.taht.net                       :        43.88        49.09 Mbits/s        1400
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :       569.10       409.50                 546
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :        42.60        41.04                 545
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :        57.14        52.53                 545
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :       126.22       114.28                 545
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :         2.58         2.28                 545
 TCP upload BE3 singapore.starlink.taht.net                    :         5.18         5.35 Mbits/s        1400
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :       161.21       169.50                 546
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :         5.12         5.20                 540
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :         6.59         6.49                 543
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       362.46       350.33                 543
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :         7.26         4.55                 543
 TCP upload BE4 fremont.starlink.taht.net                      :        22.61        26.31 Mbits/s        1400
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :       466.21       326.00                 550
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :        22.56        19.09                 547
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :        29.20        23.31                 548
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :       246.76       234.06                 547
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :         4.63         3.02                 547
 TCP upload avg                                                :        31.51          N/A Mbits/s        1400
 TCP upload fairness                                           :         0.73          N/A Mbits/s        1400
 TCP upload sum                                                :       126.02          N/A Mbits/s        1400

yes, both fremont and singapores behaviors are puzzling. I just made a configuration change to fremont a second ago… this second run is worse than your first…

1 Like

hahah , sorry I screwed up, that test ran over wifi…
I will re run now

labuser@lab1:~/flent$ flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX rtt_fair4be
Started Flent 1.3.2 using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-21T222602.107032.rtt_fair4be-SQM_fq_codel_simple_qos-upload850000_download850000_1Gfibre_MOX.flent.gz.

Summary of rtt_fair4be test run from 2022-04-21 20:26:02.107032
  Title: 'rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX'

                                                                          avg       median          # data pts
 Ping (ms) ICMP1 de.starlink.taht.net                          :        36.36        35.70 ms             1400
 Ping (ms) ICMP2 london.starlink.taht.net                      :        36.86        36.20 ms             1400
 Ping (ms) ICMP3 singapore.starlink.taht.net                   :       257.95       257.00 ms             1400
 Ping (ms) ICMP4 fremont.starlink.taht.net                     :       158.58       158.00 ms             1400
 Ping (ms) avg                                                 :       122.44          N/A ms             1400
 TCP download BE1 de.starlink.taht.net                         :        39.70        37.43 Mbits/s        1400
 TCP download BE2 london.starlink.taht.net                     :        28.21        28.41 Mbits/s        1400
 TCP download BE3 singapore.starlink.taht.net                  :        24.21        13.81 Mbits/s        1400
 TCP download BE4 fremont.starlink.taht.net                    :        25.03         9.67 Mbits/s        1400
 TCP download avg                                              :        29.29          N/A Mbits/s        1400
 TCP download fairness                                         :         0.96          N/A Mbits/s        1400
 TCP download sum                                              :       117.15          N/A Mbits/s        1400
 TCP upload BE1 de.starlink.taht.net                           :        69.15        75.19 Mbits/s        1400
 TCP upload BE1 de.starlink.taht.net::tcp_cwnd                 :       226.32       223.00                 551
 TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate        :        68.84        68.83                 551
 TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate          :        87.20        85.05                 551
 TCP upload BE1 de.starlink.taht.net::tcp_rtt                  :        36.74        36.04                 551
 TCP upload BE1 de.starlink.taht.net::tcp_rtt_var              :         0.71         0.39                 551
 TCP upload BE2 london.starlink.taht.net                       :       121.83       120.08 Mbits/s        1400
 TCP upload BE2 london.starlink.taht.net::tcp_cwnd             :       409.68       379.00                 553
 TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate    :       121.81       112.94                 552
 TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate      :       154.13       140.49                 553
 TCP upload BE2 london.starlink.taht.net::tcp_rtt              :        37.62        37.01                 553
 TCP upload BE2 london.starlink.taht.net::tcp_rtt_var          :         0.56         0.32                 553
 TCP upload BE3 singapore.starlink.taht.net                    :        50.93        50.07 Mbits/s        1400
 TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd          :      1154.41      1206.00                 551
 TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate :        50.69        53.63                 546
 TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate   :        64.79        66.53                 548
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt           :       259.02       258.17                 546
 TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var       :         1.67         0.51                 546
 TCP upload BE4 fremont.starlink.taht.net                      :        53.18        49.56 Mbits/s        1400
 TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd            :       805.76       660.50                 554
 TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate   :        53.38        47.58                 551
 TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate     :        78.91        58.86                 553
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt             :       159.55       158.46                 552
 TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var         :         1.31         0.48                 552
 TCP upload avg                                                :        73.77          N/A Mbits/s        1400
 TCP upload fairness                                           :         0.87          N/A Mbits/s        1400
 TCP upload sum                                                :       295.09          N/A Mbits/s        1400

1 Like

one of my goals is indeed to get good reference results for the turris wifi, also. could you clearly mark in the title what is under test? is this with or without sqm?

1 Like