Will do tonight.
No, with only a mild 100/36 asymmetry and a nominally non-bursty link I leave ACKs as they are.
Can do tonight once everybody else stopped using the network.
1 Like
No SQM:
moeller@work-horse:~/CODE/flent$ ./run-flent -x --ipv4 -l 120 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net rtt_fair4be --remote-metadata=root@192.168.42.1 --test-parameter=cpu_stats_hosts=root@192.168.42.1 --step-size=.05 --socket-stats -D . -t ingress_IPv4_NoSQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_netperf-eu.bufferbloat.net --log-file
Starting Flent 2.0.1-git-65cd64f using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 130 seconds.
Data file written to ./rtt_fair4be-2022-04-19T231920.248480.ingress_IPv4_NoSQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS_5_3_7-pppoe-wan-eth2_7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDS.flent.gz
Summary of rtt_fair4be test run from 2022-04-19 21:19:20.248480
Title: 'ingress_IPv4_NoSQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_netperf-eu.bufferbloat.net'
avg median 99th % # data pts
Ping (ms) ICMP1 de.starlink.taht.net : 36.60 38.20 53.82 ms 2931
Ping (ms) ICMP2 london.starlink.taht.net : 47.60 49.20 65.02 ms 2931
Ping (ms) ICMP3 singapore.starlink.taht.net : 190.00 191.00 207.00 ms 2931
Ping (ms) ICMP4 fremont.starlink.taht.net : 191.00 192.00 209.00 ms 2931
Ping (ms) avg : 116.30 N/A N/A ms 2931
TCP download BE1 de.starlink.taht.net : 44.94 48.09 65.22 Mbits/s 2931
TCP download BE2 london.starlink.taht.net : 25.23 27.17 39.78 Mbits/s 2931
TCP download BE3 singapore.starlink.taht.net : 19.34 16.23 69.87 Mbits/s 2931
TCP download BE4 fremont.starlink.taht.net : 13.39 11.72 41.27 Mbits/s 2931
TCP download avg : 25.73 N/A N/A Mbits/s 2931
TCP download fairness : 0.82 N/A N/A Mbits/s 2931
TCP download sum : 102.90 N/A N/A Mbits/s 2931
TCP upload BE1 de.starlink.taht.net : 14.08 14.46 20.12 Mbits/s 2931
TCP upload BE1 de.starlink.taht.net::tcp_cwnd : 48.29 49.00 75.00 1695
TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : 13.79 13.80 19.03 1694
TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : 17.74 17.36 33.62 1695
TCP upload BE1 de.starlink.taht.net::tcp_rtt : 39.24 40.31 54.74 1692
TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : 0.93 0.74 4.86 1692
TCP upload BE2 london.starlink.taht.net : 9.49 9.66 15.08 Mbits/s 2931
TCP upload BE2 london.starlink.taht.net::tcp_cwnd : 42.11 40.00 77.00 1697
TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : 9.32 9.19 14.39 1696
TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : 11.90 11.59 21.32 1696
TCP upload BE2 london.starlink.taht.net::tcp_rtt : 50.27 51.27 65.51 1693
TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : 1.25 1.01 4.98 1693
TCP upload BE3 singapore.starlink.taht.net : 2.50 2.58 5.48 Mbits/s 2931
TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : 44.55 40.00 78.00 1720
TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : 2.49 2.23 4.22 1715
TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : 3.19 2.79 6.39 1718
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : 201.28 202.55 219.12 1714
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : 3.43 2.87 10.46 1714
TCP upload BE4 fremont.starlink.taht.net : 2.30 2.33 5.38 Mbits/s 2931
TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : 44.15 41.00 82.00 1727
TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : 2.29 2.09 4.14 1721
TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : 2.95 2.62 6.16 1724
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : 217.27 218.09 237.47 1722
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : 3.66 3.02 10.61 1722
TCP upload avg : 7.09 N/A N/A Mbits/s 2931
TCP upload fairness : 0.67 N/A N/A Mbits/s 2931
TCP upload sum : 28.37 N/A N/A Mbits/s 2931
cpu_stats_root@192.168.42.1::load : 0.26 0.25 0.67 2292
Note what My ISP does to my latency under load, and how much worst this looks compared to the competent AQM/traffic shaper that is cake.
2 Likes
Base on that last result… your ISP is actually doing a pretty good job at defaults. Most end-users (not gamers) would be perfectly content with only the 60ms latency and jitter you are showing, as it’s kilometers better than most dsl and cable. But since we both have OCD and can do much better for all uses and users and all workloads, we sally forth, with cake.
Thx VERY much for testing this on the turris for me. I’m still hoping other turris users running at > 500Mbit can chip in, which was my original question!
Also thx for validating my new cloud!
1 Like
in terms of moving variables around, could you re-test with cake 100mbit with the default memlimit? I’m pretty sure at this point that its too low from various other tests, but…
1 Like
Yes they have come a long way… this is a VDSL2 link with bidirectional vectoring and G.INP retransmissions. But I am using my own modem and router so can not fully appreciate the experience of the bulk of my ISPs customers who likely use the all in one modem router offered by the ISP.
Again, yes. Since it is not that hard to overload/saturate my link I really appreciate the IP and flow isolation cake’s scheduler offers…
+1; over here the plan now is to convert the bulk of the links to FTTH by 2030, though that is more aspirational than reliable. So I guess if nobody beats me to it, I might be able to post such a result within the next 8 years 
My pleasure, thanks for sharing.
I can and will do tonight, but when I had a look at cake’s reported memory usage (by manually calling tc -s qdisc on a second terminal during the test) it did not seem to get close to the default 4MB.
1 Like
Reference memlimit 32mb 105/36
run-flent -x --ipv4 -l 60 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net rtt_fair4be --remote-metadata=root@192.168.42.1 --test-parameter=cpu_stats_hosts=root@192.168.42.1 --step-size=.05 --socket-stats -D . -t memlimit32mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_xxx.starlink.taht.net --log-file
Starting Flent 2.0.1-git-65cd64f using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-21T203810.994215.memlimit32mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS_5_3_7-pppoe-wan-eth2_7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-.flent.gz
Summary of rtt_fair4be test run from 2022-04-21 18:38:10.994215
Title: 'memlimit32mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_xxx.starlink.taht.net'
avg median 99th % # data pts
Ping (ms) ICMP1 de.starlink.taht.net : 19.30 19.10 26.61 ms 1604
Ping (ms) ICMP2 london.starlink.taht.net : 31.60 31.40 37.70 ms 1604
Ping (ms) ICMP3 singapore.starlink.taht.net : 172.00 172.00 179.00 ms 1604
Ping (ms) ICMP4 fremont.starlink.taht.net : 183.00 182.00 189.00 ms 1604
Ping (ms) avg : 101.47 N/A N/A ms 1604
TCP download BE1 de.starlink.taht.net : 37.21 36.22 56.90 Mbits/s 1604
TCP download BE2 london.starlink.taht.net : 34.12 34.30 46.01 Mbits/s 1604
TCP download BE3 singapore.starlink.taht.net : 3.95 4.29 7.09 Mbits/s 1604
TCP download BE4 fremont.starlink.taht.net : 19.67 21.34 34.80 Mbits/s 1604
TCP download avg : 23.74 N/A N/A Mbits/s 1604
TCP download fairness : 0.76 N/A N/A Mbits/s 1604
TCP download sum : 94.95 N/A N/A Mbits/s 1604
TCP upload BE1 de.starlink.taht.net : 9.73 9.59 22.44 Mbits/s 1604
TCP upload BE1 de.starlink.taht.net::tcp_cwnd : 22.61 22.00 39.00 854
TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : 9.30 8.70 15.50 853
TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : 12.06 11.39 20.40 853
TCP upload BE1 de.starlink.taht.net::tcp_rtt : 26.31 25.80 36.13 851
TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : 1.47 1.27 4.38 851
TCP upload BE2 london.starlink.taht.net : 9.30 9.11 16.58 Mbits/s 1604
TCP upload BE2 london.starlink.taht.net::tcp_cwnd : 30.45 29.00 53.00 852
TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : 8.98 8.44 14.59 851
TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : 11.70 10.91 20.25 851
TCP upload BE2 london.starlink.taht.net::tcp_rtt : 37.27 36.49 46.68 848
TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : 1.26 1.03 4.46 848
TCP upload BE3 singapore.starlink.taht.net : 5.74 6.37 10.10 Mbits/s 1604
TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : 92.40 97.00 140.42 859
TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : 5.73 6.02 8.41 854
TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : 7.00 7.32 10.68 857
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : 184.02 183.31 200.00 855
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : 2.11 1.22 12.11 855
TCP upload BE4 fremont.starlink.taht.net : 5.90 6.73 9.52 Mbits/s 1604
TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : 95.01 103.00 137.00 857
TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : 5.89 6.46 8.49 852
TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : 7.36 7.97 14.42 854
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : 184.54 183.87 197.84 851
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : 1.84 1.10 11.00 851
TCP upload avg : 7.67 N/A N/A Mbits/s 1604
TCP upload fairness : 0.94 N/A N/A Mbits/s 1604
TCP upload sum : 30.67 N/A N/A Mbits/s 1604
cpu_stats_root@192.168.42.1::load : 0.36 0.33 0.79 1248
And now with the OpenWrt default of 4mb per direction:
user@work-horse:~/CODE/flent$ ./run-flent -x --ipv4 -l 60 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net rtt_fair4be --remote-metadata=root@192.168.42.1 --test-parameter=cpu_stats_hosts=root@192.168.42.1 --step-size=.05 --socket-stats -D . -t memlimit4mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_xxx.starlink.taht.net --log-file
Starting Flent 2.0.1-git-65cd64f using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-21T205112.644552.memlimit4mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS_5_3_7-pppoe-wan-eth2_7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-V.flent.gz
Summary of rtt_fair4be test run from 2022-04-21 18:51:12.644552
Title: 'memlimit4mb_IPv4_SQM_cake_layer-cake_LLA-ETH_OH34_U097pct36000of36998K-D090pct105000of116797K_work-horse-eth0_2_TurrisOmnia-TurrisOS.5.3.7-pppoe-wan-eth2.7_2_bridged-BTHH5A-OpenWrt-r18441-ba7cee05ed-Hvt-VDSL100_2_xxx.starlink.taht.net'
avg median 99th % # data pts
Ping (ms) ICMP1 de.starlink.taht.net : 19.30 18.90 25.60 ms 1631
Ping (ms) ICMP2 london.starlink.taht.net : 30.90 30.60 37.60 ms 1631
Ping (ms) ICMP3 singapore.starlink.taht.net : 172.00 172.00 178.00 ms 1631
Ping (ms) ICMP4 fremont.starlink.taht.net : 183.00 182.00 189.00 ms 1631
Ping (ms) avg : 101.30 N/A N/A ms 1631
TCP download BE1 de.starlink.taht.net : 37.17 36.36 55.30 Mbits/s 1631
TCP download BE2 london.starlink.taht.net : 33.21 33.68 48.78 Mbits/s 1631
TCP download BE3 singapore.starlink.taht.net : 4.13 4.30 7.42 Mbits/s 1631
TCP download BE4 fremont.starlink.taht.net : 19.49 19.94 42.91 Mbits/s 1631
TCP download avg : 23.50 N/A N/A Mbits/s 1631
TCP download fairness : 0.77 N/A N/A Mbits/s 1631
TCP download sum : 94.00 N/A N/A Mbits/s 1631
TCP upload BE1 de.starlink.taht.net : 9.68 9.23 18.78 Mbits/s 1631
TCP upload BE1 de.starlink.taht.net::tcp_cwnd : 22.50 21.00 39.73 828
TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : 9.23 8.69 15.79 828
TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : 11.92 11.31 21.14 828
TCP upload BE1 de.starlink.taht.net::tcp_rtt : 26.36 25.79 36.15 826
TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : 1.38 1.16 4.69 826
TCP upload BE2 london.starlink.taht.net : 9.32 8.98 19.14 Mbits/s 1631
TCP upload BE2 london.starlink.taht.net::tcp_cwnd : 30.38 29.00 49.00 831
TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : 9.02 8.54 14.58 830
TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : 11.62 10.91 19.67 830
TCP upload BE2 london.starlink.taht.net::tcp_rtt : 37.16 36.64 47.22 829
TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : 1.23 0.99 4.54 829
TCP upload BE3 singapore.starlink.taht.net : 5.66 6.28 9.91 Mbits/s 1631
TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : 94.52 99.00 146.57 844
TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : 5.65 5.92 8.33 839
TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : 6.87 7.16 10.40 842
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : 191.22 190.57 205.30 838
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : 2.09 1.20 14.72 838
TCP upload BE4 fremont.starlink.taht.net : 6.18 6.95 10.18 Mbits/s 1631
TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : 98.33 107.00 143.62 839
TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : 6.16 6.78 8.71 834
TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : 7.73 8.32 15.17 836
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : 182.40 181.54 197.98 833
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : 1.85 1.10 12.08 833
TCP upload avg : 7.71 N/A N/A Mbits/s 1631
TCP upload fairness : 0.95 N/A N/A Mbits/s 1631
TCP upload sum : 30.84 N/A N/A Mbits/s 1631
cpu_stats_root@192.168.42.1::load : 0.39 0.40 0.90 1275
1 Like
In Download direction I see some degradation, upload however was/is a bit crummy tonight… I guess I will increase the ingress memlimit for the future… thanks.
P.S. Since the question came up, all tests via an ubuntu 20LTS host with on-board braodcom gigabit ethernet NIC. That host was controlled via ssh from a different wired computer.
1 Like
Well, darn it, singapore shouldn’t look like that. I’ll poke into it. thx for (inadvertently) testing out my cluster at these rtts!
1 Like
very much so! Everything except singapore seem capable of it, but in part by spreading the load worldwide, we get better data and stress out the servers less. I have other boxes up (atlanta, mumbai) that are worth testing from europe, also.
1 Like
Cool. So I hope to get my colleagues to test it next week.
I don’t know much about the turris. Does the ethernet driver have BQL and fq_codel on? It’s the default in openwrt, but a lot of devices now-adays do offloads. ?
tc -s qdisc show
So an rtt_fair result like this would be useful on a gbit-symmetric link for reference, then one with sqm on, taking a look at the cpu usage for both. IF you have working BQL, cake at line rate (no bandwidth parameter) would be interesting as well. After that tracing some cpu bottlenecks would start to be useful.
I’m pretty sure from looking at this data (so far) that the default openwrt memlimit for fq_codel and cake is too low for these rates - the linux default of 32MB we tested to 10Gbit on other networks, but we didn’t look hard enough at the long-rtt cases at the openwrt patched default of 4Mb.
1 Like
This is new tools for me and I do not have as deep knowledge as you guys but I made a couple of test runs on my Turris MOX to night.
I’m located in sweden and have a 1G uplink from MOX
labuser@lab1:~/flent$ flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H netperf-eu.bufferbloat.net -t rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX rtt_fair4be
Started Flent 1.3.2 using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-21T213249.605471.rtt_fair4be-SQM_fq_codel_simple_qos-upload850000_download850000_1Gfibre_MOX.flent.gz.
Summary of rtt_fair4be test run from 2022-04-21 19:32:49.605471
Title: 'rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX'
avg median # data pts
Ping (ms) ICMP1 de.starlink.taht.net : 36.69 35.90 ms 1398
Ping (ms) ICMP2 london.starlink.taht.net : 37.17 36.40 ms 1398
Ping (ms) ICMP3 singapore.starlink.taht.net : 258.17 257.00 ms 1398
Ping (ms) ICMP4 netperf-eu.bufferbloat.net : 7.56 6.77 ms 1398
Ping (ms) avg : 84.90 N/A ms 1398
TCP download BE1 de.starlink.taht.net : 16.99 16.62 Mbits/s 1398
TCP download BE2 london.starlink.taht.net : 16.03 11.66 Mbits/s 1398
TCP download BE3 singapore.starlink.taht.net : 16.57 5.01 Mbits/s 1398
TCP download BE4 netperf-eu.bufferbloat.net : 211.36 203.50 Mbits/s 1398
TCP download avg : 65.24 N/A Mbits/s 1398
TCP download fairness : 0.37 N/A Mbits/s 1398
TCP download sum : 260.95 N/A Mbits/s 1398
TCP upload BE1 de.starlink.taht.net : 62.83 68.66 Mbits/s 1398
TCP upload BE1 de.starlink.taht.net::tcp_cwnd : 205.25 190.00 580
TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : 62.35 57.85 579
TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : 78.38 72.13 580
TCP upload BE1 de.starlink.taht.net::tcp_rtt : 36.91 36.38 580
TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : 0.78 0.45 580
TCP upload BE2 london.starlink.taht.net : 87.41 79.88 Mbits/s 1398
TCP upload BE2 london.starlink.taht.net::tcp_cwnd : 292.67 245.50 582
TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : 87.65 75.07 581
TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : 109.57 92.84 582
TCP upload BE2 london.starlink.taht.net::tcp_rtt : 37.51 36.85 580
TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : 0.67 0.38 580
TCP upload BE3 singapore.starlink.taht.net : 12.96 13.21 Mbits/s 1398
TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : 293.20 276.00 597
TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : 13.01 12.28 592
TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : 16.73 15.10 594
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : 258.57 257.99 592
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : 1.89 0.65 592
TCP upload BE4 netperf-eu.bufferbloat.net : 392.23 395.09 Mbits/s 1398
TCP upload BE4 netperf-eu.bufferbloat.net::tcp_cwnd : 359.41 357.00 575
TCP upload BE4 netperf-eu.bufferbloat.net::tcp_delivery_rate : 387.82 386.62 575
TCP upload BE4 netperf-eu.bufferbloat.net::tcp_pacing_rate : 511.19 513.45 575
TCP upload BE4 netperf-eu.bufferbloat.net::tcp_rtt : 10.04 9.21 575
TCP upload BE4 netperf-eu.bufferbloat.net::tcp_rtt_var : 0.31 0.21 575
TCP upload avg : 138.86 N/A Mbits/s 1398
TCP upload fairness : 0.47 N/A Mbits/s 1398
TCP upload sum : 555.43 N/A Mbits/s 1398
1 Like
thank you very much for the new data. I figured things would deteriorate at higher rates, but not by that much. What happens with no sqm on at all?
-
could you use fremont.starlink.taht.net rather than netperf-eu?
-
I wish there was a way to post the flent.gz files also. As much as I like and prefer to look at the long form plot, the summary plots we get from flent-gui, especially the cdfs and totals plots are often useful.
1 Like
Glad I can help.
As your host set did not deliver anywhere near the throughput of >500 so that was why I used the netperf-eu host.
But I will rerun with your host set shortly.
1 Like
EDIT:not relevant test as I ran over wifi (not turris wifi, external AP)
labuser@lab1:~/flent$ flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX rtt_fair4be
Started Flent 1.3.2 using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-21T221617.611439.rtt_fair4be-SQM_fq_codel_simple_qos-upload850000_download850000_1Gfibre_MOX.flent.gz.
Summary of rtt_fair4be test run from 2022-04-21 20:16:17.611439
Title: 'rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX'
avg median # data pts
Ping (ms) ICMP1 de.starlink.taht.net : 114.46 101.00 ms 1400
Ping (ms) ICMP2 london.starlink.taht.net : 115.47 101.00 ms 1400
Ping (ms) ICMP3 singapore.starlink.taht.net : 336.37 322.00 ms 1400
Ping (ms) ICMP4 fremont.starlink.taht.net : 235.67 218.00 ms 1400
Ping (ms) avg : 200.49 N/A ms 1400
TCP download BE1 de.starlink.taht.net : 36.41 36.44 Mbits/s 1400
TCP download BE2 london.starlink.taht.net : 19.74 19.26 Mbits/s 1400
TCP download BE3 singapore.starlink.taht.net : 17.34 19.37 Mbits/s 1400
TCP download BE4 fremont.starlink.taht.net : 15.33 18.21 Mbits/s 1400
TCP download avg : 22.20 N/A Mbits/s 1400
TCP download fairness : 0.88 N/A Mbits/s 1400
TCP download sum : 88.82 N/A Mbits/s 1400
TCP upload BE1 de.starlink.taht.net : 54.35 55.88 Mbits/s 1400
TCP upload BE1 de.starlink.taht.net::tcp_cwnd : 696.33 410.00 543
TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : 53.62 44.62 542
TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : 70.40 58.18 542
TCP upload BE1 de.starlink.taht.net::tcp_rtt : 125.65 113.31 542
TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : 2.48 2.06 542
TCP upload BE2 london.starlink.taht.net : 43.88 49.09 Mbits/s 1400
TCP upload BE2 london.starlink.taht.net::tcp_cwnd : 569.10 409.50 546
TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : 42.60 41.04 545
TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : 57.14 52.53 545
TCP upload BE2 london.starlink.taht.net::tcp_rtt : 126.22 114.28 545
TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : 2.58 2.28 545
TCP upload BE3 singapore.starlink.taht.net : 5.18 5.35 Mbits/s 1400
TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : 161.21 169.50 546
TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : 5.12 5.20 540
TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : 6.59 6.49 543
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : 362.46 350.33 543
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : 7.26 4.55 543
TCP upload BE4 fremont.starlink.taht.net : 22.61 26.31 Mbits/s 1400
TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : 466.21 326.00 550
TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : 22.56 19.09 547
TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : 29.20 23.31 548
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : 246.76 234.06 547
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : 4.63 3.02 547
TCP upload avg : 31.51 N/A Mbits/s 1400
TCP upload fairness : 0.73 N/A Mbits/s 1400
TCP upload sum : 126.02 N/A Mbits/s 1400yes, both fremont and singapores behaviors are puzzling. I just made a configuration change to fremont a second ago… this second run is worse than your first…
1 Like
hahah , sorry I screwed up, that test ran over wifi…
I will re run now
labuser@lab1:~/flent$ flent -x --socket-stats --step-size=.05 -H de.starlink.taht.net -H london.starlink.taht.net -H singapore.starlink.taht.net -H fremont.starlink.taht.net -t rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX rtt_fair4be
Started Flent 1.3.2 using Python 3.8.10.
Starting rtt_fair4be test. Expected run time: 70 seconds.
Data file written to ./rtt_fair4be-2022-04-21T222602.107032.rtt_fair4be-SQM_fq_codel_simple_qos-upload850000_download850000_1Gfibre_MOX.flent.gz.
Summary of rtt_fair4be test run from 2022-04-21 20:26:02.107032
Title: 'rtt_fair4be-SQM_fq_codel_simple.qos-upload850000_download850000_1Gfibre_MOX'
avg median # data pts
Ping (ms) ICMP1 de.starlink.taht.net : 36.36 35.70 ms 1400
Ping (ms) ICMP2 london.starlink.taht.net : 36.86 36.20 ms 1400
Ping (ms) ICMP3 singapore.starlink.taht.net : 257.95 257.00 ms 1400
Ping (ms) ICMP4 fremont.starlink.taht.net : 158.58 158.00 ms 1400
Ping (ms) avg : 122.44 N/A ms 1400
TCP download BE1 de.starlink.taht.net : 39.70 37.43 Mbits/s 1400
TCP download BE2 london.starlink.taht.net : 28.21 28.41 Mbits/s 1400
TCP download BE3 singapore.starlink.taht.net : 24.21 13.81 Mbits/s 1400
TCP download BE4 fremont.starlink.taht.net : 25.03 9.67 Mbits/s 1400
TCP download avg : 29.29 N/A Mbits/s 1400
TCP download fairness : 0.96 N/A Mbits/s 1400
TCP download sum : 117.15 N/A Mbits/s 1400
TCP upload BE1 de.starlink.taht.net : 69.15 75.19 Mbits/s 1400
TCP upload BE1 de.starlink.taht.net::tcp_cwnd : 226.32 223.00 551
TCP upload BE1 de.starlink.taht.net::tcp_delivery_rate : 68.84 68.83 551
TCP upload BE1 de.starlink.taht.net::tcp_pacing_rate : 87.20 85.05 551
TCP upload BE1 de.starlink.taht.net::tcp_rtt : 36.74 36.04 551
TCP upload BE1 de.starlink.taht.net::tcp_rtt_var : 0.71 0.39 551
TCP upload BE2 london.starlink.taht.net : 121.83 120.08 Mbits/s 1400
TCP upload BE2 london.starlink.taht.net::tcp_cwnd : 409.68 379.00 553
TCP upload BE2 london.starlink.taht.net::tcp_delivery_rate : 121.81 112.94 552
TCP upload BE2 london.starlink.taht.net::tcp_pacing_rate : 154.13 140.49 553
TCP upload BE2 london.starlink.taht.net::tcp_rtt : 37.62 37.01 553
TCP upload BE2 london.starlink.taht.net::tcp_rtt_var : 0.56 0.32 553
TCP upload BE3 singapore.starlink.taht.net : 50.93 50.07 Mbits/s 1400
TCP upload BE3 singapore.starlink.taht.net::tcp_cwnd : 1154.41 1206.00 551
TCP upload BE3 singapore.starlink.taht.net::tcp_delivery_rate : 50.69 53.63 546
TCP upload BE3 singapore.starlink.taht.net::tcp_pacing_rate : 64.79 66.53 548
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt : 259.02 258.17 546
TCP upload BE3 singapore.starlink.taht.net::tcp_rtt_var : 1.67 0.51 546
TCP upload BE4 fremont.starlink.taht.net : 53.18 49.56 Mbits/s 1400
TCP upload BE4 fremont.starlink.taht.net::tcp_cwnd : 805.76 660.50 554
TCP upload BE4 fremont.starlink.taht.net::tcp_delivery_rate : 53.38 47.58 551
TCP upload BE4 fremont.starlink.taht.net::tcp_pacing_rate : 78.91 58.86 553
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt : 159.55 158.46 552
TCP upload BE4 fremont.starlink.taht.net::tcp_rtt_var : 1.31 0.48 552
TCP upload avg : 73.77 N/A Mbits/s 1400
TCP upload fairness : 0.87 N/A Mbits/s 1400
TCP upload sum : 295.09 N/A Mbits/s 1400
1 Like
one of my goals is indeed to get good reference results for the turris wifi, also. could you clearly mark in the title what is under test? is this with or without sqm?
1 Like