Turris limited network throughput

Thanks for the suggestions. Yes, all tests are on TP cable eth ports, no wifi.

Speed btween M and T is same in both directons, I’ve also did a test binding iperf sever on WAN port IP address and the spped was the same.

One of the clients I’ve used for testing is connected on very cable so it limits the test to 100Mbps, what if fine concerning the internet connection speed, but to make sure it is reproducible, I’ll do a test using notebook and cat5e or cat6 cable to verify the speed (I did it allready [963Mbps in both directions] but only between one of the devices and notebook, so having a test with brand new short cables might be better for check).

I also check the requeues before and after a speed test and all the figures remains same as they were before the speed test.

  • is the T still running TOS3.x or TOS > 4.x yet?
  • if still on TOS3.x would it be feasible to test with TOS4.x|5.x?
  • with the change of the ISP did the type of WAN connectivity change (e.g. ADSL to VDSL)?
  • did the ISP change the CPE (modem)?
  • what type/brand of CPE is deployed (e.g. modem only or some modem-router combo)?
  • if it is a modem-router combo is set to bridge mode?

Recap: (my apology if I haven’t read the previous one carefully)

  1. the problem when you test download is not on routers ?

  2. Manifests itself on clients connected via UTP - it is one client or has been tested by multiple clients in ropes ?

  3. What web browser is used for test ? Isn’t it Opera and active VPN ?

  4. Test in browser - what test is used what is the localization of the counterparty of the test — it is not the USA :slight_smile: ?

  5. No one specific message in log ?

  • /etc/turris-version: 3.11.16
  • Upgrade - yes, but I have to re-configure all settings in that case (but can move to BTRFS with snapshots…)
  • Former ISP was 5GHz (wifi), new one is 5G
  • Yes, I was connected to a switch, now it is ISP router
  • modem-router is deployed, I’ve tested both NAT and bridge mode, same results
  1. my turris as router (his netmeter) is the fast one (100-120Mbps), while any other device (Mox as sipmle local network host or any notebook [connected on cable]) download is limited to somewhat 23Mbsp.
  2. I’ve tested both singe client on UTP.
  3. Firefox and Cromium/chrome (on Ferdora and Win10)
  4. netmeter.cz is what I use for tests, I can compare it with netmeter results from turris 1.0 (router) and with Mox (ordinary client).

I understand … The problem with testing the connection speed is that both sides of the test are imbezing it. You tried another web speed test too … for the relevance of the results ?.

www.speedtest.ro
www.netmeter.cz
www.speedtest.net … or app Speedtest

Let’s see if the results are similar. As a possible problem, i can still think of qos limits.

Just to make it clear, I have no issue with ISP speed, only when using it through turris, once connected directly to IP eth switch, all the speed test are prefectly OK accoring to service parameters declared by provider. My problem is that when the test is executed on turris itself, the spped is OK, once I try to run the test using any client, it is something like 20-23Mbps.

I’ve tried to compose a drawing for tests I’ve done:


On 1Gbps LAN I’ll be happy with whatever speed from 900Mbps, but some of the tests are far from that. At least, running the tests once more, I’ve realized that my initial test with a client downloading data at 20Mbps instead of ~ 100Mbps was some failure on my side (cable…). I’ve used a set of new Cat 5e cables and this seems to resolve one part of my problems, but the results of many other tests are curious to me.
By the way, it looks my MOX is not as good at the network as what I’d expect since same performance figures I was able to observe while connection to the MOX directly from notebook switch and MOX as a client is very slow, but this is a different story, I’m interested in having my Turris in a good shape first.
Thanks for any comments.

1 Like

Sounds like a reasonable expectation, if the T clients (M & Notebook A) are both connected to T’s LAN ports respectively and with no VLAN tagging or packet mirroring involved (presumably the case).

The data-sheet for the QCA8337N switch chip (in the T) though mentions features of its switch engine (that could come into play):

  • QoS engine
  • Bandwidth control
  • Queue manager

some of which parameters are settable via registers.

With 280 M/s the M as iperf client (via T) this could be attributed to the M as it appears to showing also in this scenario.

If feasible replace the M as as iperf client (via T) with another able/suitable desktop/laptop instance, just to rule out the M.

If you click on the picture, there are 10 tests described, the preview shows only fef of them, so I’ve already did the MOX replacement test.

1 Like

Missed that, looks like a thorough testing and documentation.

  • test 4 & 5 indicate the T’s switch chip not being the cause and performing within reasonable parameters
  • test 9 & 10 indicate the T’s WAN port and CPU not being the cause and performing within reasonable parameters on direct ingress/egress
  • test 8 indicates the ISP’s CPE probably not being a the cause either (on frame forwarding)

From the remainder (test 6 & 7) it seems that the T is suffering some issue with packet processing on ingress.

Is PAKON installed/running on the T? Is was mentioned in other threads as cause for throughput issues.

1 Like

Is PAKON installed/running on the T? Is was mentioned in other threads as cause for throughput issues.

Yes PAKON is installed and active, I can deinstall it for testing, it is not critical for me.

Will do the tests later since now the VPN is essential and I have to move physically to Turris to avoid any cable issues…

Thanks for the sugestion!

Yes try this, PAKON will need to touch and process each package passing your router beyong what the network stack usually does. PAKON is great, but like most nice tings it does not come for free, in this case it probably requires more CPU cycles than your router can spare, and once that happens everything slows down.

I’ve unistalled PAKON and tested using a short (2m) cable and clitent PC works well. Sruprisingly, installing PAKON back did not make it any visibly slower, downloads were still a bit above 100 Mbps.
Might it be, that PAKON needed to be reinstalled. The remaining test is to use MOX as client and if it’ll work I had to admit that it works now well, but for no particlular know reason…

tbh: You might be right. ( Performance degradation with pakon or device discovery ). And maybe it is not just “pakon” maybe it is all about suricata (after that split to suricata-bin) and related packages/services (pakon, ludus, ids, suricata itself) where you need to remove it and install again. Personally i am not using ids or/and pakon at all. I am having only Ludus. I am suspecting that each package has yaml and it was called by service to load such config (and outdated/obsolete ones might cause issues we faced)

Weird things happen, my result/lessons learned is - do not use Turris 1.0 or Mox as an iperf server nor (definitely not) client.
Being frustrated from all the tests, I’ve replaced RJ45 connectors one by one… and end up with:

  • even on 1m long factory made Cat5e cable (including connectors), maximum throughput was 246Mbps (Turris as a client, notebook as an iperf server)
  • even on 1m long factory made Cat5e cable, maximum throughput was 272Mbps (Turris as an Iperf server, notebook as an iperf client)
  • Omnia (20m Cat5e cable) as client, Turris 1.0 as server 612Mbps
  • Turris 1.0 as a client (20m Cat5e cable), Omina as an iperf server 241Mbps

On the other hand when I used two notebooks, one connected using the above factory made 1m Cat5E cable to Turris 1.0 and second one was after 20m Cate5E cable a switch a another 15m of Cat5E cable, the network throughput was 855/905Mbps (changing client/server notebook roles).

Same tests (20m cable, switch) between a notebook and Omnia I’ve got 941/934Mbps.

The sad part of this testing is that Turris MOX is far from being as good as Omnia router, because
Mox an Iperf server and Omnia as an Iperf client (swicth 20m of cable) looks promising 919Mbps, while Omnia as an Iperf server to Mox an Iperf client I’ve got only 310Mbps.

Pakon was de-instaled on all the used routers, Iperf2 was used for tests.

My summary:
Part of the issue was probably caused by a bad RJ45 connector (since after replacing them, I was able to reach expected network throughputs) and long time used Pakon might contribute to the speed degradation as well since I haven’t installed it after replacing cable connectors and some improvement was achieved even before cable tests just by de-installing Pakon.
Turris 1.0 and MOX are not good devices to run iperf for any tests, but the switch chip in Turris 1.0 is capable to handle 1Gbps.

2 Likes

Far as I understand CAT5e is unshielded and max bandwidth of 100 MHz.

Could be interesting to see whether:

  • Cat 6 shielded (>250 MHz)
  • Cat 6a shielded (500 MHz)
  • Cat 7 (600 MHz)
  • CAT 7a (1,000 MHz)
  • CAT 8.1 (2,000 MHz)

would make a difference, particularly considering unshielded cabling over distances of 20 m.

Also it would be interesting whether the MOX throughput performance improves with the 5.4.x Linux kernel (HBD branch)

Cable Category 5e is the first one supporting gigabit (that not apply to Category 5) WIKI page. IN my case, the 5e cables are already shielded version I know Cat6 has also different mechanical construction, mandatory shielding and different connectors.

I would not change branch on MOX since once I did it last time, it breaks NextCloud certificates (or something else, but existing clients were rejecting connect to it). Since NextCloud is a very bad piece of software and I’ve bought MOX just to be able to abandon DropBox, but it is soo bad I can’t do that (for example, it is possible to upload more than 1 GB video file from a mobile client - it is able to upload files in chunks, but any client I’ve found - tested on Windows and on Linux is not able to download the file in chunks and the NextCloud server tries to load the WHOLE file into memory before sending it to a client - the result on MOX is OOM and reboot, but that is a different story). So if there is someone not suffering from NextCloud open to run a test the dev branch, it’ll be interesting.

Switching to HBD is probably more an issue with potential downtime. One could switch to HBD and test with iperf and afterwards rollback via schnapps to the working snapshot.


When you experienced the low throughput, assuming packets went through the CPU, did you happen to monitor packet statistics (drops, errors, overruns) on the involved interfaces?

Using snapshot for HBD is right point!

Regarding your second question, I did not collected anything more than several tc outputs collected in this discussion earlier. From my point of view it works now (almost gigabit can be reached). I can consider installing Pakon back to the stack to see if it’ll affect network trhoughupt or not (but I assume it can be noticed after several months - if the idea that a stale piece of Pakon config affected router performance on packet inspection, anyway it might be caused by something already removed from present Pakon version, thus not being reproducible).