Ingress traffic suffers from egress traffic

[morfikov]

When I set no limits for download/upload, qbittorrent use the whole upload bandwidth, but when I add  something to download and press play, it has problems with downloading. When I set some limits to upload traffic, let's say 50%, the file instantly starts to download at full speed. If I set no limits for upload again, the torrent event stops downloading, or it balances around 10-20KiB/s.

I have 10/1mbit connection speed, and I tested it with deluge, the same torrents, and deluge works just fine. It shows as below:



connections  | down speed  | up speed      | protocol traffic        | free space                                        | DHT nodes

As you can see there's something like protocol traffic, and 10mbit download speed consumes here about 36KiB/s + the upload speed 60KiB/s it gives more or less 1mbit upload speed. It started uploading at 110KiB/s, but after adding the file to download, the upload speed automatically dropped to 60KiB/s. I  didn't notice something like this in qbittorrent client. Maybe that's why the full upload speed consumes the protocol traffic upload, and thus it cannot continue downloading, is that possible?

I'm working on traffic control and want qbittorrent to send/receive data using the whole bandwidth without throttling other apps. I've done this and everything works perfectly well, but I can't force qbittorrent to prioritize ingress traffic over egress traffic. I'm not downloading much, but sometimes I have to add some linux ISOs and download them as fast as possible, and I don't want to play with qbittorrent settings and change upload speed each time I want to download a file.

Is there something I can do about it?

[ciaobaby]

First thing you need to do is forward the listen port so your client can get incoming connections.

[morfikov]

I think I can do nothing about these incoming connections because I have NAT.

[ciaobaby]

That's what port forwarding is for, to override NAT routing, so inbound traffic from the Wide Area Network (WAN/Internet). for a specific port can be routed to the correct device/node  on the Local Area Network, and if you didn't have NAT routing you wouldn't need to forward a port. So because your listen port is not forwarded,  incoming traffic intended for your machine and client is failing to pass the router.

[morfikov]

Do you mean this option?



I have it enabled all the time, as you can see below, the connection is good:



But the download sucks. I have to decrease the upload speed in order to make it work.

[loki]

Your probably saturating your connection capacity, set an upload limit to 80-90% of your measured max speed. With 1(meg) your theoretical max upload is 128KB subtract from this for protocol, overhead, etc... you're at about 100-115KB IF you get consistent/constant 1M.
Also experiment with the uTP settings, these should, in theory adjust your upload or download speeds based on demand from other programs.

Both Deluge and qBittorrent use libtorrent as backend so it must be a different setting between the two that's causing the problem.

[ciaobaby]

Don't use ports 6881 - 6999 as they are the "well known" ports used by BitTorrent clients and are often blocked by ISPs.

Do you have all the clients using different listening ports?

Does your router support UPnP and/or NAT-PMP?

[morfikov]

I don't have a router, just a cable modem.
I installed deluge just for tests, to check if the same behavior can be observed there too.
I changed the default port to a random one, but this does nothing. So I played with the upload speed limit. When I set 120, download sucks, the same with 110, but 100KiB/s fixes the download problems.  So,it's because of the saturation, but deluge can adjust upload speed when I'm downloading something. It seems that qbittorrent doesn't do this and try to upload at full speed all the time, no matter what. I think, there's no other way but to leave this at 100KiB/s.

[ciaobaby]

I don't have a router, just a cable modem.

Okay, so your earlier post was probably meant to be  "I don't have NAT".


Have you run the [url=http://"http://broadband.mpi-sws.org/transparency/glasnost.php"]Glasnost tests[/url] to see if your ISP is throttling your connection?

Have you tested with a known good payload such as Linux Mint that will not be affected by the Tit for Tat algorithm

[morfikov]

Network Address Translation (NAT) is a network protocol used in IPv4 networks that allows multiple devices to connect to a public network using the same public IPv4 address. NAT was originally designed in an attempt to help conserve IPv4 addresses.[1]

So, that means I can't get incoming connections, that I don't have a public ip address, right? I mean I have, but the same as 10k people living in the area around me. You cannot connect to me directly, or am I getting it wrong?

I can't make the Glasnost test work, I have some java problems, and I have no idea what page I should add to the exceptions.

As I said before, everything works fine, the problem is that I can't download, or the downloading is slow when upload has no limits, and it appears to be only in qbittorrent. So, that's the problem -- I don't have any problems with downloading or uploading anything via p2p in general.

[ciaobaby]

So, that means I can't get incoming connections, that I don't have a public ip address, right?

That depends on how your machine is connected to the cable modem and what IP is being assigned to the network card on your machine.

I can't make the Glasnost test work, I have some java problems, and I have no idea what page I should add to the exceptions.

http://www.howtogeek.com/165481/how-to- ... onnection/

or the downloading is slow when upload has no limits,

Testing those conditions with a known good torrent will demonstrate whether it IS a qBitTorrent problem or simply a factor of the jobs you are running. There are many reasons why jobs behave in such a way, and not all of them are the client that you are using.

[morfikov]

My ip is 10.*.*.* , so it's pretty obvious to me what that means.

I finally made that test work:

Is your upload traffic rate limited?

There is no indication that your ISP rate limits your uploads.
Is your download traffic rate limited?

There is no indication that your ISP rate limits your downloads.

[ciaobaby]

My ip is 10.*.*.* , so it's pretty obvious to me what that means

Know we also know that, it means that that the NAT routing for your connection is being done in the ISPs datacentre,.it also means that if two network nodes on your particular 10.0.0.0/8 subnet have two [or more] distinct devices simultaneously listening on the same port, BOTH devices will be missing incoming packets.

Did testing with a Linux torrent under your problem conditions yield any results?

[morfikov]

I got the following output, but I think it's because of qbittorrent:

Is your upload traffic rate limited?

There is no indication that your ISP rate limits your uploads.
However, some of the measurements were affected by noise, which limits Glasnost ability to detect rate limiting.

Details:

* There is no indication that your ISP rate limits your BitTorrent uploads. In our tests, uploads using control flows achieved up to 580 Kbps while uploads using BitTorrent achieved up to 449 Kbps.

* The measurement data is too noisy to detect whether your ISP rate limits traffic on port 6881. Re-running the test while ensuring that no other downloads or uploads are running in the background might fix this problem.
Is your download traffic rate limited?

There is no indication that your ISP rate limits your downloads.
However, some of the measurements were affected by noise, which limits Glasnost ability to detect rate limiting.

Details:

* There is no indication that your ISP rate limits your BitTorrent downloads. In our tests, downloads using control flows achieved up to 4863 Kbps while downloads using BitTorrent achieved up to 5165 Kbps.

* There is no indication that your ISP rate limits downloads on port 6881 or 52744. In our tests, downloads on port 6881 achieved up to 4863 Kbps while downloads on port 52744 achieved up to 4899 Kbps.

But what is interesting is that I couldn't download anything for the time of testing. When it was about 100secs to the end, it started to download, and ultimately it got 100% download speed.

I think it's because of the unlimited upload speed.

[morfikov]

What do you think about the following solution?

Code: Select all

iptables -t mangle -A POSTROUTING -o eth0 -m length --length 40:68 -j CLASSIFY --set-class 1:3
iptables -t mangle -A POSTROUTING -o eth0 -m length --length 40:68 -j ACCEPT

I read somewhere that the overhead packets have 40-68 bytes, and you can prioritize them.

No limits (upload 950kbit/s)

Code: Select all

Interfaces                     | RX bps       pps     %| TX bps       pps     %
  ifb0                         | 114.37KiB    117      | 114.37KiB    117
    qdisc 1: (htb)             |      0         0      | 115.58KiB    136
      class 1:1 (htb)          |      0         0      | 115.58KiB    136   100%
        class 1:2 (htb)        |      0         0      | 112.90KiB    100   617%
        class 1:3 (htb)        |      0         0      |   2.27KiB     35    9%
        class 1:4 (htb)        |      0         0      |    415B        0    1%
        class 1:5 (htb)        |      0         0      |      0         0    0%
  ifb1                         |   9.29KiB     93      |   9.29KiB     93
    qdisc 2: (htb)             |      0         0      |   9.29KiB     93
      class 2:1 (htb)          |      0         0      |   9.29KiB     93    1%
        class 2:2 (htb)        |      0         0      |      0         0    0%
        class 2:3 (htb)        |      0         0      |    270B        0    0%
        class 2:4 (htb)        |      0         0      |      0         0    0%
        class 2:5 (htb)        |      0         0      |   9.03KiB     92    5%

And while downloading:

Code: Select all

Interfaces                     | RX bps       pps     %| TX bps       pps     %
  ifb0                         | 113.79KiB    859      | 113.79KiB    859
    qdisc 1: (htb)             |      0         0      | 114.83KiB    875
      class 1:1 (htb)          |      0         0      | 114.83KiB    875   99%
        class 1:2 (htb)        |      0         0      |  63.69KiB     53   348%
        class 1:3 (htb)        |      0         0      |  50.48KiB    821   207%
        class 1:4 (htb)        |      0         0      |    345B        1    1%
        class 1:5 (htb)        |      0         0      |    372B        0    2%
  ifb1                         |   1.12MiB    847      |   1.12MiB    847
    qdisc 2: (htb)             |      0         0      |   1.12MiB    848
      class 2:1 (htb)          |      0         0      |   1.12MiB    847   99%
        class 2:2 (htb)        |      0         0      |      0         0    0%
        class 2:3 (htb)        |      0         0      |    919B        5    0%
        class 2:4 (htb)        |      0         0      |      0         0    0%
        class 2:5 (htb)        |      0         0      |   1.12MiB    842   627%

The ifb0 interface is for egress traffic, ifb1 for ingress. Queue 1.2 is for qbittorrent, and 1.3 for overhead packets. 1.3 has higher priority than 1.2. As you can see 1.2 consumes 115KiB/s -- all the bandwidth when it's free, and it dropped from 115KiB to 63 KiB immediately after qbittorrent started downloading a torrent because the overhead packets consume now around 45-50KiB/s, and they have higher priority.

I'm not sure if it's the right way to deal with it, but it seems to work.