I'm running qbit on a headless machine, set it and forget it. Which apparently i have done for quite a while, because i opened up advanced settings in 4.1.5 and discovered settings i had no idea what did. So the search began.
In the qbit wiki i've found that the explanations are mostly copy & paste from libtorrent, and searching google has not yielded much results.
"Optional IP address to bind to — TBA"
If i'm not mistaken, if set to "any" by someone who is using qbittorrent over VPN and the VPN service goes down, qbit will continue working using the local lan->wan connection. Exposing real wan ip to tracker, binding to VPN ip would prevent that from happening.
"Asynchronous I/O threads — TBA"
Correct me if i'm wrong, this setting is for disk I/O threads and set to a higher value allows more read/write I/O/queues to/from disks. Changing this value appears to mostly be useful for people with +500mbit and fast disk setups.
"Send buffer watermark"
I have not been able to find much on this besides a few comments here and there.
One saying it helps reduce pipeline stalls if increased. Another saying(old deluge thread) it should be roughly the size of your max upload speed per second & that 5mb is not sufficient for gbit. Which i read as ~1mb per 10mbit upload.
As far as i understand, changing this setting is mostly useful if you have +300mbit, to what i am not sure unless "Send buffer watermark factor" calculation gives us this value.
"Send buffer watermark factor - ...For high speed upload, this should be set to a greater value than 100."
Does this value only effect "Send buffer watermark" calculation and what the "Send buffer watermark" value should be set to?
There some calculation here which ties into "Send buffer watermark" where "Send buffer watermark factor" is multiplied with the upload rate per peer to get the "Send buffer watermark" value.
150%, 100 upload slots and a max upload speed of 100mb/s equals 1mb/s per slot.
Then the "Send buffer watermark" value should be set to 150*1*100 = 15000?
"Send upload piece suggestions"
Is this still bugged?
"Upload slots behavior"
- Fixed slots. Is this fixed to the max upload slots per torrent value?
- Upload rate based. Does this overwrite the max upload slots per torrent value?
If none of that, then what does unchoke mean?
Need elaboration on advanced settings & missing wiki explanations.
-
fusk
Need elaboration on advanced settings & missing wiki explanations.
Last edited by fusk on Tue May 14, 2019 7:45 pm, edited 1 time in total.
-
Switeck
Re: Need elaboration on advanced settings & missing wiki explanations.
"Optional IP address to bind to — TBA"
"If set to "any" by someone who is using qbittorrent over VPN and the VPN service goes down, qbit will continue working using the local lan->wan connection."
For those on a VPN, setting that to "any" indeed sounds like a bad idea.
"IP address to report to tracker" (at very bottom of Advanced settings) might also be helpful for a VPN or proxy.
"Asynchronous I/O threads — TBA"
"this setting is for disk I/O threads and set to a higher value allows more read/write I/O/queues to/from disks. Changing this value appears to mostly be useful for people with +500mbit and fast disk setups."
I believe you're correct there.
What's more, changing that value to more than double the number of CPU cores (or virtual cores, in the case of Hyperthreading) seems like it would have extremely diminishing improvements. The extra threads would not be fighting other apps for CPU time, they'd be fighting each other with increasing overheads that likely slow everything down including qBitTorrent.
Worth testing, but don't expect to see much improvement -- setting it to 1 or 100+ might at least show much lower speeds.
"Send buffer watermark"
In my testing of that and similar low water mark one, raising the low water mark one helped the most...but at high speeds (>100 mbit/sec) I needed to raise the max too to maximize speeds.
Definitely needs some testing trials to find sweet spot, but I doubt raising them over 1 megabyte is advisable -- in the event of 100+ connections, each connection's buffer could be that 1+ MB size assuming high enough speeds. By the nature of the wording, send buffer implies UPLOAD sending buffer only! This is above-and-beyond qBitTorrent's general cache size.
"Send upload piece suggestions"
no longer seems bugged, I'm not sure it ever was...it may have been another speed bug making that look like the cause when I was testing it.
"Upload slots behavior"
"- Fixed slots. Is this fixed to the max upload slots per torrent value?"
Fixed slots is almost certainly limited to the max upload slots global AND per torrent.
"- Upload rate based. Does this overwrite the max upload slots per torrent value?"
Upload rate based may go up to the max upload slots global and per torrent but often due to temporarily slow upload speeds will limit upload slots even fewer than the max.
It likely isn't going to work well on torrents that are intermittently active with sometimes fast peers and sometimes slow ones -- during slow times, fast peers could connect and not be given an upload slot because the upload rate based method is at its artificially lowered limit.
Incidentally, uTorrent has a hidden always-on upload slot rate limiter. I asked for it, but it ended up being slightly poorly implemented. If there's 1 very busy torrent with lots of peers and another torrent goes active (from a peer joining it) with a very slow peer, then the fast torrent loses potentially half or more of its upload slots. (especially true if using uTP) A 2nd or 3rd slow torrent starting could have the single fast torrent going from 20+ upload slots to <3 despite the slow torrents not using 5% of the total upload speed. I often saw this when a single slow ip tried to download 2+ torrents at once from me (often grabbing a series of related torrents all at once) -- my 1-3 busy torrents would go from uploading near my max upload limit to <10%.
uTorrent v3.0 to about v3.1 tried to implement a multiple optimistic unchoke method on a per-torrent basis that ended up being a complete flop. It might work ok on very fast lines but did terrible with 1 mbit/sec upload (like on ADSL 2+) or lower...it'd still try to give out 2-4 optimistic unchoke upload slots on each torrent even if that means each ran at <1 KB/sec.
"If none of that, then what does unchoke mean?"
unchoke = telling the connected peer that it is allowed to start downloading NOW as opposed to choked = connection stops upload. choke/unchoke is on a per-peer basis -- unchoke corresponds to 1 upload slot, possibly even the "optimistic unchoke" one meant to give new peers with 0% complete something. (Even that one counts against the total upload slot limits.)
If you're just looking for max speeds and don't care so much about looks...disabling some other features and hiding much of qBitTorrent's GUI might help slightly as well.
"Download tracker's favicon" might be slightly buggy still, at least when a tracker tries to pass a very large "icon" through as the result...or when the same tracker appears on 100+ torrents -- does qBT download the favicon once or 100+ times? There was also some talk of favicon exploits/backdoors that sounded like it's fixed now but briefly (in earlier qBT versions) allowed someone to hack into peers/seeds to gain partial/full control over those computers.
"Resolve peer host names" and "Resolve peer countries (GeoIP)" make additional connections as well as hide what the real ips for peers/seeds are.
"uTP-TCP mixed mode algorithm" pretty much has to be left set to "Prefer TCP" until some massive uTP bugs get fixed in libtorrent's core.
https://github.com/arvidn/libtorrent/is ... -461294985
...demonstrates how poor libtorrent's uTP performance is.
Still, too many peers/seeds use uTP to completely disable it at least for low seed+peer count torrents.
"Always announce to all tiers" makes qBT work like uTorrent -- each tracker tier will have one-and-only-one tracker announced on, unless it fails...then it's supposed to try another in the same tier (but may fail due to bugs?) I think this defaults to checked for new installs of qBT because qBT/libtorrent is so horrible switching to working trackers when the 1st one fails.
"Always announce to all trackers in a tier"
...Is buggy especially with magnet links with multiple added trackers OR when manually adding multiple trackers (where there seems no way to sort them into tiers like uTorrent can):
https://github.com/qbittorrent/qBittorrent/issues/8099
https://github.com/qbittorrent/qBittorrent/issues/8397
Most trackers explicitly want peers and seeds to use only 1 of their sub-trackers at a time -- such as their udp one as their main one, their http one for a backup for those who can't use the main one (such as proxies/Tor that don't support UDP), or in rare cases only their HTTPS one for encryption to get by ISP throttling and monitoring.
Also qBitTorrent doesn't seem to have even the most basic means to prevent announces to similar or duplicate tracker names.
Everyone that set up their BitTorrent app to add a long list of trackers to every torrent...then sets qBT to announce to all of them regardless of tier adds to this problem.
"If set to "any" by someone who is using qbittorrent over VPN and the VPN service goes down, qbit will continue working using the local lan->wan connection."
For those on a VPN, setting that to "any" indeed sounds like a bad idea.
"IP address to report to tracker" (at very bottom of Advanced settings) might also be helpful for a VPN or proxy.
"Asynchronous I/O threads — TBA"
"this setting is for disk I/O threads and set to a higher value allows more read/write I/O/queues to/from disks. Changing this value appears to mostly be useful for people with +500mbit and fast disk setups."
I believe you're correct there.
What's more, changing that value to more than double the number of CPU cores (or virtual cores, in the case of Hyperthreading) seems like it would have extremely diminishing improvements. The extra threads would not be fighting other apps for CPU time, they'd be fighting each other with increasing overheads that likely slow everything down including qBitTorrent.
Worth testing, but don't expect to see much improvement -- setting it to 1 or 100+ might at least show much lower speeds.
"Send buffer watermark"
In my testing of that and similar low water mark one, raising the low water mark one helped the most...but at high speeds (>100 mbit/sec) I needed to raise the max too to maximize speeds.
Definitely needs some testing trials to find sweet spot, but I doubt raising them over 1 megabyte is advisable -- in the event of 100+ connections, each connection's buffer could be that 1+ MB size assuming high enough speeds. By the nature of the wording, send buffer implies UPLOAD sending buffer only! This is above-and-beyond qBitTorrent's general cache size.
"Send upload piece suggestions"
no longer seems bugged, I'm not sure it ever was...it may have been another speed bug making that look like the cause when I was testing it.
"Upload slots behavior"
"- Fixed slots. Is this fixed to the max upload slots per torrent value?"
Fixed slots is almost certainly limited to the max upload slots global AND per torrent.
"- Upload rate based. Does this overwrite the max upload slots per torrent value?"
Upload rate based may go up to the max upload slots global and per torrent but often due to temporarily slow upload speeds will limit upload slots even fewer than the max.
It likely isn't going to work well on torrents that are intermittently active with sometimes fast peers and sometimes slow ones -- during slow times, fast peers could connect and not be given an upload slot because the upload rate based method is at its artificially lowered limit.
Incidentally, uTorrent has a hidden always-on upload slot rate limiter. I asked for it, but it ended up being slightly poorly implemented. If there's 1 very busy torrent with lots of peers and another torrent goes active (from a peer joining it) with a very slow peer, then the fast torrent loses potentially half or more of its upload slots. (especially true if using uTP) A 2nd or 3rd slow torrent starting could have the single fast torrent going from 20+ upload slots to <3 despite the slow torrents not using 5% of the total upload speed. I often saw this when a single slow ip tried to download 2+ torrents at once from me (often grabbing a series of related torrents all at once) -- my 1-3 busy torrents would go from uploading near my max upload limit to <10%.
uTorrent v3.0 to about v3.1 tried to implement a multiple optimistic unchoke method on a per-torrent basis that ended up being a complete flop. It might work ok on very fast lines but did terrible with 1 mbit/sec upload (like on ADSL 2+) or lower...it'd still try to give out 2-4 optimistic unchoke upload slots on each torrent even if that means each ran at <1 KB/sec.
"If none of that, then what does unchoke mean?"
unchoke = telling the connected peer that it is allowed to start downloading NOW as opposed to choked = connection stops upload. choke/unchoke is on a per-peer basis -- unchoke corresponds to 1 upload slot, possibly even the "optimistic unchoke" one meant to give new peers with 0% complete something. (Even that one counts against the total upload slot limits.)
If you're just looking for max speeds and don't care so much about looks...disabling some other features and hiding much of qBitTorrent's GUI might help slightly as well.
"Download tracker's favicon" might be slightly buggy still, at least when a tracker tries to pass a very large "icon" through as the result...or when the same tracker appears on 100+ torrents -- does qBT download the favicon once or 100+ times? There was also some talk of favicon exploits/backdoors that sounded like it's fixed now but briefly (in earlier qBT versions) allowed someone to hack into peers/seeds to gain partial/full control over those computers.
"Resolve peer host names" and "Resolve peer countries (GeoIP)" make additional connections as well as hide what the real ips for peers/seeds are.
"uTP-TCP mixed mode algorithm" pretty much has to be left set to "Prefer TCP" until some massive uTP bugs get fixed in libtorrent's core.
https://github.com/arvidn/libtorrent/is ... -461294985
...demonstrates how poor libtorrent's uTP performance is.
Still, too many peers/seeds use uTP to completely disable it at least for low seed+peer count torrents.
"Always announce to all tiers" makes qBT work like uTorrent -- each tracker tier will have one-and-only-one tracker announced on, unless it fails...then it's supposed to try another in the same tier (but may fail due to bugs?) I think this defaults to checked for new installs of qBT because qBT/libtorrent is so horrible switching to working trackers when the 1st one fails.
"Always announce to all trackers in a tier"
...Is buggy especially with magnet links with multiple added trackers OR when manually adding multiple trackers (where there seems no way to sort them into tiers like uTorrent can):
https://github.com/qbittorrent/qBittorrent/issues/8099
https://github.com/qbittorrent/qBittorrent/issues/8397
Most trackers explicitly want peers and seeds to use only 1 of their sub-trackers at a time -- such as their udp one as their main one, their http one for a backup for those who can't use the main one (such as proxies/Tor that don't support UDP), or in rare cases only their HTTPS one for encryption to get by ISP throttling and monitoring.
Also qBitTorrent doesn't seem to have even the most basic means to prevent announces to similar or duplicate tracker names.
Everyone that set up their BitTorrent app to add a long list of trackers to every torrent...then sets qBT to announce to all of them regardless of tier adds to this problem.
-
fusk
Re: Need elaboration on advanced settings & missing wiki explanations.
[quote="Switeck"]
"Send buffer watermark"
In my testing of that and similar low water mark one, raising the low water mark one helped the most...but at high speeds (>100 mbit/sec) I needed to raise the max too to maximize speeds.
Definitely needs some testing trials to find sweet spot, but I doubt raising them over 1 megabyte is advisable -- in the event of 100+ connections, each connection's buffer could be that 1+ MB size assuming high enough speeds. By the nature of the wording, send buffer implies UPLOAD sending buffer only! This is above-and-beyond qBitTorrent's general cache size.
[/quote]
What about the calculations of "Send buffer watermark factor"
"The current upload rate to a peer is multiplied by this factor to get the send buffer watermark. This product is clamped to the "Send buffer watermark" setting so as to not exceed the max."
As i understand that, it's a way to find what the "send buffer watermark" value should be. Ie: 150*1*100=15000. To prevent "so as to not exceed the max."
Posted the settings i use below.
[quote="Switeck"]
"Send upload piece suggestions"
no longer seems bugged, I'm not sure it ever was...it may have been another speed bug making that look like the cause when I was testing it.
[/quote]
Ah, i was referring to your testing but didn't want to name names. Saw no mention of it on github as verified/fixed, so i have left it off until i knew for sure.
[quote="Switeck"]
"If none of that, then what does unchoke mean?"
unchoke = telling the connected peer that it is allowed to start downloading NOW as opposed to choked = connection stops upload. choke/unchoke is on a per-peer basis -- unchoke corresponds to 1 upload slot, possibly even the "optimistic unchoke" one meant to give new peers with 0% complete something. (Even that one counts against the total upload slot limits.)
[/quote]
So this is better suited for slower connections, and not so much for high speed that are running higher amount of fixed slots and don't want slots to be choked. Using fixed & fastest with plenty of slots makes the unchoke algorithm not do much?
I already have "Download tracker's favicon" & "Resolve peer host names" off. But i'll turn "Always announce to all tiers" on because one of the private trackers i use has multiple trackers.
Present settings.
Active protocols = TCP
Upnp, random port, DHT, pex & local: Off
Global connections settings: disabled, per torrent: 150, per torrent upload slots: 150.
Rate limit overhead = On
Require encryption.
Asynchronous I/O threads: 4
Disk cache: 3000
Disk cache expiry interval: 600
Enable OS cache: No
Guided read cache: Yes
Coalesce reads & writes: Yes
Send buffer watermark: 9000
Send buffer low watermark: 400
Send buffer watermark factor: 150
Maximum number of half-open connections: 10
µTP-TCP mixed mode algorithm: Prefer TCP
Upload slots behavior: Fixed
Upload choking algorithm: Fastest
Send upload piece suggestions: Now on.
Always announce to all tiers: Now on.
"Send buffer watermark"
In my testing of that and similar low water mark one, raising the low water mark one helped the most...but at high speeds (>100 mbit/sec) I needed to raise the max too to maximize speeds.
Definitely needs some testing trials to find sweet spot, but I doubt raising them over 1 megabyte is advisable -- in the event of 100+ connections, each connection's buffer could be that 1+ MB size assuming high enough speeds. By the nature of the wording, send buffer implies UPLOAD sending buffer only! This is above-and-beyond qBitTorrent's general cache size.
[/quote]
What about the calculations of "Send buffer watermark factor"
"The current upload rate to a peer is multiplied by this factor to get the send buffer watermark. This product is clamped to the "Send buffer watermark" setting so as to not exceed the max."
As i understand that, it's a way to find what the "send buffer watermark" value should be. Ie: 150*1*100=15000. To prevent "so as to not exceed the max."
Posted the settings i use below.
[quote="Switeck"]
"Send upload piece suggestions"
no longer seems bugged, I'm not sure it ever was...it may have been another speed bug making that look like the cause when I was testing it.
[/quote]
Ah, i was referring to your testing but didn't want to name names. Saw no mention of it on github as verified/fixed, so i have left it off until i knew for sure.
[quote="Switeck"]
"If none of that, then what does unchoke mean?"
unchoke = telling the connected peer that it is allowed to start downloading NOW as opposed to choked = connection stops upload. choke/unchoke is on a per-peer basis -- unchoke corresponds to 1 upload slot, possibly even the "optimistic unchoke" one meant to give new peers with 0% complete something. (Even that one counts against the total upload slot limits.)
[/quote]
So this is better suited for slower connections, and not so much for high speed that are running higher amount of fixed slots and don't want slots to be choked. Using fixed & fastest with plenty of slots makes the unchoke algorithm not do much?
I already have "Download tracker's favicon" & "Resolve peer host names" off. But i'll turn "Always announce to all tiers" on because one of the private trackers i use has multiple trackers.
Present settings.
Active protocols = TCP
Upnp, random port, DHT, pex & local: Off
Global connections settings: disabled, per torrent: 150, per torrent upload slots: 150.
Rate limit overhead = On
Require encryption.
Asynchronous I/O threads: 4
Disk cache: 3000
Disk cache expiry interval: 600
Enable OS cache: No
Guided read cache: Yes
Coalesce reads & writes: Yes
Send buffer watermark: 9000
Send buffer low watermark: 400
Send buffer watermark factor: 150
Maximum number of half-open connections: 10
µTP-TCP mixed mode algorithm: Prefer TCP
Upload slots behavior: Fixed
Upload choking algorithm: Fastest
Send upload piece suggestions: Now on.
Always announce to all tiers: Now on.
Last edited by fusk on Wed May 22, 2019 11:06 am, edited 1 time in total.
-
fusk
Re: Need elaboration on advanced settings & missing wiki explanations.
Edited for clarity.
Ok, so apparently it works like this.
I have less in my buffer than specified here "Send buffer low watermark" please get more data from disk.
I have more in my buffer than specified here "Send buffer watermark" please get less data from disk.
It's an upload data buffer, so these settings won't help your download.
"Send buffer low watermark" is the lowest amount of "upload" data in the buffer before it requests more, watermark and disk cache are two different kind of "buffers" and therefore watermark is not included in the show/statistics/total buffer size afaik.
"Send buffer watermark" is the maximum amount of "upload" data in the watermark buffer. The buffer is dynamic and speed dependant, the faster you upload, the more data is filled in "Send buffer watermark".
"Send buffer watermark factor" value seems to be the speed of which it should fill the buffer. The default is 50%, which is more than enough for most sub 100mbit users as your disks most likely won't have any issues keeping up.
Higher values ie: 100%, 120% etc. means the buffer fills at a faster rate, because the peer speed is multiplied with this value(The descriptions says "per peer" and not "peers", so my guess is that it's an average).
A value over 125% for anyone seems unnecessary. For gbit users i see no reason why "Send buffer watermark" couldn't be ~100000 as you can send that amount each second. Because it's dynamic, the faster you upload the more system memory qbit will use until "Send buffer watermark" is full.
The assumption here is that it's 1:1, so a value of 100000 equals 100mb of memory usage when full.
It would be nice if we had more info in statistics. Buffer size, watermark buffer size & total buffer size. Then it would make it easier to see how much is actually being used and for what.
Ok, so apparently it works like this.
I have less in my buffer than specified here "Send buffer low watermark" please get more data from disk.
I have more in my buffer than specified here "Send buffer watermark" please get less data from disk.
It's an upload data buffer, so these settings won't help your download.
"Send buffer low watermark" is the lowest amount of "upload" data in the buffer before it requests more, watermark and disk cache are two different kind of "buffers" and therefore watermark is not included in the show/statistics/total buffer size afaik.
"Send buffer watermark" is the maximum amount of "upload" data in the watermark buffer. The buffer is dynamic and speed dependant, the faster you upload, the more data is filled in "Send buffer watermark".
"Send buffer watermark factor" value seems to be the speed of which it should fill the buffer. The default is 50%, which is more than enough for most sub 100mbit users as your disks most likely won't have any issues keeping up.
Higher values ie: 100%, 120% etc. means the buffer fills at a faster rate, because the peer speed is multiplied with this value(The descriptions says "per peer" and not "peers", so my guess is that it's an average).
A value over 125% for anyone seems unnecessary. For gbit users i see no reason why "Send buffer watermark" couldn't be ~100000 as you can send that amount each second. Because it's dynamic, the faster you upload the more system memory qbit will use until "Send buffer watermark" is full.
The assumption here is that it's 1:1, so a value of 100000 equals 100mb of memory usage when full.
It would be nice if we had more info in statistics. Buffer size, watermark buffer size & total buffer size. Then it would make it easier to see how much is actually being used and for what.
Last edited by fusk on Thu May 23, 2019 5:23 pm, edited 1 time in total.
-
Switeck
Re: Need elaboration on advanced settings & missing wiki explanations.
Yes, more statistics instead of having to estimate usage based on Windows Task Manager, Process Explorer, or something similar.
So "Send buffer low watermark" could be for the total upload data (packets?) to hold in reserve to keep a constant upload rate? (instead of per peer)
...While the download buffer would be at least all the partial pieces held in ram till they complete and get committed to the destination storage drive.
And speaking of qBT's caches...this seems strange:
https://github.com/qbittorrent/qBittorr ... -421966116
So "Send buffer low watermark" could be for the total upload data (packets?) to hold in reserve to keep a constant upload rate? (instead of per peer)
...While the download buffer would be at least all the partial pieces held in ram till they complete and get committed to the destination storage drive.
And speaking of qBT's caches...this seems strange:
https://github.com/qbittorrent/qBittorr ... -421966116
-
fusk
Re: Need elaboration on advanced settings & missing wiki explanations.
I edited the post above for clarity.
[quote="Switeck"]
So "Send buffer low watermark" could be for the total upload data (packets?) to hold in reserve to keep a constant upload rate? (instead of per peer)
[/quote]
Assume you mean "Send buffer watermark" without "low", but yes, that is how i understand it. Wonder if "send buffer watermark" influences "Send upload piece suggestions".
[quote="Switeck"]
...While the download buffer would be at least all the partial pieces held in ram till they complete and get committed to the destination storage drive.
[/quote]
I'm also guessing that disk cache pieces held in ram are still used for upload, even if both are enabled.
[quote="Switeck"]
And speaking of qBT's caches...this seems strange:
https://github.com/qbittorrent/qBittorr ... -421966116
[/quote]
I have not experienced that issue, ever.
I don't know if it's just me, a language thing or whatever. But some of these descriptions, i can see the words are english, i understand the words. But how it's put together and after reading them a few times i'm just thinking "what?"
[quote="Switeck"]
So "Send buffer low watermark" could be for the total upload data (packets?) to hold in reserve to keep a constant upload rate? (instead of per peer)
[/quote]
Assume you mean "Send buffer watermark" without "low", but yes, that is how i understand it. Wonder if "send buffer watermark" influences "Send upload piece suggestions".
[quote="Switeck"]
...While the download buffer would be at least all the partial pieces held in ram till they complete and get committed to the destination storage drive.
[/quote]
I'm also guessing that disk cache pieces held in ram are still used for upload, even if both are enabled.
[quote="Switeck"]
And speaking of qBT's caches...this seems strange:
https://github.com/qbittorrent/qBittorr ... -421966116
[/quote]
I have not experienced that issue, ever.
I don't know if it's just me, a language thing or whatever. But some of these descriptions, i can see the words are english, i understand the words. But how it's put together and after reading them a few times i'm just thinking "what?"
Last edited by fusk on Fri May 24, 2019 8:03 am, edited 1 time in total.