Read/Write Cache Overload

Other platforms, generic questions.
Post Reply
below9k

Read/Write Cache Overload

Post by below9k »

I'm having some issues with read and write, mostly read and sometimes write, overload blocking qBittorrent. As I'm writing this I have a queue of over 1k I/O jobs and read overload of ~70% and 0% write
Should I be changing my cache settings? I figured auto would take care of me :/
I ran iperf during this transfer and still was able to get 255mbps.

Setup:
I have one box for downloading that after a download is complete it moves the files across the local network to another box. I assume this has something to do with network latency/bandwidth, but they seem to be decent (ping:<1ms / iperf:400+mbps).

Boxes:
- Both
  - Ubuntu 14.04

- Download box
  - Software vpn to remote server for anonymity (I don't think this is relevant)
  - 500Gb 10k rpm raptor - staging drive where completed downloads are moved from
  - i5 Quad / 8GB Ram

- Other box
  - Varying drives mostly 5.4k rpm - mapped to network with cifs/samba utility
  - i7 Quad / 12GB Ram

Thanks! Hopefully I can learn something here..
Last edited by below9k on Fri Feb 12, 2016 7:51 pm, edited 1 time in total.
ciaobaby

Re: Read/Write Cache Overload

Post by ciaobaby »

A 'read' overload, provided it is all or mostly attributable to qbittorrent would be indicative of your upload slots to connections ratio  being higher than your system can actually cope with.

One upload slot can handle multiple connection so you have to calculate what ratio is 'right' for your 'use case'.

index.php/topic,3325.msg16478/topicseen.html#msg16478

index.php/topic,3714.msg18708/topicseen.html#msg18708

Plus;
https://github.com/qbittorrent/qBittorr ... ed'-issues
below9k

Re: Read/Write Cache Overload

Post by below9k »

Okay that makes sense. This points to a lack of network bandwidth or connection issue.
I've limited my upload slots to only a couple.

Now the issue is when a download completes and the files are moved from the staging drive to their final destination, everything switches to "Stalled", write overload spikes to ~80%, and finally resumes downloading other files after that file finishes moving. I get a notification of download success afterwards.
Can this be prevented some how?
Perhaps this because the network transfer bandwidth consumes all the available bandwidth, leaving none for downloading?
ciaobaby

Re: Read/Write Cache Overload

Post by ciaobaby »

Can this be prevented some how?
It can't,  that is how libtorrent woks, it is single threaded so can only handle a single  intensive process, so when a payload is being moved,  ... ... Nothing else happens until the storage move is complete.

Though keeping payloads on the same filing system (or drive for Windows) throughout, means that files do not require a physical move between file systems, only a 'virtual' move of the reference points (directories).
laceybeck982

Re: Read/Write Cache Overload

Post by laceybeck982 »

I fixed uTorrent 2.2.1 by selecting Pre-allocate all files in Preference  General Might work for other versions as well but I haven't tested.
ciaobaby

Re: Read/Write Cache Overload

Post by ciaobaby »

[quote="laceybeck982"]
I fixed uTorrent 2.2.1 by selecting Pre-allocate all files in Preference  General Might work for other versions as well but I haven't tested.
[/quote]

If you ARE going to copy and paste, .... .... .... Try and keep then related at least.
Post Reply