Adding new Torrents sometimes causes UI to freeze

[sledgehammer_999]

If this is related to file I/O it would explain why qbt freezes or is slow. I/O with that many files creates possible bottlenecks. Nevertheless this is way too much slow down. And maybe this is related to libtorrent. Unfortunately I don't have time to look into the code (for either qbt or libtorrent). This be reported to the bug tracker: bugs.qbittorrent.org

[GenericComrade]

https://github.com/qbittorrent/qBittorrent/issues/288

[GenericComrade]

[quote="sledgehammer_999"]
If this is related to file I/O it would explain why qbt freezes or is slow. I/O with that many files creates possible bottlenecks.[/quote]Honestly, I'm not sure what is exactly causing the issue. It shouldn't take 100% CPU for a second, to check if four files exist. For a normal program, it would use 0% CPU, and maybe a moment for the Operating System to read the file tree. Lets not ignore the fact, that it shouldn't even be checking to see if the files exist.

I'm not pointing any fingers at anyone, but just saying that it doesn't seem to make any sense.

[GenericComrade]

Took roughly 95 minutes of CPU time (as shown in Task Manager) and the torrent added as 100% complete. Reopened qBittorrent, and it properly showed 0.0%.

Success (Kinda)!

[tekko]

Seems like it's the renaming step after all.

Good luck with getting your solution. 

[tekko]

[quote="ironcross"]
...
But i'm wondering why would you want to rename that folder? Download it, then rename it as you wish. Or i'm missing something about that renaming stuff?
[/quote]

For me, I come across many torrents without meaningful names anywhere (torrent's name, folder's name, filename, etc.). It would be much easier to rename those names(random numbers/letters) to something more meaningful as I add them. If there is a bug in the renaming part of the code, I hope it will be fixed sooner or later.

[sledgehammer_999]

The slow down seems to be happening inside the for loop in qbtsession.cpp at lines 1274-1290.

[GenericComrade]

It's been years since I've coded C++ (I usually code in C#), but after staring coldly at the code for awhile, this stuck out:

Code: Select all

for (int i=0; i<h.num_files(); ++i) {
  QString old_path = h.absolute_files_path().at(i);
  old_path.replace("\\", "/");
  if (!QFile::exists(old_path)) {
    // Remove old parent folder manually since we will
    // not get a file_renamed alert
    QStringList parts = old_path.split("/", QString::SkipEmptyParts);
    parts.removeLast();
    if (!parts.empty())
      QDir().rmpath(parts.join("/"));
  }
  const QString &path = files_path.at(i);
  if (!force_recheck && QDir(h.save_path()).exists(path))
    force_recheck = true;
  qDebug("Renaming file to %s", qPrintable(path));
  h.rename_file(i, path);
}

This line in particular:

Code: Select all

QString old_path = h.absolute_files_path().at(i);

That's an extremely expensive operation of order n, in the loop. Since it's in a for loop that iterates as many elements as itself, it makes it become at least order n squared. That's huge, especially for a torrent with many files. Since n is 18000 (how many files in the torrent), that becomes 18000 squared, thus 324000000!

That might explain why it's so slow to add. Move it outside of the for loop like files_path is.

Of course, thanks to sledgehammer_999 for narrowing down where the problematic code is, as this is my first time looking at the source code for qbittorrent, therefore I would have never have found it normally.

[sledgehammer_999]

@GenericComrade

Wow. You might be spot on. I used a temp variable outside the loop to hold h.absolute_files_path() (of type StringList) and used the temp variable inside the loop. The torrent is added now. It still freezes on my computer for ~20secs but it adds it.

I am sleepy now. I will make a merge request tomorrow on github. Thanks.

[GenericComrade]

No, thank you for finding the problematic area in the code . Like I said, I would have never have figured that out if I didn't know where to look.

I'm glad that I was able to contribute in some manner though. 20 seconds is still better than 95 minutes . I'm still looking at the code though, it's kinda confusing .

[GenericComrade]

(Happy Holidays Everyone!)

I think the 20 second delay may be reasonable since it does roughly 4 file/disk operations for every file in the torrent (unless force_recheck becomes true somewhere), so 72000 file/disk operations in 20 seconds (3600 operations/second).

If you're looking for more ways to improve the code, you can reduce the amount of QDir().rmpath calls by perhaps adding another temp variable to store the previous directory QDir().rmpath was called on and skipping it if it was the same as the previous.

The advantage of that is, if you have a torrent with thousands of files in a single folder (the worst case scenario), you don't have to call QDir().rmpath on the exact same directory thousands of times, which seems wasteful.

I kind of still don't quite understand the purpose of removing empty directories, that are unlikely to be there, so maybe I'm missing some deeper intent of the QDir().rmpath code.

[sledgehammer_999]

To tell you the truth I already thought of that and commented out lines 1275-1284. It provides a slight improved it saves 2-3 secs overall. I also can't think of a use case for those lines, they should be removed. But I am too busy today to work on this problem.

[sledgehammer_999]

I have pushed my changes and made a pull request. I re-counted how much time it takes. It takes ~5-6secs on my AMD x2 4200+ CPU to add the torrent now.

[sledgehammer_999]

I also made an installer of the latest git master containing my changes. Please use it, and report if it actually works on your PC.
http://qbforums.shiki.hu/index.php?topi ... 71#msg5271

[GenericComrade]

Used Windows XP SP3 Virtual Machine (vmware workstation) running on a Single Core - Pentium 4 2.8GHz:

Old Version Test (v3.0.6):
Saved Snapshot
Installed v3.0.6 (qbittorrent_3.0.6_setup.exe)
Added Torrent: Touhou album image collection v.15.torrent
Set "Save As" Path: C:\Temp\1\
Expanded Settings
Unchecked Start Torrent
Renamed base folder from "Touhou lossless music collection" to "Touhou"
Clicked OK
Killed qBittorrent after freezing for too long (it reached 4:00 CPU Time from 0:17)

Alpha Version Test (v3.1.0alpha 261212):
*Reverted back to Snapshot*
Installed v3.1.0alpha 261212 (qbittorrent_3.1.0alpha_setup261212.exe), linked here: http://qbforums.shiki.hu/index.php?topi ... 71#msg5271
Added Torrent: Touhou album image collection v.15.torrent
Set "Save As" Path: C:\Temp\1\
Expanded Settings
Unchecked Start Torrent
Renamed base folder from "Touhou lossless music collection" to "Touhou"
Clicked OK
QBitTorrent froze for ~32 Seconds of 100% CPU time (0:10->0:42)
Torrent appears on list successfully added.

Success!

I'll test it on my normal machine very soon!