Hi theodorius,
I used it on VPN, so it should work in your case. In Settings, you have to fill the Hostname/IP address field with your qBittorrent server's IP address. To check you are reaching your server from the phone, first connect to the Web UI, which should be something like:
http://192.168.1.100:8080
Where 192.168.1.100 is the qBittorrent server's IP address...change 192.168.1.100 with your server's IP address.
If your server's IP address is changing dynamically, then the only way it's a domain...but I think it should be the case, because commonly on a VPN all IP addresses are in the same segment and the dhcp should keep your server IP assignation (lease), therefore, your server should have the same IP address.
I hope all this information helps you!
Best regards,