For a long time, I have been wishing for an easier way to get torrents added to a remote box, download there, and then securely transfer them to my home server. There are multiple things to think about when doing this like the following:
1) The torrents must stay on the remote box and continue seeding.
2) They must be unrared when i go to view them on my home server.
3) I’d really really like to use labels with rutorrent and have those define if that torrent should be sent to my home server or not (and if so, maybe go to a certain directory)
4) My home server will not have any ports open to the world.
5) I don’t want a full sync both ways or anything like that, just a simple transfer from the remote box
to my home server.
6) This needs to be fairly fast. I don’t want to be waiting to get the latest awesome [insert linux ISO here]
7) Adding a torrent really needs to be easier (like one click on the download button and boom, done).
To sum it up, this is a way, using rutorrent labeling and easy torrent adding, with rtorrents “execute on done” function to start a torrent and forget about it moving to a remote box of your chosing.
With that out of the way, let me take you through how I’ve accomplished this through a “torrentflow” (think “workflow”) of how this entire process will work from start to finish.
Step 1: Downloading/Importing the torrent into rutorrent
Since no one likes having to download a torrent file and then importing it into rutorrent manually, dont!
A plugin called Remote Torrent Adder allows you to configure it to automatically push a torrent, when the download link is clicked, over to rutorrent (and a bunch of other webui’s too!) with the ability to specify options like the download directory and label.
Here’s a snippet of how I like to configure mine:
You can see that I have the Interactivity on so that I can easily specify labels for my downloads. These labels will come in handy later.
–> So now, when you click a torrent download, you should see a little popup telling you if it got pushed or not to rutorrent:)
Step 2: rtorrent launching a handy script after download finishes
So in order for this to work, we need rtorrent to call a script when any download finishes and pass it some variables like the rutorrent label (yes this is possible, I didn’t think so before, but I have seen the light), the base filename, and full path. This is totally possible by adding the following to your .rtorrent.rc file:
system.method.setkey = event.download.finished,checkcomplete,"execute=/usr/bin/sort.sh,$d.getcustom1=,$d.getbasepath=,$d.getbase_filename= "
Here you can easily see that on the download finished event, we are running a script and passing it those weird looking $d.etc… variables.
Now lets take a look at the script:
#!/bin/bash SORTLOG="$HOME/sort.log" #below path must have trailing slash SYMDIR='/srv/ready/' if [ "$1" "mv" ]; then ln -s "$2" "$SYMDIR" fi echo "d.getcustom1 = $1" > "$SORTLOG" echo "d.getbasepath = $2" >> "$SORTLOG" echo "d.getbase_filename = $3" >> "$SORTLOG"
So essentially what this does is:
Checks the label variable (d.get_custom1 which was the first argument passed…hence $1) to see if its something we want to move.
- If it is, we create a symlink of the download name in /srv/ready which points to the full path of the download residing in /srv (assume that /srv is the default download directory).
- It then prints out the variables it got to a log file (I did this mainly for testing to ensure we were getting what we needed…I also passed the third varialbe which we dont end up actually using) which looks like:
$ cat sort.log d.getcustom1 = mv d.getbasepath = /srv/debian-7.1.0-amd64-netinst.iso d.getbase_filename = debian-7.1.0-amd64-netinst.iso
Ideally, you probably would want to throw another if statement in there in the event ln doesn’t run correctly and then have it email you or something.
So now looking at our /srv/ready folder, we see something like this:
/srv/ready$ ls -lah lrwxrwxrwx 1 user user 27 Jul 27 21:20 debian-7.1.0-amd64-netinst.iso -> /srv/debian-7.1.0-amd64-netinst.iso
Step 3: The home server wants to omnom t3h files…
Since we don’t have any direct way of getting to the home server (which i recommend for obvious security reasons), the home server must poll the remote boxes’ /srv/ready directory and look for anything new. It must then download those contents and remove the symlink when its done.
I do this with a script that runs in cron every minute utilizing rsync and ssh with pub/priv keys:
#!/bin/bash if [ -e $HOME/rsync.lock ] then echo "Rsync job already running...exiting" exit fi touch $HOME/rsync.lock #your code in here newstuff=$(ssh [email protected] ls /srv/ready) rsync -avL -e "/usr/bin/ssh" [email protected]:/srv/ready/* /mnt/new/ if [ $? 0 ]; then #transfer was successful, lets remove the symlinks ssh [email protected] rm -rf /srv/ready/* fi #delete lock file at end of your job rm $HOME/rsync.lock if [ -e $HOME/rarcursive.lock ] then echo "Rarcursive is already running" exit fi touch $HOME/rarcursive.lock # actual launcher code here $HOME/rarcursive/rarcursive.sh /mnt/new/ delete if [ $? != 0 ]; then #unrar was unsuccessful #no yays for you yay=0 fi #delete lock file at end of your job rm $HOME/rarcursive.lock #And of course, we want pushover notifications sent to our phone if [ $yay == 1 ]; then curl -s \ -F "token=APPKEY" \ -F "user=USERKEY" \ -F "message=A transfer has completed: $newstuff" \ https://api.pushover.net/1/messages.json fi
So basically, we:
- Check to ensure rsync isnt already running.
- Run rsync to download the remote stuff to a local directory
- If that was successful, we can remove those remote symlinks
- Remove our lock file and check for a rarcursive lock file
- Run rarcursive (a recursive unrar script I made) on our local directory to take care of any rar files
- Finish up by removing the lock file and optionally you could send an email or some sort of notification somewhere (like pushover!!)
In cron, I have something like:
*/1 * * * * $HOME/sync.sh
and thats pretty much it…keep in mind that “-L” option I gave to rsync…it ensures that rsync actually follows the symlinks and doesn’t just copy the links themselves. And ensure you have your pub/priv keys setup.
NOTE: There are better ways and nicer scripting methods to do a lot of these things. I encourage you develop your own and heavily modify/create something better than mine. Also, I won’t be held responsible for any problems/deletions/destruction you encounter while following this guide etc… This is merely a way I did it and here for aiding you in understand how you can work with these utilities to make this work fairly efficiently.
UPDATE: Scenario – But Mario, what if I accidentally don’t set the label when I add the torrent. I can’t add it after it has finished downloading….what do I do!%#$%@#$^
You go to this link and love me forever?