Raspberry Pi download scheduler

My solution to using bandwidth that's only available when I'd rather be asleep.
Last update: 19 March 2019

My Internet service limits data. Between midnight and 7 am, I have twice as much available as during the rest of the day. Naturally enough, I'd like to make use of that data. I'd also like to get some sleep.

I'd rather not leave my desktop running overnight. An old Raspberry Pi Model B should be able to do the job. For this project, the Pi is running Raspbian Stretch Lite.

Assumptions

  1. A functioning Raspberry Pi, with working network connection, Internet access, SSH enabled and fully updated system software.
  2. Some familiarity with Linux.
  3. Capacity to access the Pi from another computer. On Windows, I use the "PuTTY" program.

File down-loader

I couldn't find a download scheduler with a web interface that works in a way that I'm comfortable with. There seems to be more interest in making downloads faster than in making them run unattended on a schedule. My fall-back is to schedule a command-line download accelerator.

First, the scheduler. File downloads are one-off tasks. The "at" command will do the job.

Install "at" and set daemon to autostart:

sudo apt-get install at
sudo service atd start
sudo update-rc.d atd defaults

To subsequently disable and re-enable autostart, use:

sudo update-rc.d atd disable|enable

update-rc.d has a 'remove' option, but that's only used after a program has been uninstalled and all scripts deleted.

More on update-rc.d here.

To check what services are running:

sudo service --status-all

More on reading service staus here.

The "wget" command is installed by default, but it has some nasty habits. I prefer "axel".

Install "axel" download accelerator:

sudo apt-get install axel

By default, "at" mails output to the user's local account. Check mail regularly (use the "mail" command, then "d" to delete messages and "q" to exit), to prevent a buildup.

Create a directory for downloads (I named mine "downloads") :).

Sample command sequence (modify according to your download time, directory and the file to be downloaded):

sudo at 4 AM tomorrow

Response:

warning: commands will be executed using /bin/sh
at>

At the at> prompt:

cd /home/pi/downloads

then

axel https://report.ipcc.ch/sr15/pdf/sr15_spm_final.pdf

Finish with Control-D.

The above is an example only. Replace https: and the rest of that line with the URL of the file you want to download. Finding the URL can be difficult. Some sites lead you through pages of advertising or begging for donations before getting to the actual file. I use Free Download Manager (set to manual download), a browser extension that traps download requests and records them for future download. I copy the URL from FDM, then delete the download from the list.

It actually gets quite easy after you've done it a few times.

BitTorrent client

The go-to client for Linux is Transmission. It's light on resources and has a handy web interface, so it's easy to access from machines on the same network. It does take a bit of setting up though. I hope I've remembered everything.

Install Transmission daemon:

sudo apt-get install transmission-daemon

Add user to transmission group (I'm using the default user, "pi"):

sudo usermod -a -G debian-transmission pi

From the user's home directory (again, I'm using "pi"):

mkdir transmission
cd transmission
mkdir complete
mkdir incomplete

Change group ownership of newly created directories:

sudo chgrp debian-transmission /home/pi/transmission/
sudo chgrp debian-transmission /home/pi/transmission/incomplete
sudo chgrp debian-transmission /home/pi/transmission/complete

Even when upload and download speeds are set to zero, Transmission seems to use data. I therefore stop the daemon when not in use. Starting and stopping during off-peak hours is handled by cron. Outside those hours, to start Transmission:
sudo transtart.sh
and to stop it:
sudo transtop.sh

Copy shell scripts to /usr/bin/. For that, I used FileZilla.

Make sure scripts are owned by root and executable:

sudo chown root: /usr/bin/transtart.sh
sudo chown root: /usr/bin/transtop.sh
chmod +x /usr/bin/transtart.sh
chmod +x /usr/bin/transtop.sh

Modern versions of Transmission seem to like a fair lot of memory.
service transmission-daemon status
will sometimes show something like:
UDP Failed to set receive buffer: requested 4194304, got 327680 (tr-udp.c:84)
and:
UDP Failed to set send buffer: requested 1048576, got 327680 (tr-udp.c:95)

Transmission should still work, but adding these lines to /etc/sysctl.conf or to a file with the .conf extension under /etc/sysctl.d/ seems to make it faster:

net.core.rmem_max = 16777216
net.core.wmem_max = 4194304

At the very least, it should stop the messages in the status log.

Some explanation here.

By default, Transmission is set up to start on bootup. To prevent that, first make sure the daemon isn't running, then:
update-rc.d transmission-daemon disable

Edit cron to start and stop Transmission during off-peak hours:
sudo crontab
The first time that command is run, it will ask which editor to use. The easiest is "nano". Scroll down to the end of the file. This is what I added:
30 01 * * * /usr/bin/transtart.sh
55 06 * * * /usr/bin/transtop.sh

Those lines will start Transmission at 1:30 am and stop it at 6:55, every morning. For different times, just remember that the first figure is the minutes and the second one is the hours. Press Control-O, to write the lines out, then Control-X, to exit the file.

Within Transmission's web interface ("Alternative Speed Limits" on the "Speed" tab, under the spanner icon at bottom left), I have speeds set to zero between 6:45 am and 2:00 am (more than 19 hours). That means that, when I add entries during peak hours, downloading doesn't begin immediately. The cron job starts Transmission, which comes out of zero speeds 30 minutes later, then goes back 10 minutes before cron stops the daemon. I could fine-tune it, but that works well enough.

Settings are held in /etc/transmission-daemon/settings.json. Contents of my file are listed here. Before running the daemon, you'll need to edit it.

The first seven lines, beginning "alt-speed-" relate to the speed tab mentioned above.
The "download-dir" and "incomplete-dir" lines should be set to the directories created above.
The "incomplete-dir-enabled" line should be set to "true". That segregates fully-downloaded files from those that haven't been completely downloaded.
Setting "rpc-enabled" to "true" allows access to the web interface.
Set "rpc-password" to your plain-text rpc password. When first run, it will be run through a hash function. The hash will then replace the password in the file.
The "rpc-port" can be set to any valid port. Just try to avoid conflicts with any other software running on the Pi.
The "rpc-url" line is the last element in the url to the web interface.
The "rpc-username" can be changed, but I prefer to leave it.
Next, the "rpc-whitelist", which lists IP addresses permitted to access the web interface. Mine limits it to the Pi itself (127.0.0.1) and machines on my home network (192.168.1.*).

To access the web interface, type:
[machine address]:[rpc-port]/transmission/web/
into the address bar of a web browser. You should then be asked for your RPC credentials.
To get it to work reliably,I found that I had to give my Pi a static IP address.

Site Index

Valid HTML 4.01 Strict

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

Feedback: This e-mail address is being protected from spam bots, you need JavaScript enabled to view it.