Automating Remote backups of important data with SCP & Bash.
at some point I had to get some files that change regularly backed up, those files were on a remote server and I wasn’t always in front of the computer to get it done.
so there are (few) solutions for that matter.
let’s say this is the topology of my system:
Let’s say I have a file server with the address of fileserv.penguin.com and I’d like to retrieve and backup a specific directory from that remote server to a dedicated backup server with the address of backup.penguin.com.
This task can be done in several ways, but it won’t be as effective as making it all automatic without user interference now would it? ;)
Sure, you can copy files over the net with scp, but the whole point of this post is not to take any actions in order for the backup to go through, and copying with scp requires user/passphrase.
So for that Issue we need to get several things done:
on the fileserver, issue the following:
ssh-keygen -t rsa # This will generate public/private rsa key pairs
you will be prompted for for file save, press enter.
Then, you will be prompted for passphrase, pressing enter again. this is for no passphrase (this is what we need in our case)
afterwards, you will see a message that your public/private key pair were saved (in /home/<user>/.ssh/id_rsa.pub)
Now, what we have is two different files called id_rsa and id_rsa.pub on our file server (fileserv.penguin.com)
At the moment, our task is to get this id_rsa.pub file to the computer it needs to react with, that would be our backup server.
issue the following command to get it with scp (*notice that if authorized keys file contain keys of other PCs, make sure to APPEND the key so you won’t demolish the file!)
scp /home/<user>/.ssh/id_rsa.pub firstname.lastname@example.org:/home/<user>/.ssh/authorized_keys
The blue part is the remote server we ssh’d to, and the red part is our backup server.
after issuing this command you will be prompt for user credentials.
now we should be able to ssh without the server asking for a passphrase ;)
and now for the artistic part of the post:
Automating it with bash
echo “backup log for `date`” >> $LOG
#Remote Backup Server
SUBJ=”Backup from fileserver for `date`”
echo “[1/6] Compressing wordpress directory” >> $LOG
tar cf $TAR$EXT.tar $DIR
echo “…done.” >> $LOG
echo “[2/6] Moving tar file to the local backup dir” >> $LOG
mv “$TAR$EXT.tar” $LDIR
echo “[3/6] Calculating WordPress compressed file size… `ls -lh $TAR$EXT.tar | cut -d’ ‘ -f5`” >> $LOG
echo “[4/6] Preparing to send the compressed file…” >> $LOG
echo “[5/6] Establishing a connection to the remote computer & copying files…” >> $LOG
scp ”$LDIR/$TAR$EXT.tar” “$SERVER:$RDIR” > /dev/null 2>&1
echo “[6/6] Mailing results to admin…done” >> $LOG
mail -s $SUBJ $RCPT < $LOG
if [ "$DAY" -eq 6 ]; then
This very simple script does the following:
- Compresses the desired backup folder
- SCPs it (without any user involvement)
- Mails the size of the compressed output file together with the rest of the log to the admin’s mail address
- executes on saturdays *
* can be changed according to your needs.
the log file will eventually look like this ( you will receive it by mail ;) )