DIY Backups – Part IV

 DIY, Linux, Programming, Technology  No Responses »
Dec 182009
 

We are finally ready to complete this series and leave you with a fully functional and cost free backup solution.

Now that you have your server setup and have an initial copy of the data you want to backup on the box, we’re ready to go back to our source machine and setup our backups.  As you saw in the last post, we’re using rsync to conduct the backups. The beauty of rsync is that it only moves changed blocks of data.  Let’s say you have a 3GB mail store such as an Outlook PST.  Each day you use your mail, the file will change.  Rather than copying the complete file each time, rsync will only copy the blocks that have changed.  Not only does this save a ton on time, but it will save a ton of bandwidth and frustration. Here’s how we set it up:

cd ~
vi backup.sh

Add the following line then save and exit:

rsync -rtvh --size-only --delete /source/volume/ xxx.xxx.xxx.xxx:/target/volume/backup/

Now let’s setup our cron job

sudo crontab -e

Now add the following:

10 */12 * * * /script/location/backup.sh

Conclusion

That’s it! You now have your own off-site backup server for which you have complete control and pay no monthly subscriptions.

Geek out and ENJOY!

~GT~

 Posted by James L2 at 9:29 am

DIY Backups – Part III

 DIY, Linux, Programming  No Responses »
Dec 152009
 

In the last two articles, we have laid the groundwork for getting our backups in place.  By now you should have your site selected, your server built and your router and firewall(s) configured. Now let’s talk about the last big preparatory item before we get to the backup itself.

So what is this last step I refer to …. snapshot!  If you don’t take the time to snapshot your data source and dump a copy of the data onto your new box, you may find yourself with a backup that runs for days or even weeks.  To save yourself a huge headache snapshot your data source and dump a copy of that data onto the backup server using some kind of USB device.  If you have a small amount of data then a thumb drive will do the trick.  If you have a large amount of data then use a hard-drive instead.

Damn the Torpedoes: One note of caution here about disk formats. If your source server is a Mac, as mine is, you are going to have filesystem issues with your external USB device on one of your machines. The solution is to ensure that your external device is formatted as FAT32.  There is a very good chance that it is, but make sure.  Once you have validated this, trudge on through to the next step.

Making the Snapshot: Plug your device into your source machine.  Assuming you are running a modern version of Linux, BSD or OS/X the kernal will see the plug-in event and may even automatically mount the device.  If the device does not automatically mount then mount it manually.  Once complete let’s use rsync to make our snapshot:

rsync -av /source/volume/ /target/volume/

GNU and All That Jazz: Just another note of caution. OS/X does not use the same version of command line tools that ship with most GNU/Linux distros.  If you are using a non-GNU Unix-like OS (e.g FreeBSD or OS/X) as your source then be aware that the arguments passed to rsync can differ.

Restoring the Snapshot: Now let’s plug the USB device into our new Linux box.  If you’re using Ubuntu Server, it will see the plug-in event but will not automatically mount the device.  Use dmesg to view the kernel buffer and determine the device name, create a mount point, mount the device and then copy our data.

cd ~
mkdir usbdrive
sudo mount /dev/xxx /user/xxxxx/usbdrive/ -t vfat -o iocharset=utf8,umask=000,shortname=winnt
sudo rsync -av –partial ~/usbdrive/ ~/backup/

Take note of the mount command I’ve used. Even though FAT filesystems are typically case insensitive, I’m telling my system to pay attention to case. If we don’t mount the filesystem this way we will, unnecessarily, copy unchanged mixed-case files from our source to our target when we first run our backup script.

Trust Me: Unless you plan to run your backups by hand, which is not a good idea, then you will need to “convince” your backup server to automatically trust your source server.  We do this by generating a key on our source and then telling the target to trust that key.  Here’s how we do it:

ssh-keygen -t rsa

This will prompt for a passphrase. Just press the enter key. It will then generate an identification (private key) and a public key. Do not ever share the private key with anyone! ssh-keygen shows where it saved the public key. This is by default ~/.ssh/id_rsa.pub. Now copy the contents of id_rsa.pub to ~/.ssh/authorized_keys on the target.

In my next, and final entry on this series, we will create the backup script and learn how to automate its execution so we are always up-to-date and safe from data loss.

Happy computing!

~GT~

 Posted by James L2 at 9:28 am

DIY Backups – Part II

 DIY, Linux, Technology  No Responses »
Dec 132009
 

In the last article we talked about the fundamental requirements for our DIY backup solution. Now that we understand what’s required, it’s time to set the wheels in motion.  As I mentioned previously, our off-site backup server will be running Linux.  For this type of workhorse function, I prefer a lightweight OS, so I chose Ubuntu Server 8.04 LTS, but feel free to install any other Linux or BSD distro.  What makes the Ubuntu server distro so great is that it ships without a graphical interface which keeps the install lightweight and clutter free.

Server Installation: Since this box will contain sensitive data, it’s important to ensure that you only install essential services.  Installing unnecessary services such as a web or ftp server only opens your server up to potential hacks.  To limit our risks, perform a basic install and only install the ssh server and nothing else.  Be sure to setup a static IP address.  If you don’t know how to do this, a quick Google search will get you the information you need. 

Server Firewall Configuration: Ubuntu server ships with a firewall but it’s disabled by default. I recommend enabling the firewall so that it disables everything and then poking holes through the firewall for the specific services we’ll need.  In our case, we’ll be using ssh and rsync.  I also recommend that you limit access to these services from a specific IP address, in this case, your home address.  Here’s how we set it up.

Login to your server then issue the following commands:

sudo ufw default deny
sudo ufw allow from xxx.xxx.xxx.xxx to any port 22
sudo ufw allow from xxx.xxx.xxx.xxx to any port 873
sudo ufw enable

Router Configuration: Don’t forget to configure your router to allow the ssh and rsync services through.  Most routers provided by ISPs have a firewall built into them that, by default, blocks all services.  We setup the firewall on our server which protects us from “inside” malicious behavior, but we will still need to repeat the process on our ISP provided router.  Setting this up is usually pretty simple.  Just login to the web interface and fiddle with the configuration until you figure it out.

Next time we will cover the steps required to make an initial data snapshot and how to put that data on our new server.  We’ll also talk about how we “convince” our new server to always trust our source server so that it will not ask for passwords when performing our backup operations.

Until next time!

~GT~

 Posted by James L2 at 9:27 am
 Older Entries
© 2012 Geek-Tips Suffusion theme by Sayontan Sinha