Using rclone to backup your cPanel backups to a remote destination

cPanel/WHM has a robust backup system that can create .tar.gz archives of your accounts, combining email, web files, databases, etc. into a single archive that can be used to restore the account in the case of emergency, or to move to another server.

What it isn’t so good at is putting them somewhere off the server to ensure that if your it dies a horrible death (multiple hard drive failures, spontaneous combustion, human error, etc.) you can restore all your accounts.  Much of the backup system depends on third-party remote mounts, Amazon S3 or FTP servers.

Worry no more!  For one of the directors of Memset, the company that employs me to do things, has created a multi-purpose transfer tool called rclone.  It can be set-up to copy or sync data to a variety of multiple destinations, including:

  • Google Drive
  • Amazon S3
  • Openstack Swift / Rackspace cloud files / Memset Memstore
  • Dropbox
  • Google Cloud Storage
  • Amazon Cloud Drive
  • Microsoft One Drive
  • Hubic
  • Backblaze B2
  • Yandex Disk
  • The local filesystem

Since this site is hosted on a Memset server, it makes sense to backup my cPanel accounts over to my Memstore account, an object storage system that uses the OpenStack Swift protocol.  While we have custom FTP and SFTP proxies, it’s important to note that you can’t upload a file that’s greater than 5Gb in size. Thankfully rclone speaks native Swift and can handle sizes beyond 5Gb.

The following assumes a basic knowledge of Linux and access to SSH as root..

So the first thing to do is download a copy of rclone for your server.  Most people will be running a 64-bit Linux, so you’ll need to download the tarball for that.    The next step is to unpack the archive and install the binaries and manpage as per the instructions.  Skip the sudo parts if you’re on cPanel – it’s not needed, so:

Now run:

You’ll see something like this:

Press ‘n’ for New Remote.  You’ll then be prompted to give this a name.  You can call it whatever you like.  In this example I’ll be using the name ‘memstore’.  Once you’ve given it a name, you’ll be prompted for the storage type.  In our example, it’s OpenStack (number 10):

I’ve created a user within my Memstore/Memset account control panel called “cpanel” that I’ll be using to connect to the Memstore container “cpaneldemo” that will hold my backups:

2016-04-01_11-08-20

I then assign read and write permissions for user “cpanel” to the container “cpaneldemo”:

2016-04-01_11-10-23

Now to configure rclone:

So the username is the right-hand part of the Memstore username, and the tenant is the left-hand part (e.g. msdrakeab2.cpanel becomes user = cpanel, tenant = msdrakeab2).  The key (or password) will be displayed in plain text at all times, and is stored within the /root/.rclone.conf file.  Make sure that only root has permission to read this file – it should do by default, e.g.:

So we’re ready to rock and roll.  We don’t have any data in the container, but we can give a quick test to make sure we’re able to connect:

All seems to be working.  So let’s manually move some backups to Memstore.  Memset configures cPanel backups to be dumped to /backup on your server.  So based on that, the initial upload will look like this:

When we look at the contents of the container through the Memset account control panel:

2016-04-01_11-43-58

How do I retrieve backups?

Very easily done.  Let’s say we want to grab the account called ‘mice’ that was backed up on the 31st.  In the cPanel backup hierarchy, it’ll look like this:

So to get that back from Memstore, we’d do this:

where /tmp is the local filesystem where you want the file to be placed.  Can be anywhere on the filesystem.   You can leave out the file and have the entire contents of the ‘accounts’ directory transferred too (although in this example, there is only one file in ‘accounts’):

How do I automate the backups?

Simple, just add it as a cron job:

which will run at 1:30am and will dump the output to /var/log/rclone.log.

Other ideas

You could use rclone to create historical backups within Memstore, handy if you keep a set of daily backups that you’d like to keep around longer than the cPanel keeps them for on your server’s filesystem.  To do this, you ensure you have a destination container to sync to.  So let’s create one:

then sync the contents of one container to another.  Note that all of this is done on Memstore – no data is transferred from your server to Memstore (or vice versa).

The following example demonstrates a sync of the existing data in the “cpaneldemo” container to the new container “cpdev”.  I could  automate this by adding a cron job to sync data from “cpaneldemo” to “cpaneldev” on a weekly basis, for example.