ArchiverFS - Moving Old Files To Amazon Glacier

Home | Product Home | Pricing | Download | Documentation | Release Notes | Customers

How to use ArchiverFS to archive old files to Amazon Glacier.

Amazon are one of the biggest players in the online cloud computing sector, and amongst the many services they offer is a storage tier called 'Glacier'. Amazon Glacier is designed to provide cheap storage for cloud users, it is excellent value for money and it makes an excellent storage medium for a file system archiving solution if you are looking for a way to archive your old files to the cloud.

At the time of writing storage costs start from $0.004 per GB per month and it's easy to see why this would be attractive to businesses looking to place data into cheap long term storage.

There is one major issue though, and that is you can't mount an Amazon Glacier volume to a machine and share it to the network like you can with a normal volume. Normally you can only move data to a Glacier volume via either an API, the Amazon Console tools or via a storage and migration policy on an Amazon Storage Gateway. This makes it really hard to use Glacier to provide archive storage for an on site file server..

By working in tandem with an Amazon Storage Gateway, ArchiverFS is capable of archiving your old and outdated content seamlessly from your live file servers to an Amazon volume hosted on a Storage Gateway. From there it can be migrated to Amazon Glacier automatically for long term storage..

How does it work?

By following the steps below you will be able to archive the old and outdated files on your file server to Amazon Glacier, via an Amazon Storage Gateway.

Step 1 - Setup your storage gateway: First of all, you should have a look at the Amazon Storage Gateway getting started guide. This will walk you through the setup of a Storage Gateway.

Step 2 - Create your archive share: Next you'll need to create a share on the Storage Gateway that will be presented to the network. It is this share that you will configure ArchiverFS to move old files too, and it must be accessible across the network as a UNC path (e.g. "file://server/share"). The share will most likely need to be AD integrated and Amazon have created this guide which explains how to setup the share.They also have a dedicated guide explaining how to enable Windows ACL's on a Storage Gateway SMB share as by default it uses POSIX permissions here, but we recommend you test permission persistance as we have had mixed reports from customers.

Step 3 - Setup your Amazon lifecycle migration policies: These are the policies that will move files from your archive share (the share that ArchiverFS archives too) off to Amazon Glacier. Like the previous steps, Amazon have a good guide to setting up a lifecycle policy. We suggest moving files to Amazon Glacier once they have been sat on the archive volume for a few months, but it is totally up to you how you configure this.

Step 4 - Setup ArchiverFS: Assuming you already have a configured and working copy of ArchiverFS, the last thing you'll need to do will be to setup a new archive job and configure the correct maintenance job settings. When setting up your new archive job choose either 'Move (No Links)', 'Move (Per File .Lnk Shortcuts)' or 'Move (Per File Hard\Symbolic Links)' as the desired action, set the target to be the new share you created on the Storage Gateway. If you selected 'Move (Per File .Lnk Shortcuts)' or 'Move (Per File Hard\Symbolic Links)' then open up the maintenance job options and tick the 'Delete links in the live file system that point to non-existent archived files' option. Ticking this will enable ArchiverFS to remove links in the live file system that point to files which have been moved to Amazon Glacier by the Storage Gateways storage policy.

What Next?

If you haven't already got a copy of ArchiverFS setup then we'd recommend having a good read of the QuickStart Guide and\or the full User Guide before installing ArchiverFS.

You can grab a copy of ArchiverFS from the download page here.