ArchiverFS - How To Archive Old Files To The Cloud

Home | Product Home | Pricing | Download | Documentation | Release Notes | Customers

How to setup ArchiverFS to archive old files to the cloud.

ArchiverFS is capable of integrating with most cloud storage offerings. Our file system archiving solution can migrate old files to any cloud storage that can be accessed via a UNC path (e.g. \\Host\Share\ ). This includes offerings from Oracle, Amazon, Azure and others...

Before deciding to setup cloud archiving there are a couple of items that have to be considered::

- How much data do you want to migrate??

- How fast is your internet connectivity??

- How large are the largest files that users may want to access via seamless links???

All three of these items are crucial to the performance of any cloud archiving system.

The first task that needs to take place is the migration of old files to the second tier cloud based archive storage across the local internet connection. This is a case of simple math's, take the total size of files to be archived and divide that by the speed of the local internet connection. That will give you a best case scenario of how long it will take to do your initial archiving. 

ArchiverFS includes a 'pre-scan' function which can be used to calculate exactly how much data your initial archive will migrate. It's a simple feature to use, just setup your archive job exactly as you would do but tick the 'pre-scan' option. When the job runs it will scan the file system and record exactly what would have been archived if it had run for real, the figures can be used reviewed at the end of the log file.

Now that you know how much data you will archive to cloud storage you can work out based on the speed of your local internet connection how long it will take. The most common and biggest mistake we see people make when moving to cloud based computing is that they massively under-spec the speed of internet connection needed.

Next you should establish what the largest size file is that your users will typically need to access, and work out how long it will take a user to open that size file of file over the internet connection. You can set size limits on files to be migrated on a job by job basis, which may help if there are a small number of files that are significantly larger than the average. It's completely possible to archive all old files under a certain size to cloud storage, and all files over that size to local archive storage..

If your files are particularly large, if the internet connection isn't fast enough or if it will take a really long time to migrate all your files to cloud storage then you may be better off using stand alone storage on site like a NAS device.

As long as you are happy to proceed and all the above numbers add up, setting up archiving is quite simple.

Archiving over a VPN.

Once you have your cloud VM up and running with a Windows OS you can setup your site-to-site VPN. Almost all cloud providers will let you configure a site-to-site VPN between your cloud provision and your on-site infrastructure. This VPN is crucial, it will allow connectivity between your cloud service and your internal infrastructure, it is across this VPN that files will be transferred and retrieved.

When configuring outbound connections for your VPN you need to ensure that the server ArchiverFS will run on has access to it, and also ensure that all of your workstations can access it. When a workstation click on a link left behind by ArchiverFS all of the IO goes directly from the workstation to the second line storage, it doesn't go via the ArchiverFS server..

Then when configuring the inbound connections you need to ensure the cloud archive server can connect to at least one domain controller on your network. The cloud server will need to be a member of your domain if you want to maintain permissions (we are assuming you do).

Once you have setup your VPN and configured any firewall rules you should be able to join your cloud server to your domain as a member server. Next, create a test share on your archive server and ensure both the ArchiverFS server and your users can access the new test share over the VPN..

Once you have configured and tested your cloud server and VPN you should work through the User Guide or the Quick Setup Guide to complete the setup.

Archiving to Amazon Glacier via an Amazon Storage Gateway.

Archiving old files to Amazon Glacier offers almost unmatched per GB storage costs, they are roughly on par with a locally deployed NAS device (depends heavily on the specific device, service life, etc). Amazon Glacier is a fantastic product, but there is one problem...

It is difficult to integrate Amazon Glacier with on-premise services. At the time of writing, you can only interact with it via the Amazon Glacier via the Amazon console, via the HTTP API or via the archiving policies on a Storage Gateway..

The Amazon Storage Gateway ((https://aws.amazon.com/storagegateway/) allows you to present Amazon S3 storage volumes to your local network as if they were hosted on your local on-site servers. You can then setup policies to move old files from your Amazon S3 storage volumes to Glacier storage automatically. We have a special guide that covers archiving to Amazon S3 & Glacier volumes via a Storage Gateway.

Getting further help.

If you have any further questions then please don't hesitate to contact us via support@mlteksoftware.com.