ArchiverFS - Choosing The Best Archiving Software For Your File Server

Home | Product Home | Pricing | Download | Documentation | Release Notes | Customers

Why can't we just be tidy?

Everyone who has worked in the field of IT infrastructure for more than a few years is bound to have faced the issue of a congested file server. It doesn't matter what infrastructure you have, Windows, Linux, Apple.. the same problem always crops up. The big challenge is how do you deal with it? How do you stop the sprawling growth of the file system in the first place, and how do you deal with it once it has happened? Do you deploy a full blown document management system and force people to use it, or do you try to implement some kind of product to archive your old files to second line storage like a NAS device where they can then be retired to long term storage?

These days most file archiving solutions could easily be re-branded as a document management systems, there is often little difference between the two. Companies frequently produce a product that could easily be renamed and perform either role. There is typically some kind of proprietary file storage mechanism involved (a database or an encrypted container), added features like version control, search functions and a bunch of other 'stuff' that all adds up to make what should be an efficient light weight product much more complicated and expensive than it needs to be.

If you wanted a full blown document management system then you'd buy one and store ALL of your companies files in it, not just the old ones! Yet with a lot of software that's exactly what you are expected to do. Pay out for an expensive re-purposed document management system, then store all your old files in it. To us this just seems crazy.

Even if you did use a full blown document management system, and lets be realistic here... you will more than likely still have a share somewhere on your network that contains a mixture of user profiles, departmental folders, temporary areas (network copier scans anyone?) and a gazillion other files in a mish-mash of folders.

The real problem isn't the technology; the problem arises due to the natural inclinations of the people using it. Human beings are messy and cautions animals. We like to keep multiple copies of things as we work on then, just in case we need them. What's more we like things that feel familiar, just look at your own desktop. Are all of your icons, files and shortcuts ordered in a nice list from A->Z from left to right across your screen? Nope? Neither are mine!

Everything is grouped together in different bunches. I have all my development apps on one side of the screen, productivity on the other side, with general utilities and a few games in another group. There's no real order to any of it. My user folders aren't any better as I have at least 3 copies of anything that is important (current, previous and one from a few months back just in case). I'm perfectly happy with things that way, it feels comfortable. What's more accessing anything I need is at most 3 clicks away, normally 2 and often just 1.

The network file share is here to stay.

Based on conversations we have had with our customers and from personal experience, it's safe to say that people don't like document management systems. Give most people the choice of storing their items in a folder in a share on a server or uploading them to a document management system, and most people will choose the folder option. Document management systems are typically seen by users (at least initially) as something that restricts their freedom and makes what should be simple tasks complicated. For example, instead of just clicking 'File->New Document' to create a new file, the end user is typically forced to make at least 5 extra decisions and perform multiple extra mouse clicks.

In the time pressured world we live in people are more and more inclined to do things in whichever way saves them the most time.

There is a reason the humble file system has been with us since the early days of computing and why it isn't going away any time soon.... it just works. A network file system is the cheapest, most reliable, and highest performing way to store user generated files, period. There are an almost uncountable number of alternatives... Sharepoint from Microsoft, Zoho Docs from Zoho Corporation, even Google Drive for Business to name a few, but none of them can match the simplicity of a network folder form a users perspective.

Even the big names like Google and Dropbox seem to appreciate this, why else would they release desktop apps that give users access to their services via a local folder on their machine instead of forcing them to access their services through a web interface?

All of this adds up to one inescapable fact. Network file systems are here to stay for the foreseeable, at least until someone comes up with a way to store 100TB of files as cheaply and efficiently as a network file share can!

It's not all bad news, the 'evolving' file system.

File systems have evolved a LOT since the early days. If you want to know more about the history of the ever evolving file system ARSTechnica have an excellent article on the subject here. As the file system has evolved so have the tools to manage it. These include backup tools like Windows Backup and some truly excellent third party products, search utilities including the brilliant and completely free Microsoft Search Server as well as Antivirus products and even quota management.

Technologies like DFS (Distributed File System) and deduplication have helped elevate the humble network share to new heights, turning it into a powerful, cost effective enterprise level tool. The solitary file server sat in the corner of the office really has come a long way.

Of course, like any comic book super hero the ubiquitous file server share has its weakness, and that is it's flexibility. It is the very thing that makes it so popular.

The problem.

The very thing that makes a file share on a server so easy to use also keeps many System Administrators up at night. Users can typically create whatever they want, where ever they want. There will normally be a pre-defined folder structure in place but within that users can have free reign. If they want to keep multiple versions of of old files 'just in case' they can, and they often do.

This is great for the user, but what about compliance issues? The fifth principal in the UK Data Protection Act states.

"Personal data processed for any purpose or purposes shall not be kept for longer than is necessary for that purpose or those purposes"

How on earth does a company ensure they comply with this requirement (and similar ones in other countries) when it comes to the average file system? Based on the feedback we have had from our customers we estimate that on average over 65% of the contents of the typical file system hasn't been accessed in at least 3 years. It is a pretty safe bet that if a file hasn't been used in 3 years than it is probably not needed any more!

All those extra files also have a direct financial cost. You need to backup more data, which is ironic as users tend to keep multiple copies in case they need a previous version. But any IT department worth their salt will keep multiple copies on various backups.

In addition, server hardware costs more. You need bigger SAN's with more disks, faster networks etc. and that gets expensive fast.

Whats the anwser?

There are several options when it comes to choosing a suitable product to deal with the old files on your file servers. These include our product ArchiverFS. Obviously we recommend that you give it a try. It includes everything you could possibly want in a file lifecycle management tool and it can handle huge file systems (the largest is currently 3.4 PetaBytes!). It is incredible value for money, has a large user base made up of companies like Wells Fargo and GEA, it can even migrate old files to cloud based storage. What's more there is no file storage database or proprietary storage mechanism, it is NTFS from start to finish.

We believe it is so good and provides such a great cost\benefit ratio (especially with file systems over 10TB's) that we will happily provide links to other products in the same field. We are confident that if you compare other offerings to ArchiverFS you will see that there is nothing quite like it.

• ArchiverFS - MLtek
• Open Archive - Open archiving project.
• Messagesolutions - Hybrid document mangement system.
• Waterford Technologies - Archival utility from a suite of products.
• Hub Stor - Azure based archiving product.

Should you wish to take a look at ArchiverFS you can download a copy of the software here. ArchiverFS comes with a free licence that includes all of the features but has a limit of 50 files processed each time a job runs.

If you have any questions then please don't hesitate to contact us via sales@MLtekSoftware.com, we would love to hear from you!