Backups, the everlasting topic. Years after several hard drive crashes, after manual backups and semi automatic backups I’m still thinking about the right solution.
Current Setup
Currently I’m using an USB / SATA adaptor to connect various hard drives without enclosure to my computer to run Time Machine semi automatic backups. With OS X 10.7 (Lion) Time Machine will add local snapshots for the time I’m not connected to my backup drive and it will write those snapshots to the backup drive once I connect it again. That is all quite nice but not an optimal solution. Over the years I’ve collected quite a few drives and with every new drive I get more annoyed by the pile they form on my desk. This is still better than no backup but it requires me to connect my drives regularly to my computer. Sometimes I do – sometimes I don’t.
NAS Options
My setup could be vastly improved by using a NAS or Apples Time Capsule. This would allow me to back up over the air with no wires and drives lying around on my desk.
I don’t like Time Capsule because it is just one drive in a plastic box with no direct access to the hard disk or the files. You can attach a recovery disk but that just makes it a little better. Only one drive also means no data redundancy. If the disk crashes the backup is gone. As far as I know there are also no checks for data integrity so if some bits flip on the disk you never know. If thats not correct please leave a comment.
When I buy something like a network attached storage I would also want to use it as a fileserver and other things and Time Capsule just doesn’t offer this kind of flexibility.
The other option I considered was buying a NAS. I really don’t like the consumer plastic boxes like the Drobo, Qnap and Synology products. On the one side they offer lots of nice features like Time Machine compatibility, web interfaces, fileserver and file sharing features. On the other side they are quite expensive, most of them are ugly, they use Filesystems like Ext3/4 or HFS+ and I also heard real horror stories of complete data losses especially with Drobo. To be fair, these stories are one ore two years old.
Then I thought about buying an »Acer Aspire easyStore H341« or an »HP ProLiant – MicroServer« and building my own custom NAS. These are basically small Atom powered PCs with four HDD slots and no display connector. They usually come with Microsoft Windows Server and require some time investment to get going. I thought about using FreeNAS as I like FreeBSD and it uses the ZFS as filesystem. It comes with a nice interface and offers everything I would need. Modern filesystem with data integrity checks, fileserver and file sharing capabilities and Time Machine compatibility.
But again, with hard drives included it wouldn’t be cheap. It would still be no real offsite backup, it could still suffer from hardware failure or theft. Besides the costs for the hardware and the time of setting everything up, there are also some costs for power as this machine would have to run 24/7/365.
One final disadvantage of all those »local« backup solutions is that I have to be home to run my backups but I also want to backup when I’m at work or somewhere entirely different.
This is why I think that those NAS solutions are not right for me.
Online Backup Services
So what are the alternatives? After coming to the conclusion that I actually don’t want a NAS device I thought about online backup services.
Of course, the first question that comes to mind is privacy and security in general. I don’t want to hand over my precious data to some company without strong and secure encryption and by that I mean that nobody but me should ever be able to get my data.
The second thing to consider is storage. How much do I have to pay for how many gigabytes? How redundant is my data stored and is it checked for data integrity?
Luckily a quick google search revealed that there are a couple of interesting options available:
All of those offer encrypted backups. The data is encrypted locally before it is send to the storage servers in the internet. All of those services have detailed informations about their architecture and features and all of them seem to have happy customers.
Personally after researching for two hours I think I will try Crashplan and here is why:
Pricing
Crashplan is cheap. Its not the cheapest but its cheap enough. You get to backup one computer with unlimited online storage for 49$/year and they have offerings for multiple computers too.
Arq and Jungledisk store the data on Amazon S3 or Rackspace Cloud which are a little more expensive than the other services with their own data centers. The client software of Arq costs another 29$.
Spideroak is carging 10$ / Month / 100 GB.
Security
All the named services offer good encryption and they seem to take similar approaches as well. The important thing is that they offer the option to use a self generated private key which is crucial for having completely private backups. Even if the police would take away all the machines they wouldn’t be able to get to the actual data. Spideroak and Crashplan explain the encryption process very detailed on their websites.
Consistency / Data Integrity
Arq and Jungledisk can use S3 which is considered to be quite save from data corruptions but there are also stories of missing data floating around. But nobody is giving you a full guaranty. Spideroark is claiming a 0.0000% error margin. Crashplan claims daily data verification and auto repair should it ever get corrupted.
The named services seem to have a good reputation of not losing data.
On the Arq website there is also a section about metadata and how the different services manage to keep track of it. The systems are tested with a software called Backup Bouncer. JungleDisk and Arq seem to be the only ones passing all tests, Crashplan fails in one test, Dropbox and Backblaze fail in 19 of 20! The section might be outdated though and since Backup Bouncer is a free tool you can verify it yourself.
Software / Integration
With every of these services comes some kind of software. Arq and Backblaze have native OS X clients while the others have mutli platform tools that do not feel like native apps. This is the only real drawback I found with Crashplan.
Extras
Interestingly enough you also get de-duplicated, compressed and encrypted backups on all these services. With Crashplan you can even choose to not use de-duplication to reduce potential cpu load on your computer while checking for duplicate data. The backups are of course differential which means that only changed data is transmitted, not entire snapshots (except the first). Crashplan allows unlimited file sizes while other services have file size limits of 4GB! It can backup locked and files and if you decide to backup your OS X unix directories Crashplan will happily do so.
Over all Crashplan seems to offer fine grained control over varius aspects of backups – which I like. Their support seems to be alright too. I’ve asked how de-duplication actually works and I got a reply within four hours on a sunday without having an account or anything.
As I said, I will try Crashplan and in addition I will keep backing up irregulary to my external Time Machine disk – just to be sure.
I know that there a a lot of other tools out there and I’m still interested in other suggestions although I’ve probably checked them out already.
You might also want to check out Wikipedias »Comparison of online backup services«
Its worth checking out the FAQs and detailed features of all those services as they usually answer most of the questions you come up with.
Lastly you can google for “Service A vs Service B” and you will get a lot of more articles like these on the web to make up your own mind.
UPDATE 1
Somebody on twitter just pointed me to this post in the Crashplan Support forum where a native mac menu bar app in beta status is available.
UPDATE 2
Thomas posted a link in the comments to a comparison matrix that he made.
UPDATE 3
Another interesting hint from the comments: Dolly Drive
Apparently they offer TimeMachine backups in the “cloud”. Unfortunately their faq is a little short on details especially on security and data integrity so I guess I will write them a mail and put the info into another post.
UPDATE 4
Several (european) readers pointed out that the upload to the crashplan datacenter is really slow, maxing out at 1.3Mbps. This is definitively one major drawback for european customers and something where Arq or other european providers could shine.
Thanks for evaluating and even more sharing! Will take a look crashplan, too. I’m particularly interested in how the’re able to do dedup and (real) client side encryption at the same time…
What about Wuala?
Any reason why you didn’t include it in the comparison?
@markus: The de-duplication is probably done on your system as well, before the data is encrypted and transmitted:
“With Crashplan you can even choose to not use de-duplication to reduce potential cpu load on your computer while checking for duplicate data.”
When I asked about it on twitter before starting my research there was nobody mentioning it. I’ve looked at it and their service doesn’t look too bad either especially because their servers are in Europe. Its more expensive than the services with unlimited storage plans and their website could be a little more informative but other than that I think its also a worthy contender.
Wuala could be great.
‘Could’ because it’s not a Mac citizen yet: It’s not running in the background, it doesn’t support essential Mac metadata and the Java-based GUI is a nightmare. De-duplication as used by Wuala also raises some questions regarding the encryption, i.e., the encryption is supposed to be secure, however, only as long as you don’t store the same files as other users.
Wuala might be more expensive than simple backup services because it offers more options including sharing and sync, i.e., Dropbox but secure.
http://wualablog.blogspot.com/2011/04/wualas-encryption-for-dummies.html
Crashplan is a good choice. I’ve been using it since about 18 months. A killer feature is the backup to a computer of a friend (for free).
I’m quite happy with arq. Using it with Reduced Redundency Storage S3 for 2 years now to Backup approx 50GB Data with multiple Versions consuming at max $5/month. What I like about it is that it just works, I don’t have to thnink about my backup anymore.
Using CrashPlan for over a year now, and I’m pretty happy with it (backed up about 230 GB).
Hi,
thank you for the quick overview. I tried almost everything in the last years and i am most happy wih Arq. It really just works. Also you can choose in which S3 region you want to store your data. If you have a fast connection, you will notice differences if you choose your closest region.
The developer gives great support and feedback on twitter, he is working currently working on Arq 2, which will be a free update.
Has the GUI been improved in the last few months?
I got frustrated with Arq because file selection was broken. It was never clear which files were actually selected. And going directly from an already selected sub-directory to its root directory, i.e., selecting both for backup, wasn’t possible either.
I agree that the support is great but GUI design is apparently not a strength of the developer. Even all the Java-based apps have a more usable GUI.
Did you look at nomadesk?
Not yet – but I will.
Be careful with JungleDisk, its encryption is not as secure as it should be:
http://www.daemonology.net/blog/2011-06-03-insecurity-in-the-jungle.html
I liked Mozy a lot. Nice Mac intergration and smooth backuping. They fucked up, as they cut the unlimited service down t 50gb storage. At the moment they have the unlimited service again. Maybe i’ll give them a secound chance.
I made a small comparison sheet on some backup-services: http://skowron.biz/artikel/backup-services/
Hi,
did you have a look at Dolly Drive? Looks really nice and seems to have a great mac os x integration.
No I haven’t, thanks for the hint!
I’ve been using CrashPlan for 2years now, use nearly 1Tb of Space and i like it a lot.
By using CrashPlan+ you can create a new version everything minute.
Perfect Solution for me.
Hi Hukl,
thanks for this comparison! I just listened to MM #69 where you mentioned Crashplan’s option of having them send you a hard-drive with all your backed up data for faster recovery. Be warned, they will only send it to US addresses (or APO addresses) as stated here: http://www.crashplan.com/consumer/store.vt .
On the same topic: Amazon has a different but similar offering called Import/Export. With Amazon S3 you send in your own disk (and enclosure), they will copy your stuff on the disk and send it back to you. Seems to work world-wide. See http://aws.amazon.com/importexport/ for details.
Just thought I’d mention this.
Best,
Stefan
Thank you! Will post another update soon
Thanks for the comparison. I just tried CrashPlan but I can’t see how that could work for me. The initial backup is a real problem.
I work at an ISP and so we have (more or less) unlimited bandwith. 😉 The point is, that CrashPlan maxed out at 1,3Mbps during upload. ^^ It would take months to finish. And what is the meantime? I don’t have any backups?
I read a lot about that particular problem where people max out at ridiculous slow upload rates compared to the possible speed. I canceled by 1 year family subscription after one day when I realized that this won’t work out for me.
I used up my trial period a few years ago so I had to take the leap and order right away. Glad they have a “cancel anytime” policy. 🙂
Greets,
Thomas
Yes, I hear that more often. I have yet to figure out if it is only slow for europeans or for us customers as well. But thank you nonetheless for the feedback. That is definitively a big drawback.
Any new information on the slow upload speed problems?
Yes, I got a mail from the crash plan support and they state that there is no bandwidth limit on their site and that slow uploads must be traffic related. There are a few settings which potentially limit the throughput but even when I removed all potential limits it was still way too slow.
Hi,
I got the same answer. And the US customers seems to have only minimal faster upload speeds. Well, I can’t say for sure, but that’s what I read during my investigations.
Greets,
Thomas
I’m also in Europe and have tried a few of these services. Whilst I want to use Crashplan I have cancelled it because, A it isn’t a mac native app and B its so slow in Europe. The same issue occurs for DollyDrive Revo which is a nice bit of kit but again slow in Europe.
I’m currently using BackBlaze, which although doesn’t keep unlimited versions is lightening fast in Europe and unlimited storage for not much. So for what its worth that is the way for me in Europe backups at 20mbps no issue to BB.
Thanks for the review! I’ve just come across another cool Crashplan feature: they’re offering to “seed” the initial backup to them. That is, they send you an empty disk with instructions and you send them back the encrypted backup. Very cool for really large backups 🙂
One point on Backblaze’s size is the nature of their company setup. Apparently, they are fully owned by the employees, no venture capital. A plus on “better capitalism”-scale IMHO.
http://blog.backblaze.com/2011/07/20/petabytes-on-a-budget-v2-0revealing-more-secrets/
well, ironically that’s funny because quoting their CEO in their own contact page:
“There are companies I love: Craigslist, IKEA, and a local sushi joint among them. I want to make sure Backblaze is one of those companies for you.”
Thanks for the review! Tried Arq, worked like a charm, restore as well. Upload of 35 GB took a weekend. Data usuage stays within the USD 5.00 per month budget on S3. Until now, all good. Decided to end trial and buy Arq instead.
Thanks for your great comparison!
If you would like to get rid of your “few drives” the linux4afrika.de project collects every drive with 20GB+
http://www.linux4afrika.de/click-to-help/requirements.html
I’m not using Crashplan’s online service but rather use it to keep a second backup on a NAS connected through a gigabit ethernet connection. My experience so far is not very positive either, as the initial backup takes ages (14 days to complete half of my 22gb backup). After a certain amount of time crashplan starts to “maintain” the backup files, which obviously takes ages. My HDDs are constantly working but there’s absolutely no progress with the backup. I mean, come on, 22gb of data is not that much on a local network.
I emailed crashplan’s support but without a response yet.
I would recommend using Iozeta. I’ve been having no problems with backup since I was using their unlimited online backup.
«Spideroak and Crashplan explain the encryption process very detailed on their websites.»
So you have absolute trust that the documentation they provide is exactly corresponding to the implementation you get?
And you have absolute trust on perfect implementation quality?
Remember Schneier’s law: http://en.wikipedia.org/wiki/Schneier%27s_Law
Any encryption which is not completely open sourced and is using well reviewed algorithms in a well reviewed implementation is not trustworthy.
There are 1000s of examples where companies simply lied, fucked up implementations, or thought it’d be great to invent their own algorithms.
Solution: choose one or two cheap online storage services, encrypt locally using established and well studied software tools and move those already encrypted data in the cloud.
I’m using Arq now. Where encryption and storage are done by different entities. That matches my personal paranoia level good enough 🙂
Hi,
has anyone tried dolly drive?
We had a very same cnofiguration at the company. Hundets of harddrives with data on them – harddrives from 1996 with 2 GB up to harddrives with 1 or 2 terabytes.
I wanted german Ordnung so i decided to build up a small server with a few harddisks running in mirrored raid. The system works perfectly and all the data is stored on it so nobody need to spend a whole working day to find data.
🙂
Is Acronis TrueImage something you would consider as an equivalent solution to, say, Cloudberry?