How do you permanantly store a LOT of data?
Discussion
Friend of mine is a semi-pro (ust about to go full time) photographer and is struggling with storage.
He currently has 2 external 1TB HDDs connected via USB, he is thinking of buying 2x 4TB external HDDS. Problem is that there is no data backup, no relibility, and its costing him a lot o money- every couple of yeras he's shelling out £300-£400 on extrenal hard drives.
So is there a cheaper and better option?
Obviously a NAS device would be first choice- but it would need to be SATA with RAID 5 (as a minimum)- but alot of the NAS devices seem to have a limit of 4 bays that only take 1TB each. Ideally i woud like to recommend something that he could scale up to say 16TB (or more) in the coming years.
It should be a box he can spend £500-600 on now, where the discs are hot swappable, and has some sort of RAID backup, but most importantly where he can upgrade storage in the future.
ideas?
He currently has 2 external 1TB HDDs connected via USB, he is thinking of buying 2x 4TB external HDDS. Problem is that there is no data backup, no relibility, and its costing him a lot o money- every couple of yeras he's shelling out £300-£400 on extrenal hard drives.
So is there a cheaper and better option?
Obviously a NAS device would be first choice- but it would need to be SATA with RAID 5 (as a minimum)- but alot of the NAS devices seem to have a limit of 4 bays that only take 1TB each. Ideally i woud like to recommend something that he could scale up to say 16TB (or more) in the coming years.
It should be a box he can spend £500-600 on now, where the discs are hot swappable, and has some sort of RAID backup, but most importantly where he can upgrade storage in the future.
ideas?
Not for that sort of money.
ReadyNAS boxes are very good IME. Even the "prosumer" NV+ range will take 2Gb+ drives, bit still only 4 bays. They do rackmount stuff that will take much more, as do many manufacturers, but they cost a lot.
For backup, I'd have another array somewhere and simply backup/copy to that. Ideally off site. I'd also have spare drives on hand.
Costs will get lumpy though. Definitely 4 figures.
ReadyNAS boxes are very good IME. Even the "prosumer" NV+ range will take 2Gb+ drives, bit still only 4 bays. They do rackmount stuff that will take much more, as do many manufacturers, but they cost a lot.
For backup, I'd have another array somewhere and simply backup/copy to that. Ideally off site. I'd also have spare drives on hand.
Costs will get lumpy though. Definitely 4 figures.
There are lots of manufacturers out there who do network bays with over 4 HD spaces. Drobo and Qnap to name two off the top of my head although they start getting expensive once you start purchasing the necessary drives as well.
How about him picking up a cheap case, motherboard etc and just loading it with 1.5/2TB HD's and using it as just a storage device?
http://www.avforums.com/forums/networking-nas/1347...
I understand the above link is a bit more money than your mate wants to spend but it gives you an idea.
How about him picking up a cheap case, motherboard etc and just loading it with 1.5/2TB HD's and using it as just a storage device?
http://www.avforums.com/forums/networking-nas/1347...
I understand the above link is a bit more money than your mate wants to spend but it gives you an idea.
A NAS is the way to go but it is only a back-up in the sense that (assuming RAID 5 or 6 is used) it will cope with hard drive failures. If the NAS unit itself fails and another is not available then you're potentially in trouble.
There are NAS devices that take more than 4 drives; look at QNAP or Thecus. I think you'll struggle to find non-SATA units so don't let that worry you.
Good luck
Skier
There are NAS devices that take more than 4 drives; look at QNAP or Thecus. I think you'll struggle to find non-SATA units so don't let that worry you.
Good luck
Skier
Proper backup isn't cheap. £300-400 every couple of years is peanuts in the context of backup/storage.
Does he want everything available all the time or does he just need some form of long term backup?
If it's the former then other than spending a lot of money on something "enterprise" I'd have said your best bet is likely to be a manual process involving a bunch of external HDDs and multiple copies (not RAID1) of data - you can easily go out and buy a box that will store 16tb and survive a drive failure but when he f
ks up and has some finger trouble he's still lost the lot.
If it's the latter LTO tapes are cheap for the capacity they give and are arguably a much safer bet to stash in a cupboard and rely on it working when you need it.
Does he want everything available all the time or does he just need some form of long term backup?
If it's the former then other than spending a lot of money on something "enterprise" I'd have said your best bet is likely to be a manual process involving a bunch of external HDDs and multiple copies (not RAID1) of data - you can easily go out and buy a box that will store 16tb and survive a drive failure but when he f
ks up and has some finger trouble he's still lost the lot.If it's the latter LTO tapes are cheap for the capacity they give and are arguably a much safer bet to stash in a cupboard and rely on it working when you need it.
A very low spec computer in a large tower with a load of 2TB drives running this ( http://freenas.org/ ) from a pen drive??
If he gets a big enough tower (and motherboard) he can add as many drives as he wants at any time!
If he gets a big enough tower (and motherboard) he can add as many drives as he wants at any time!
ukwill said:
Whats wrong with Flickr pro?
Have you ever worked out how long it would take to upload 1.5TB of data on your typical 256kbps broadband upload speed? (I pick 1.5TB as the original poster mentioned 2 x 1TB drives almost being full and the requirements incraeasing rapidly). Just for fun work out how long it would take to upload a single 20MB file - a very conservative size for a single image in raw format.Skier
Edited by Skier on Thursday 30th December 21:06
Edited by Skier on Thursday 30th December 21:06
Skier said:
ukwill said:
Whats wrong with Flickr pro?
Have you ever worked out how long it would take to upload 1.5TB of data on your typical 256kbps broadband upload speed? (I pick 1.5TB as the original poster mentioned 2 x 1TB drives almost being full and the requirements incraeasing rapidly). Just for fun work out how long it would take to upload a single 20MB file - a very conservative size for a single image in raw format.Skier
NAS local. NAS remote that can be used to backup to regularly.
More than just disks can fail on a NAS. If the data's important, rule out as many points of failure as you can
The biggest SATA disk readily available at the moment is 3TB, but they're quite a bit more expensive than the 2TB disks.
4x 2TB in RAID 5 will give you fault tolerance on 1 disk and a 6TB partition to play with, or you could use RAID 6 which most of the new storage devices come with, which would give you fault tolerance on 2 disks and a 4TB partition.
I'd personally recommend a QNAP device for storage, i use the rack mounted ones personally and at work. The TS-419P looks to be what you want, RAID 0/1/5/6 and space for 4 hot-swappable disks, and look to start at about £300 online.
Add 4 Western Digital Caviar Green 2TB disks at £80 each brings it to ~£620.
If you want bigger, the TS-859 has room for 8 hot-swappable disks and starts from about a grand, so along with 8x 2TB disks would be about £1700 and give you 14TB in RAID 5 or 12TB in RAID 6.
4x 2TB in RAID 5 will give you fault tolerance on 1 disk and a 6TB partition to play with, or you could use RAID 6 which most of the new storage devices come with, which would give you fault tolerance on 2 disks and a 4TB partition.
I'd personally recommend a QNAP device for storage, i use the rack mounted ones personally and at work. The TS-419P looks to be what you want, RAID 0/1/5/6 and space for 4 hot-swappable disks, and look to start at about £300 online.
Add 4 Western Digital Caviar Green 2TB disks at £80 each brings it to ~£620.
If you want bigger, the TS-859 has room for 8 hot-swappable disks and starts from about a grand, so along with 8x 2TB disks would be about £1700 and give you 14TB in RAID 5 or 12TB in RAID 6.
paddyhasneeds said:
Proper backup isn't cheap.
+1 wot he said. Firstly there is no permanent option. The most permanent is onto negatives for photos. as we have 100 years of experiance in managing that medium, and hudreeds , if not thousands of years managing paper and vellum.
Secondly the photographer needs to identify:
why they need to store the data
how long they need to store the data
how they are going to structure and store the data so it can be easily pruned once it meets its end of life. assuming it has a EOL
Tape is the longest reliable long term archive option, but even then its only 5 - 7 years IIRC and then needs to be restored and put on a new tape (or refresh the old one)
The issue with any digital medium is wether it will be able to be read correctly in 50 years time. will will the tapes be able to be read by a tape drive (try gettinga LS120 drive these days) in 30 years time as tape technology will of changed. ( just go back 30 years and look a the storage devices available and wether they can be read now and the software formats of the time and whether they can be read)
Raided storage with security permissions to add only and no delete by default, will help solve the fat fingered issue identified by paddyhasneeds and replicating the storage (ie synctoy in contribute mode) or backup to tape and kept off site will help DR
Skier said:
ukwill said:
Whats wrong with Flickr pro?
Have you ever worked out how long it would take to upload 1.5TB of data on your typical 256kbps broadband upload speed? (I pick 1.5TB as the original poster mentioned 2 x 1TB drives almost being full and the requirements incraeasing rapidly). Just for fun work out how long it would take to upload a single 20MB file - a very conservative size for a single image in raw format.Skier
Edited by Skier on Thursday 30th December 21:06
Edited by Skier on Thursday 30th December 21:06
ukwill said:
Skier said:
ukwill said:
Whats wrong with Flickr pro?
Have you ever worked out how long it would take to upload 1.5TB of data on your typical 256kbps broadband upload speed? (I pick 1.5TB as the original poster mentioned 2 x 1TB drives almost being full and the requirements incraeasing rapidly). Just for fun work out how long it would take to upload a single 20MB file - a very conservative size for a single image in raw format.Skier
Edited by Skier on Thursday 30th December 21:06
Edited by Skier on Thursday 30th December 21:06
Sonic said:
ukwill said:
Skier said:
ukwill said:
Whats wrong with Flickr pro?
Have you ever worked out how long it would take to upload 1.5TB of data on your typical 256kbps broadband upload speed? (I pick 1.5TB as the original poster mentioned 2 x 1TB drives almost being full and the requirements incraeasing rapidly). Just for fun work out how long it would take to upload a single 20MB file - a very conservative size for a single image in raw format.Skier
Edited by Skier on Thursday 30th December 21:06
Edited by Skier on Thursday 30th December 21:06
Gassing Station | Computers, Gadgets & Stuff | Top of Page | What's New | My Stuff