Video encoding for archival - format/codec?
Discussion
Hi all,
It was a toss up between posting this in Photography and Video and here but I thought it might be more suited to the more technical forum.
What is the consensus on a video encryption for archival. The source is DV and I want to keep the best quality I can but with an eye on file size. I also want to be able to re-encode the files as new codecs come along in the future as the archival will be "for ever". Given this, I'd like the most "loss-less" format as each re-encoding will, presumably, result in more and more accumulated loss.
Now, these are home movies (of my son before you get any dirty ideas!) so I don't care about broadcast quality but it'd be nice to have the best quality I can without having to have loads of storage space.
Having said that, storage space is cheap and getting cheaper, isn't it? Maybe original DV will be the best option?
So - original DV? MPEG-4? MPEG-2?
What's your thoughts?
Cheers,
Phil.
It was a toss up between posting this in Photography and Video and here but I thought it might be more suited to the more technical forum.
What is the consensus on a video encryption for archival. The source is DV and I want to keep the best quality I can but with an eye on file size. I also want to be able to re-encode the files as new codecs come along in the future as the archival will be "for ever". Given this, I'd like the most "loss-less" format as each re-encoding will, presumably, result in more and more accumulated loss.
Now, these are home movies (of my son before you get any dirty ideas!) so I don't care about broadcast quality but it'd be nice to have the best quality I can without having to have loads of storage space.
Having said that, storage space is cheap and getting cheaper, isn't it? Maybe original DV will be the best option?
So - original DV? MPEG-4? MPEG-2?
What's your thoughts?
Cheers,
Phil.
I've been involved with this kind of stuff (although archival storage of financial records, rather than video) and the answer is that there's no good answer. You have two problems, which you've already quite rightly identified - format (the physical media) and standard (codecs).
Both of these go out of favour, and quite rapidly. (Tried to buy a Betamax VCR lately? And even VHS is beginning to disappear.) There are people making reasonable livings recovering data from obsolete media, and I personally have archives all the way back to 8 track paper tape. Because of the difficulty of finding and maintaining obsolete hardware, I would suggest that you keep your archive on "soft" media (disk) and make several copies (perhaps some hosted on the Internet). Keep at least one of them "off site" so if your house burns down you don't lose it.
As for codecs, I can only suggest using a popular one and keeping copies of the software used to manipulate it and the specs defining it. There are going to be Wintel computers with us for several decades to come, so it makes sense to use one of those.
Every few years, review your strategy and copy the archives to new format & standard.
I suspect lots of people are going to lose their family history because of failing to do things like this. There are a lot of official archivists worried about it!
Both of these go out of favour, and quite rapidly. (Tried to buy a Betamax VCR lately? And even VHS is beginning to disappear.) There are people making reasonable livings recovering data from obsolete media, and I personally have archives all the way back to 8 track paper tape. Because of the difficulty of finding and maintaining obsolete hardware, I would suggest that you keep your archive on "soft" media (disk) and make several copies (perhaps some hosted on the Internet). Keep at least one of them "off site" so if your house burns down you don't lose it.
As for codecs, I can only suggest using a popular one and keeping copies of the software used to manipulate it and the specs defining it. There are going to be Wintel computers with us for several decades to come, so it makes sense to use one of those.
Every few years, review your strategy and copy the archives to new format & standard.
I suspect lots of people are going to lose their family history because of failing to do things like this. There are a lot of official archivists worried about it!
Always a tricky one, this.
The problem with any kind of computer based file (Mpeg etc) is that you are compressing the file for file size reasons, but you do compromise the quality (albeit very slightly with some of the fancy new codecs that are available)
Depends what your budget is, and what the archive is for.
We archive all of our work onto tape in Digital Betacam format. Which ain't cheap. But the quality is superb.
If you wanted to archive onto a tape format on a budget, I would suspect that DV or DVCam would be a suitable format based on cost issues. I also suspect that you will be able to get equipment to view DV-based tapes for many years to come, as it is currently so widely used.
As for computer based formats... Full-Frame Uncompressed AVI or Full-Frame MOV would probably be the best bet for retaining high quality, but the file sizes will be huge.
The problem with any kind of computer based file (Mpeg etc) is that you are compressing the file for file size reasons, but you do compromise the quality (albeit very slightly with some of the fancy new codecs that are available)
Depends what your budget is, and what the archive is for.
We archive all of our work onto tape in Digital Betacam format. Which ain't cheap. But the quality is superb.
If you wanted to archive onto a tape format on a budget, I would suspect that DV or DVCam would be a suitable format based on cost issues. I also suspect that you will be able to get equipment to view DV-based tapes for many years to come, as it is currently so widely used.
As for computer based formats... Full-Frame Uncompressed AVI or Full-Frame MOV would probably be the best bet for retaining high quality, but the file sizes will be huge.
I dont think this is as big a concern as you think. Firstly if you choose to archive to PC then the popular codecs ie mpeg2 offer good size/quality and will be around for longer than you or I care to worry about. Yes they will be superceeded(check spelling) with other codecs or compression technology but lets not forget the original file was on a mini DV! Not great to start with. You will be able to find a suitable program to play Mpeg2 files for years to come. Secondly are you looking to wipe the original DV tape? I would keep that too as a backup of the original. Mini DV wont be around for ever but I am sure you would always find a way to copy it again in the future in an emergency.
road2ruin said:
but lets not forget the original file was on a mini DV! Not great to start with.
Bit harsh really.
Depends on how good the DV camera used to shoot the footage is. You can spend £100 on a DV camcorder, you can also spend £6k on a DV Camcorder, then stick a £10k broadcast lens on the front of it. The results you get will be pretty damned good.
DV is now used regularly as a broadcast quality format. I'm not necessarily saying that it IS broadcast quality, but broadcasters accept it as a quality format.
road2ruin said:
I dont think this is as big a concern as you think. Firstly if you choose to archive to PC then the popular codecs ie mpeg2 offer good size/quality and will be around for longer than you or I care to worry about. Yes they will be superceeded(check spelling) with other codecs or compression technology but lets not forget the original file was on a mini DV! Not great to start with. You will be able to find a suitable program to play Mpeg2 files for years to come. Secondly are you looking to wipe the original DV tape? I would keep that too as a backup of the original. Mini DV wont be around for ever but I am sure you would always find a way to copy it again in the future in an emergency.
Yeah, my leanings were that mpeg2 is good enough for DVD so it should be good enough "for now" - but what I didn't want was to have to keep re-encoding and losing quality. "Years to come" for each codec might not be that long - but so long as I don't jump for each new codec/standard as it appears it will be long enough each iteration... I guess. Yep - the original is mini-DV and, as you say, not great - hence worrying about the loss of quality each time.
I guess my concern is that I will need to worry about this for at least 20 years. Now, 20 years ago VHS would have been the format and whilst that's still available to some extent, I think the rapidity of change in video encoding has accelerated in recent years - and I suspect will continue to do so with the demand for better quality/smaller files "on line". I wonder whether formats will become obsolete in shorter time spans. Maybe not - essentially I only need "worry" about consumer codecs (i.e. mpeg2 at the moment, mpeg4 for Sky HD - as I understand it).
I think I'll plump for mpeg2 (DVD, right?)
As for the tapes. Yeah, I want to wipe them. Certainly don't want to worry about them. They're pretty fragile and awkward to use when compared to electronic copies. Also, I've had a couple of tapes get chewed up by the camcorder (cost £1200 back in '99 and is/was a pretty good machine but is not without it's faults).
freddies_dad said:
Yep - the original is mini-DV and, as you say, not great - hence worrying about the loss of quality each time.
But if you transfer from DV to a PC (for editing) via firewire, then transfer back to DV tape after editing via firewire, (Assuming you can "modify" your camcorder to accept DV-in) then there is no loss in quality. Subsequent firewire copies will not give any quality drop either.
That is one of the benefits of the digital age!
meeja said:
freddies_dad said:
Yep - the original is mini-DV and, as you say, not great - hence worrying about the loss of quality each time.
But if you transfer from DV to a PC (for editing) via firewire, then transfer back to DV tape after editing via firewire, (Assuming you can "modify" your camcorder to accept DV-in) then there is no loss in quality. Subsequent firewire copies will not give any quality drop either.
That is one of the benefits of the digital age!
Sorry - I meant the re-encoding each time would introduce loss. I wasn't planning on backing up to DV, I wanted to store a compressed version on HDD somewhere and move it around as-and-when. If you see what I mean.
freddies_dad said:
meeja said:
freddies_dad said:
Yep - the original is mini-DV and, as you say, not great - hence worrying about the loss of quality each time.
But if you transfer from DV to a PC (for editing) via firewire, then transfer back to DV tape after editing via firewire, (Assuming you can "modify" your camcorder to accept DV-in) then there is no loss in quality. Subsequent firewire copies will not give any quality drop either.
That is one of the benefits of the digital age!
Sorry - I meant the re-encoding each time would introduce loss. I wasn't planning on backing up to DV, I wanted to store a compressed version on HDD somewhere and move it around as-and-when. If you see what I mean.
Understood.
Always going to be a difficult one, that... because if you are going to use a compressed format (and any encoded format has compression) as your master to encode from, then you will get an effective generation loss.
However, as I think has already been mentioned, if you go for one of the best current encoding formats, and have copies of all of the encoding/playing utilities then you should be ok.
It still may be worth keeping an archive of the original on tape though... even if it is locked away in a cold dark room somewhere.... that way, you are keeping an original "high quality" archive - just in case. Even if you don't ever use it again, it's always nice to have the ability to view a high quality master should you ever need to.
DV tape costs these days (even high quality ones) are not prohibitively expensive.
Gassing Station | Computers, Gadgets & Stuff | Top of Page | What's New | My Stuff