Warning of Data ticking time bomb
Warning of Data ticking time bomb
Author
Discussion

qube_TA

Original Poster:

8,405 posts

276 months

Wednesday 4th July 2007
quotequote all
http://news.bbc.co.uk/1/hi/technology/6265976.stm

To me this is something M$ has created, as it keeps altering its' file standards you're bound to end up with data you can't read.

Seems a bizarre approach they've adopted, to read a word 97 document they want to run Windows 95 and Office 97 as a VM just to read the file!


FourWheelDrift

92,404 posts

315 months

Wednesday 4th July 2007
quotequote all
I'm sure if the data centre that claims to have lost loads of information because it's now unreadable really wants to read it they can get someone to create a very small program to convert the data or even a program to read it. There are many talented people both professional and amatuer who can write these kinds of things.

Or am I just using my brain too much?

mcflurry

9,192 posts

284 months

Wednesday 4th July 2007
quotequote all
Surely they could load an older operating system onto a pc, and then translate the documents??

HiRich

3,337 posts

293 months

Wednesday 4th July 2007
quotequote all
FourWheelDrift said:
I'm sure if the data centre that claims to have lost loads of information because it's now unreadable really wants to read it they can get someone to create a very small program to convert the data or even a program to read it. There are many talented people both professional and amatuer who can write these kinds of things.

Or am I just using my brain too much?
Not at all. On Mac (just as the example I know best), you have:
- MacLink Plus to translate damn near every document format ever, for the grand total of £50. Add a YTS kid to batch update and burn to DVD (and you should be reburning your archive every few years anyway).
- Err, hold onto one old PPC machine with Classic installed.
- (also) Err: Should I stick those floppy disks of Quark v.2.0 in the archive store, or in the bin?
- If that pre-OS9 file is really that important, there's this thing called ebay that seems to have a lot of old computers on it.

As an aside, the British Library are being a bit disingenuous in that article. Would this be the same British Library that created the Domesday Project in the '80s, then managed to throw away the last of their BBC computers and Videodisc players that ran the software?

Podie

46,649 posts

306 months

Wednesday 4th July 2007
quotequote all
Load of hot air.

If you have documents that are that important, you have a process in place to ensure that the file format is always readable and current.

Fidgits

17,202 posts

260 months

Wednesday 4th July 2007
quotequote all
Podie said:
Load of hot air.

If you have documents that are that important, you have a process in place to ensure that the file format is always readable and current.
what he said, file formats are compatible for at least 3 generations - if you dont update the file within that time period (usually about 10 years) then its not important..

FourWheelDrift

92,404 posts

315 months

Wednesday 4th July 2007
quotequote all
I also think the concept of storing precious data on punch cards amusing hehe

Good job they now have a Colossus working at Bletchley Park.

ThePassenger

6,962 posts

266 months

Wednesday 4th July 2007
quotequote all
FourWheelDrift said:
I'm sure if the data centre that claims to have lost loads of information because it's now unreadable really wants to read it they can get someone to create a very small program to convert the data or even a program to read it. There are many talented people both professional and amatuer who can write these kinds of things.

Or am I just using my brain too much?
Patents, Copyrights and licenses. That closed source proprietary format you're trying to read is owned by someone, somewhere and they won't take kindly to you reverse engineering anything to do with it. Which in the US would be a breach of the DMCA, I belive it might fall under Computer Misuse or RIPP act's in this country as well.
Your programmer will take one look at the situation, laugh and, if he's got any sense, not touch it with a 10ft barge pole. As if that company, or a holding company with the IP rights still exists... he's liable for some hoooge fines and jail time if its ever discovered what he did.

My opinion of this whole data archiving malarky is, having talked to geeks involved in them, the bed the companies made. Us geeks know precisely how to build them, the time frames various things will last (give or take) and the formats to use: ASCII text and RGB uncompressed images. Everything from a 1960's frame to an ultra-modern PDA will read those (ok, the frame will blak at punting an image to a text only terminal but hey... beeping computers are funny). And should ASCII text ever stop being read, it's an incredibly stupid format, no compression, no tags, no formatting just CR/LF's to worry about, same with a Red Green Blue value'd image file. They're also 'cockroaches', exceptionally resiliant to corruption and read/write errors, look at it this way if 25% of your ASCII text file is duff you've got 75% good and 25% bad on your screen, a spellchecker or spud with a dictionary can repair it. 25% of your RGB values going bonkers... you've still got a good image, just with a few random pixels in it.
But no, companies seem to think bold and Italic and such are business critical as are pivot tables and pie charts; thus you end up with long term archives stuffed full of proprietary formats that have no guarantees about longevity and no real inbuilt resilience to random corruption (25% of that .doc goes... it's just gone unless you're really lucky). The chart is not important, the data that generates it is.

To go back to your example of the programmer, if the company had listened and done it right the first time, he'd be busy building a metadata based, user friendly front end to the ASCII/RGB store rather than fighting with a hex editor to figure out just WTF the long dead company were smoking when they dreamt that format up. And embedded in the store, somewhere, would be two simple little C programs (C being one of the oldest programming languages around) one that cat output text, one that can display pictures, both would be pure, raw, source code and ready for tweaking and compiling at some distant point in the future if needed.

Podie

46,649 posts

306 months

Wednesday 4th July 2007
quotequote all
Fidgits said:
Podie said:
Load of hot air.

If you have documents that are that important, you have a process in place to ensure that the file format is always readable and current.
what he said, file formats are compatible for at least 3 generations - if you dont update the file within that time period (usually about 10 years) then its not important..
In fact, how much stuff is STILL run on mainframe and UNIX boxes...

Fidgits

17,202 posts

260 months

Wednesday 4th July 2007
quotequote all
Podie said:
Fidgits said:
Podie said:
Load of hot air.

If you have documents that are that important, you have a process in place to ensure that the file format is always readable and current.
what he said, file formats are compatible for at least 3 generations - if you dont update the file within that time period (usually about 10 years) then its not important..
In fact, how much stuff is STILL run on mainframe and UNIX boxes...
in the commercial world - hardly anything (except IBM of course)...

In the Goverment and MoD.... well, i dont want to know frankly!

ThePassenger

6,962 posts

266 months

Wednesday 4th July 2007
quotequote all
Podie said:
Fidgits said:
Podie said:
Load of hot air.

If you have documents that are that important, you have a process in place to ensure that the file format is always readable and current.
what he said, file formats are compatible for at least 3 generations - if you dont update the file within that time period (usually about 10 years) then its not important..
In fact, how much stuff is STILL run on mainframe and UNIX boxes...
Royal Sun Alliance's main back end is a `frame. It's the only thing big and nasty enough to cope with the sheer volume of data hitting it. hehe
Cahoot/Abbey is/was AIX + DB2.

A lot more stuff is running on *NIX and frame backends that most people suspect.

More one for Rich1231 to answer, he does banking stuff, but I think the arse end of the stock exchange is some form of unix system. They're, again, cockroaches and don't fall over easily (if ever), banks like that smile

Edited by ThePassenger on Wednesday 4th July 13:49

Podie

46,649 posts

306 months

Wednesday 4th July 2007
quotequote all
Fidgits said:
Podie said:
Fidgits said:
Podie said:
Load of hot air.

If you have documents that are that important, you have a process in place to ensure that the file format is always readable and current.
what he said, file formats are compatible for at least 3 generations - if you dont update the file within that time period (usually about 10 years) then its not important..
In fact, how much stuff is STILL run on mainframe and UNIX boxes...
in the commercial world - hardly anything (except IBM of course)...

In the Goverment and MoD.... well, i dont want to know frankly!
You reckon..? wink

Hmmm... lick AS/400... hehe

Fidgits

17,202 posts

260 months

Wednesday 4th July 2007
quotequote all
okay, i stand corrected - i forgot about the financial peeps...

ThePassenger

6,962 posts

266 months

Wednesday 4th July 2007
quotequote all
Podie said:
Hmmm... lick AS/400... hehe
And it's big brother the RS/6000. God you find those 400's in some of the weirdest places. Little beasties (often) covered in dusty and crap... still ticking along with uptimes that'd make a Windows admin blush hehe
Once got interviewed for a cruise ship. AS/400's in the belly of the best looking after navigation, propulsion, communications and so on.

Fidgits said:
okay, i stand corrected - i forgot about the financial peeps...
And telecoms hehe

O2 & Voda are AIX back ends. True the websites are Win2k IIS based, talking to.... WebSphere (Unix) with talks to... Oracle (on a Hooooge Unix machine or frame) smile

As I said, more stuff that most people realise.



Edited by ThePassenger on Wednesday 4th July 13:57

Fidgits

17,202 posts

260 months

Wednesday 4th July 2007
quotequote all
I dunno, most telecoms people I talk to are upgrading..

but then I suppose i tend to talk to the equipment manufactuers who are making the replacement for all these mainframes with IP stuff...

ThePassenger

6,962 posts

266 months

Wednesday 4th July 2007
quotequote all
Fidgits said:
I dunno, most telecoms people I talk to are upgrading..

but then I suppose i tend to talk to the equipment manufactuers who are making the replacement for all these mainframes with IP stuff...
Thing to keep in mind, is a 'mainframe' isn't a big box in a room with a terminal bolted to it any more. It's a big box 'somewhere in the world' and you can talk to it over TCP/IP directly or treat it much like like an SQL database and have a Windows client chatting to it over 'whatever'. You might find they're upgrading away from the old serial terminals and that the frame is being relocated to a new data centre, but I'd wager good money that frame is staying put or being upgraded to a faster, more grunty... frame.

FunkyGibbon

3,864 posts

295 months

Wednesday 4th July 2007
quotequote all
Fidgits said:
Podie said:
Load of hot air.

If you have documents that are that important, you have a process in place to ensure that the file format is always readable and current.
what he said, file formats are compatible for at least 3 generations - if you dont update the file within that time period (usually about 10 years) then its not important..
In not always legacy files that are the problem - new ones can be too...

Word 2007 and its handling of equations as OMML (amongst other features) is not semantically compatible with previous versions.

http://blogs.nature.com/wp/nascent/2007/06/word_20...

A serious issue if you are in STM publishing that has lots of equations.

Podie

46,649 posts

306 months

Wednesday 4th July 2007
quotequote all
ThePassenger said:
Podie said:
Hmmm... lick AS/400... hehe
And it's big brother the RS/6000. God you find those 400's in some of the weirdest places. Little beasties (often) covered in dusty and crap... still ticking along with uptimes that'd make a Windows admin blush hehe
Once got interviewed for a cruise ship. AS/400's in the belly of the best looking after navigation, propulsion, communications and so on.
Funniest thing is when they turn off the box as part of a decommissioning job, only to discover that a load of other systems are still linked into it.. hehe


ThePassenger

6,962 posts

266 months

Wednesday 4th July 2007
quotequote all
Podie said:
ThePassenger said:
Podie said:
Hmmm... lick AS/400... hehe
And it's big brother the RS/6000. God you find those 400's in some of the weirdest places. Little beasties (often) covered in dusty and crap... still ticking along with uptimes that'd make a Windows admin blush hehe
Once got interviewed for a cruise ship. AS/400's in the belly of the best looking after navigation, propulsion, communications and so on.
Funniest thing is when they turn off the box as part of a decommissioning job, only to discover that a load of other systems are still linked into it.. hehe
Yep the old "Hey, where the fcensoredk did my billing system/customer accounts/trouble ticket system go???"

Einion Yrth

19,575 posts

275 months

Wednesday 4th July 2007
quotequote all
Podie said:
Load of hot air.

If you have documents that are that important, you have a process in place to ensure that the file format is always readable and current.
A lot of the more interesting documents from the point of view of historians tend to be those that the originators will not have considered important. And archaeologists are going to be royally screwed.