Monumental work cockups

Monumental work cockups

Author
Discussion

Glade

4,266 posts

223 months

Saturday 15th October 2016
quotequote all
I work in Quality Assurance so i'm heavily biased/bitter about this biggrin

My time is usually spent figuring out and rectifying issues caused by lack of rigour in design or discipline in operations. Once you have a grip on the problem, and correct it, you have to give the customer a big cuddle... then internally be a pain in the arse about process improvement... But mostly get ignored because it's "Low risk", and sounds like hard work (the people who fked it up usually don't feel the pressure to change in future). Rinse and repeat.

The truth is it's not that hard to follow a structured process for pretty much anything to do things right first time, but there is understandable pressure to bring products/services to market quickly. But when you take enough shortcuts, eventually one catches you out.

Chronic things that affect a manufacturing company are usually lack of rigour in early stages of a product's lifecycle. That causes compliance issues, poor hardware design and poor industrialisation. These can end in recalling 1000's of products, invariably followed by £100,000's of site rectification and the occasional £1M-3M law suit.

Added to those general small recurring issues, st design or industrialisation causes slow haemorrhaging of cash via factory scrap and failure rate & warranty claims. Late design changes cause excess inventory, and shortages of what you do need. This can easily contribute to millions in cost of non quality each year, obsolescence - ultimately low margin and poor cashflow. Poor delivery then causes lack of sales and down the spiral we go. Add the occasional major fk-up in production - perhaps a missed heat processes, dodgy counterfeit components in the supply chain etc i could go on and on and on!

There's definitely lots of ways to fk things up, and even the best companies have some really good ones you get to hear about from time to time. In companies under financial stress it gets worse and worse. Horsemeat in your lasagne? ha no st, that's probably not the worst of it either wink

Joys of manufacturing... however it keeps me employed and the stter a company is to start with the easier it is to make big savings, by doing really simple things biggrin


Edited by Glade on Saturday 15th October 03:10

Gun

13,431 posts

218 months

Saturday 15th October 2016
quotequote all
I worked on a project a few years ago during which anything that could go wrong, did. This resulted in it being handed over about 6 months late, with a penalty of £200k for every week that we ran over the proposed completion date.

GT03ROB

13,262 posts

221 months

Saturday 15th October 2016
quotequote all
There was a project I was on the periphery of once. It was completed "successfully". The company then decided to use the project as a model for the new graduate intake. So over a 4 month period the graduates used the project data to run there own project. At the end of the project they presented their results. These results differed significantly from the results of the original project. So to help them see the error of their ways the "experts" went in to assist. The "experts" came up with the same results as the graduates. After much scratching of heads it slowly dawned that the original project had made a monumental cock-up in certain calculations that the graduates had now got right. Fortunately there was no long term damage & the error was corrected before the client could pick it up.

brickwall

5,250 posts

210 months

Saturday 15th October 2016
quotequote all
Personally, nothing that hasn't been rescued by my seniors before it got too serious. Now I am more senior, obviously I have to do more rescuing...

These two examples (not mine) stand out:

List of Excel cockups that make it into the press:
http://www.eusprig.org/horror-stories.htm
Don't know if East Coast Main Line is on there, but I pity whichever (ex..)analyst did that one.

Dropping a £50k piano off the back of a delivery truck:
http://www.company7.com/bosendorfer/april2007_275d...

PositronicRay

27,010 posts

183 months

Saturday 15th October 2016
quotequote all
cossy400 said:
PositronicRay said:
Peugeot bought a site in coventry for a new parts distribution depot ....................... only to discover the site was designated by the council for manufacturing only.
Torrington Ave??
No Torrington Ave was the original site, they had to stay, instead of moving to Browns Lane.
The old Ryton plant had been sold on at this point, otherwise part of this would have been ideal.


Edited by PositronicRay on Saturday 15th October 08:24

eltax91

9,871 posts

206 months

Saturday 15th October 2016
quotequote all
Mine comes from when I was doing my placement year at a large gypsum products manufacturer.

I was testing the boss' PA's out of office as it wasn't working, and of course she dare not leave that Friday without the world knowing she was off on the Monday rolleyes

Anyways, couldn't get it to work on the PC so went onto the server admin console. Being young and inexperienced, I didn't notice the check box in exchange that says 'send only one reply per recipient' and being even more dumb I decided the best way to test this first was to copy the rules to my profile too.

Then I sent her an email.... from my email address. Cue the email server sending reply after reply to our two mailboxes. This is around 2003/4, so everything ran on real tin and nobody had failover and the server fell over.

1200 people without email, in a business where the vast majority of ordering was over email. The call centre went into meltdown with building firms shouting the odds as they had not received or confirmations and for the first time in its history the call centre had to work Saturday morning calling all the builders that couldn't get through to take their orders the old fashioned way.

No my finest hour.

Eric Mc

122,007 posts

265 months

Saturday 15th October 2016
quotequote all
eltax91 said:
Mine comes from when I was doing my placement year at a large gypsum products manufacturer.

I was testing the boss' PA's out of office as it wasn't working, and of course she dare not leave that Friday without the world knowing she was off on the Monday rolleyes

Anyways, couldn't get it to work on the PC so went onto the server admin console. Being young and inexperienced, I didn't notice the check box in exchange that says 'send only one reply per recipient' and being even more dumb I decided the best way to test this first was to copy the rules to my profile too.

Then I sent her an email.... from my email address. Cue the email server sending reply after reply to our two mailboxes. This is around 2003/4, so everything ran on real tin and nobody had failover and the server fell over.

1200 people without email, in a business where the vast majority of ordering was over email. The call centre went into meltdown with building firms shouting the odds as they had not received or confirmations and for the first time in its history the call centre had to work Saturday morning calling all the builders that couldn't get through to take their orders the old fashioned way.

No my finest hour.
Sounds bad but I still haven't a clue what you are describing.

FN2TypeR

7,091 posts

93 months

Saturday 15th October 2016
quotequote all
I connected a motor up incorrectly several years ago when I was an apprentice, it wasn't worth a colossal amount of money, a couple of grand, but being a lowly apprentice I was crapping myself - alas, nothing ever came of it, that I know of anyway and I'm not even sure if the supplier sent us a bill.

It had been supplied to us by a company called Becker as a trial unit as we were looking at purchasing quite a few of them for a machine we were to be building, after a few moments that awful burning smell associated with things like motors and transformers burning out started.... ooops!

Robbo 27

3,634 posts

99 months

Saturday 15th October 2016
quotequote all
Worked at some cement works and we had a pier for unloading from ships. The insurance brokers were asked to insure the pier and we included impact. A boat hit the pier. They had insured the pier just for impact by vehicles.

Went to a meeting, six people, 4 of them, 2 of us. Meeting started at 5 and we knew it would be a 2 hour job. I was doing most of the talking in a Q and A session. The other 4 were making loads of notes. I had a director of the company with me, I nodded to the four and then to his blank piece of paper. He started making making notes.

In the morning I started doing the Minutes.

"Andrew, have you got the minutes from yesterdays meeting?"

"Oh , Ah you see"

"What do you mean? You saw all the note taking, you did take notes didnt you?"

"I just pretended"

Could have killed him.


EC225Eng

75 posts

162 months

Saturday 15th October 2016
quotequote all
I've binned a Makila 2A1 jet engine by shearing a bolt in the combustion chamber casing that wouldn't be drilled out cleanly. It sheared going back in, I really should have checked the condition of the bolt. Cue the engine going back to supplier, losing a day of North Sea oil and Gas support flying and all the hours of rework and testing. No action taken against me as doughnuts were bought and no more said. They reckon the total cost including aircraft downtime, penalties, securing another engine and testing it was well into six figures!

rossub

4,442 posts

190 months

Saturday 15th October 2016
quotequote all
QS working for a construction company doing a project for us discovered he had entered £150k into a spreadsheet cell instead of £1.5m...on a job that had already been approved by our board. The type of contract meant that we had no option but to suck it up.

He wasn't popular.

lufbramatt

5,344 posts

134 months

Saturday 15th October 2016
quotequote all
Eric Mc said:
eltax91 said:
Mine comes from when I was doing my placement year at a large gypsum products manufacturer.

I was testing the boss' PA's out of office as it wasn't working, and of course she dare not leave that Friday without the world knowing she was off on the Monday rolleyes

Anyways, couldn't get it to work on the PC so went onto the server admin console. Being young and inexperienced, I didn't notice the check box in exchange that says 'send only one reply per recipient' and being even more dumb I decided the best way to test this first was to copy the rules to my profile too.

Then I sent her an email.... from my email address. Cue the email server sending reply after reply to our two mailboxes. This is around 2003/4, so everything ran on real tin and nobody had failover and the server fell over.

1200 people without email, in a business where the vast majority of ordering was over email. The call centre went into meltdown with building firms shouting the odds as they had not received or confirmations and for the first time in its history the call centre had to work Saturday morning calling all the builders that couldn't get through to take their orders the old fashioned way.

No my finest hour.
Sounds bad but I still haven't a clue what you are describing.
Out of office set up on both accounts, which then sent each other a reply each time they received an out of office email, this then made the email server crash and the company went into meltdown. Hth.

Philb1

121 posts

153 months

Saturday 15th October 2016
quotequote all
My wife used to work for RBS. She was field based but had gone into the office in London. She was talking to the IT guys from Accenture who told her a story of an ex employee. He was the lead guy in a scheduled shutdown of some network hub which had been months in planning. It was happening at the weekend and was going to be switched off and on. Supposedly when it came back online the RBS cash machine network was no longer working.

Was apparently quite costly. Not sure how true it was and it was over five years ago but it didn't end well for the guy.

Dogwatch

6,228 posts

222 months

Saturday 15th October 2016
quotequote all
lufbramatt said:
Out of office set up on both accounts, which then sent each other a reply each time they received an out of office email, this then made the email server crash and the company went into meltdown. Hth.
Self denial of service. Wonderful!

ROSSinHD

823 posts

151 months

Saturday 15th October 2016
quotequote all
I lost just shy of £10m by sending it to a number that I transposed. took 30 days to trace in the system and get re applied. Didn't really sleep much that month. Strange as it sounds the client wasn't too fussed as it was pocket change to him.

hora

37,122 posts

211 months

Saturday 15th October 2016
quotequote all
Minor compared to ^

In my second job out of uni I placed an order for two years worth of forward sales on wooden blinds from Indonesia. It arrived and promptly sold out (bit bizarre)!

Cyder

7,052 posts

220 months

Saturday 15th October 2016
quotequote all
Between me and our test department we had a breakdown of communication over a test procedure that ended up costing my employer somewhere in the region of £1.5m to recall thousands of cars to fix them.
Caused a lot of fun and headaches that one.

Jonboy_t

5,038 posts

183 months

Saturday 15th October 2016
quotequote all
One of my own, unfortunately.

I reconcile various balance sheets each month for a very large company (£4b+ a year turn over). I set one up so most calculations were preset and I didn't have to do much to get it to a certain point each time I did it, saved me hours. Anyway, this same account had a normal monthly write of budget for a certain thing that we knew would always show up, about £30k per month - lots of cash but a drop in the ocean in the grand scheme.

Had a systems 'improvement' project that changed practically everything, including which cells the data good into Excel when it is extracted. Thought I'd tested that all the formulae were working properly, which they were.... They were just looking at a different bit of information.

Totally my own fault, but it didn't get picked up for almost a year. Queue me having to sit in front of the FD explaining why I need to write of over £300k, most of which was from the previous financial year and so was not within the budget for this year. Ooops.

TwigtheWonderkid

43,347 posts

150 months

Saturday 15th October 2016
quotequote all
I ordered 1m tax disc sized stickers for an organisation, that were meant to go on the inside of a glass object facing out, so people on the outside could read it. It just didn't occur to me until we came to use them that they had to be sticky on the side with writing on, and not sticky on the back like a conventional sticker.

D'oh.

surveyor

17,817 posts

184 months

Saturday 15th October 2016
quotequote all
Couple spring to mind

First was overheard in a distribution company. Bosses son who was not well liked in charge of fuel.. got a good deal so filled the depot bunkers.

Half the fleet had filled before he discovered it was cheap because it was petrol..

Another

Worked in a county council over ten years ago and the chief executive liked to send a whimsical thought of the day in a Friday to all mailboxes.

One Friday he attaches a 3mb picture. Mail servers took several days to recover....

Future emails were checked by IT first..

Edited by surveyor on Saturday 15th October 13:06