Natwest/RBS glitch

Author
Discussion

hornetrider

Original Poster:

63,161 posts

206 months

Saturday 23rd June 2012
quotequote all
Is there a thread about this already, can't find one.

Seems no-one can access accounts or process payments, and it's been like this for three days! Stories of house sales falling through too, nightmare.

Apparently caused by a software upgrade balls up. Someone's head will be on a spike.

judas

5,996 posts

260 months

Saturday 23rd June 2012
quotequote all
There's thread in the finance forum here.

It's going to be interesting on Monday if it's not fixed - my work banks with Natwest and I'm with RBS so who knows how this will affect my pay going in frown

Heads should certainly roll for this - a cock-up of these monumental proportions shouldn't be possible.

Kudos

2,672 posts

175 months

Saturday 23rd June 2012
quotequote all
I foresee an opening for an IT Director

Mst007

472 posts

223 months

Saturday 23rd June 2012
quotequote all
and perhaps 800 IT jobs coming back from India to the UK?

MarkRSi

5,782 posts

219 months

Saturday 23rd June 2012
quotequote all
Kudos said:
I foresee an opening for an IT Director low paid IT monkey and a massive bonus for an IT Director for fixing the problem
FTFY wink

Ozzie Osmond

21,189 posts

247 months

Saturday 23rd June 2012
quotequote all
Shock headline! (again)

"UK banks are not very good at banking!" smile

turbobloke

104,181 posts

261 months

Saturday 23rd June 2012
quotequote all
Ozzie Osmond said:
Shock headline! (again)

"UK banks are not very good at banking!" smile
Or possibly "UK bank back office IT geeks not very good geeks".

hornetrider

Original Poster:

63,161 posts

206 months

Saturday 23rd June 2012
quotequote all
turbobloke said:
Or possibly "UK bank back office IT geeks not very good geeks".
Yes, this. Probably outsourced, and heads they are a rollin'.

Would love to be in the know as to precisely how this happened, all I've seen is 'failed upgrade' but Software Testing 101 is always ensure there is a blackout/rollback strategy.

Ozzie Osmond

21,189 posts

247 months

Saturday 23rd June 2012
quotequote all
hornetrider said:
all I've seen is 'failed upgrade' but Software Testing 101 is always ensure there is a blackout/rollback strategy.
Precisely. That's why I see it as a fundamental failure of the bank rather then just an excuse to blame the geeks.

I'll bet some twit decided a decent rollback strategy wasn't worth the cost.

fandango_c

1,921 posts

187 months

Saturday 23rd June 2012
quotequote all
Ozzie Osmond said:
hornetrider said:
all I've seen is 'failed upgrade' but Software Testing 101 is always ensure there is a blackout/rollback strategy.
Precisely. That's why I see it as a fundamental failure of the bank rather then just an excuse to blame the geeks.

I'll bet some twit decided a decent rollback strategy wasn't worth the cost.
But that's nothing to do with "banks being no good at banking".

hornetrider

Original Poster:

63,161 posts

206 months

Saturday 23rd June 2012
quotequote all
Ozzie Osmond said:
Precisely. That's why I see it as a fundamental failure of the bank rather then just an excuse to blame the geeks.

I'll bet some twit decided a decent rollback strategy wasn't worth the cost.
Just noticed my iPhone spell correcting backout hehe

I read on the Register that RBS have cut 1000 techy staff in the last year to push through efficiency savings whilst maintaining frontline staff. If you cut back on testing or quality of staff there is always the risk of something like this happening.

This is going to be a massively expensive mistake when people start hitting them with incidental costs they have incurred because of this. Imagine homeowners unable to move and the associated costs!!

miniman

25,095 posts

263 months

Saturday 23rd June 2012
quotequote all
Someone interviewed on R4 yesterday with exactly that problem - all their belongings in a van, their house sold, unable to complete payment for the new one.

Ozzie Osmond

21,189 posts

247 months

Saturday 23rd June 2012
quotequote all
fandango_c said:
But that's nothing to do with "banks being no good at banking".
In much the same way BP's disastrous oil spill in the Gulf of Mexico didn't mean "energy company no good at being an energy company".

fandango_c

1,921 posts

187 months

Saturday 23rd June 2012
quotequote all
Ozzie Osmond said:
In much the same way BP's disastrous oil spill in the Gulf of Mexico didn't mean "energy company no good at being an energy company".
It's an IT problem not specific to the banking industry.

The Deepwater Horizon was caused by an explosion of methane whilst drilling for oil - not something that is a risk for companies which don't drill for oil.

Puggit

48,526 posts

249 months

Saturday 23rd June 2012
quotequote all
As someone who works for a software vendor (selling diasater recovery and infrastructure monitoring software no less) I find it inconceivable that they haven't managed to roll-back the changes,

Software 101 is making sure you have a plan to undo the changes you just made - after you've tested them elsewhere, of course,

groak

3,254 posts

180 months

Saturday 23rd June 2012
quotequote all
Usually Monday's RBS banking appears from about 2am on Saturday. But it seems to be stuck at Friday (yesterday).

Puggit

48,526 posts

249 months

Saturday 23rd June 2012
quotequote all
[quote=Top rated comment on Telegraph]NatWest sacked 1,800 highly-trained and experienced UK IT staff, the last having left a few weeks ago.  NatWest now has virtually no staff who understand how its banking systems work in the UK or employed by it.  All the 'work' and 'testing' is now carried out by 800 'contractors' in India.  The replacements have demonstrated they do not have a clue what they are doing.

Who, out of the senior management which signed this off, who thought 'IT is not at the heart of what we do, let's outsource it', who decided 'these guys in India know what they are doing' is going to take responsibility for this fiasco?

If you're a customer, these are the two questions you need answered next week:

1. Is NatWest/RBS going bring the design and maintenance of its IT infrastructure back to the UK and under its control?

2. Has Susan Allen, Director, Change and Business Services, responsible for 'Transformation Programme across the Retail Division encompassing all channels and functions. Establish Lean capability. Ensure services provided by Business Services (IT, Ops, Property) support delivery of the objectives of the RBS UK Retail, Wealth and Ulster businesses' resigned yet?

If the answer to either question is no, move your accounts to another bank next week.  If you can, if the systems are working again before the next IT disaster.

PS  The problem was caused by a botched 'upgrade' to the CA7 batch scheduling suite and the loss of the schedule.  There is, of course, no RBS employee left in the UK who understands this properly any longer.

PPS Those current NatWest adverts demonstrating the ease with which money can be transferred using a smartphone look pretty ridiculous now, don't they?

PPPS Where is Stephen 'My leadership is worth millions' Hester? He'll appear on TV and radio to defend his salary and bonus but mysteriously fails to say a word when millions of customers can't access their money or obtain a balance...[\quote]

Piersman2

6,606 posts

200 months

Saturday 23rd June 2012
quotequote all
Puggit said:
As someone who works for a software vendor (selling diasater recovery and infrastructure monitoring software no less) I find it inconceivable that they haven't managed to roll-back the changes,

Software 101 is making sure you have a plan to undo the changes you just made - after you've tested them elsewhere, of course,
I've been test managing at large companies for the last 15 or so years.

The larger the company the greater the disconnect between those looking after the functionality of the system and those looking after the infrastructure, especially as the modern push to offshore everything means people are not only remote in location but remote in mentality.

I'm not usually involved in DR testing, I'm more functional, but I'm struggling to remember when any company I've worked for has done a proper DR test. I bet Natwest will be in the next few months, once they've worked out who to scapegoat for this fk up! smile



Rocksteadyeddie

7,971 posts

228 months

Saturday 23rd June 2012
quotequote all
I'm not a big one for conspiracy theories but I reckon they've been hacked. Look at the number of large companies, and governments that have owned up to it recently. "Software glitch" my arse...

Riff Raff

5,145 posts

196 months

Saturday 23rd June 2012
quotequote all
Somebody who is not Puggit said:
If the answer to either question is no, move your accounts to another bank next week.  If you can, if the systems are working again before the next IT disaster.
Easier said than done. I bank with RBS, and am about to be forcibly transferred to Santander. I decided to move my accounts internally within the group to NatWest. Four months later, they still haven't managed to transfer across my regular payments.

Edited by Riff Raff on Saturday 23 June 12:44