Helicopter on A40 for collision within average speed cameras

[Devil2575]

Dave,

This is from your opening post:

Some colleagues have said that, in decades travelling the A40, they have never known the A40 closed due to a collision, others have said they've experienced closure due to a collision once before. No-one has heard of the air ambulance landing on the A40 before.

This potentially major incident has occurred within a month of the new average speed cameras going live on 26th October 2015. Did the speed cameras contribute to this incident? The problem is that we may never know because TfL (Transport for London) did not run their new average speed cameras within scientific trials.

The problem with this is that it starts off with anecdotes, which as you will not is not good evidence.

You could have tried to dig out some actual evidence rather than starting a post based purely on what come colleagues have said.

You then make a link to the accident and the new speed cameras, throwing in the question "did the new cameras contribute to this incident?". This sounds very much like the "Just asking questions" fallacy.

http://rationalwiki.org/wiki/Just_asking_questions

Rather than starting this thread your first question should have been: Are the anecdotes of my colleagues correct and is this accident exceptional or not?
You would also need to understand how the use of the air ambulance could have changed. Is it just that there has not been one available before or has there been a change of the way accidents are prioritised?
You also need to consider whether this accident is just a one off. As I'm sure you're aware single events do not make statistics and there is a high degree of randomness involved in road traffic accidents. Any number of changes could be made to a road to make it statistically far safer but it will never be 100% safe so another accident could occur.
To put it in simple terms think of a game of Russian Roulette. There are two real bulletts and 2 rubber bulletts in 6 chambers, so 2 chambers with nothing in. Over the course of the last game no one got a real bullett but two people were hit by a rubber bullett and sustained a non lethal injury. So in order to make the game safer a rubber bullet and a real bullet are removed. However there is still a chance that the next person to play will be killed, despite the fact that the game is now significantly safer than before.

In essence this thread was started on the basis of loaded speculation, no pun intended.

[WD39]

Dave Finney said:

Thanks TWK but I was thinking of examples where the A40 was completely closed due to a collision within what is now the average speed camera system, all your examples are outside the average speed camera system and none of them near where you live. So have you also never experienced the A40 was completely closed due to a collision from the Polish to the end of the raised section?

Dave, you are chasing your tail, and not getting a bite.

[Pete317]

Devil2575 said:

The problem with this is that it starts off with anecdotes, which as you will not is not good evidence.

Are you implying that you've never started a conversation with words like, "I hear that..."?

But that's the way it seems to go on this forum - someone says something quite innocently, and the resident troop of howler monkeys starts tearing into them mercilessly.

Edited by Pete317 on Wednesday 2nd December 18:20

[Devil2575]

Pete317 said:

Devil2575 said:

The problem with this is that it starts off with anecdotes, which as you will not is not good evidence.

Are you implying that you've never started a conversation with words like, "I hear that..."?

Edited by Pete317 on Wednesday 2nd December 18:20

Not in a situation where I am expecting other people to consider me something of an expert on a subject.

It's the kind of question I'd expect from a layperson, not someone who wants other contributors to take his work on the subject seriously.

[TwigtheWonderkid]

Devil2575 said:

Over the course of the last game no one got a real bullett but two people were hit by a rubber bullett and sustained a non lethal injury.

A rubber bullet fired at point blank range into the temple, Russian Roulette Style, would almost certainly be fatal.
No need to thank me for making such a useful contribution to the debate.

[Devil2575]

Devil2575 said:

When I get a chance I'll read the reports and let you know what I think. I currently don't have time.

Ok, so I've read some of the stuff. There's a lot to get through and I have limited time.

I have a few observations.

I understand the concept of RTM and I understand the example used to demonstrate it's effect. However the usefulness of the Oxbube example is only to demonstrate the concept, because it is by design simply random numbers. Actual road traffic accidents are not simply random. There is a random element to them but aspects will influence the probability of an accident occurring.

You said If the start and end of the SSP for every site within the analysis can be aligned, a graph should show a sudden (vertical) change in the collision rate at the start of the SSP, and another sudden (vertical) change at the end. This should prove (as far as is possible) that this was indeed RTM and not something else.

Why? You are assuming that a camera site is only selected when there is a period of above average accidents. Also even if this is the case there is no guarantee that there would be a vertical change at the start and the end. While it is probable that events regress towards the mean, it is not a certainty at all. Just because I have rolled 3 sixes doesn't mean that I definitely won't roll another 6. With a large number of rolls it is inevitable but with only another few it is entirely possible.

You have set the criteria that you need to positively identify the period of the SSP, which you base on clearly observing a sudden rise and fall. You have then gone on to exclude all those sites where you can not positively do this. The problem with this is that you have actually cherry picked the data which you are going to accept. You have been working under the false impression that the data within the SSP will automatically be above average and be bounded by a sudden rise and fall as well as the notion that the data in both the pre SSP and the ASBiC periods will be representative of the mean. If the pre SSP and ASBiC periods is a large enough dataset then you can work on the assumption that it will be average, but is this case is that genuinely the case? How many years of data did you actually have? I have had a look at some of the links but can't find this data. I may have missed it however, it's just not clear where it is.
It could actually be the case that the sites where you were able to observe this sudden change at either end of a period you have assumed to be the SSP are actually showing a lower than average accident rate over the pre SSP and ASBiC. This means that you are adjusting for the RTM based on an incorrect mean.

Also don't forget that the years you are stating are the SSP due to being above average are actually still part of the average. By removing them from the data you could be reducing the average value. Again, with a big enough data set for pre SSP and ASBiC this will not be the case, or at least it will not be significant. But imagine having data on 20 rolls of a dice then removing a run of 3 sixes from the data. The average number from rolling a dice is 3.5. So assuming over 20 rolls you get the average, then the total will be 70. Remove 18 from this to account for 3 sixes then divide by 17 and you get 3. You are now working under the assumption that 3 is the average from rolling a dice, but it isn't.

I do accept that the best way to determine the effectiveness of speed cameras would be to do a randomised trial, I just don't accept that your method tells us anything due to the fact that you have selelcted the data to which you are going to pay attention based on false assumptions.

[Devil2575]

TwigtheWonderkid said:

Devil2575 said:

Over the course of the last game no one got a real bullett but two people were hit by a rubber bullett and sustained a non lethal injury.

A rubber bullet fired at point blank range into the temple, Russian Roulette Style, would almost certainly be fatal.
No need to thank me for making such a useful contribution to the debate.

Very helpful...

[Pete317]

Devil2575 said:

Not in a situation where I am expecting other people to consider me something of an expert on a subject.

That's not how it started.

And what's wrong with giving a reasonable, sensible and civil reply like, "That's not quite true", instead of going straight for the jugular?

Edited by Pete317 on Friday 4th December 07:49

[Dave Finney]

Devil2575 said:

Devil2575 said:

When I get a chance I'll read the reports and let you know what I think. I currently don't have time.

Ok, so I've read some of the stuff. There's a lot to get through and I have limited time.

I have a few observations.

1) I understand the concept of RTM and I understand the example used to demonstrate it's effect. However the usefulness of the Oxbube example is only to demonstrate the concept, because it is by design simply random numbers. Actual road traffic accidents are not simply random. There is a random element to them but aspects will influence the probability of an accident occurring.

2) You said If the start and end of the SSP for every site within the analysis can be aligned, a graph should show a sudden (vertical) change in the collision rate at the start of the SSP, and another sudden (vertical) change at the end. This should prove (as far as is possible) that this was indeed RTM and not something else.

3) Why? You are assuming that a camera site is only selected when there is a period of above average accidents. Also even if this is the case there is no guarantee that there would be a vertical change at the start and the end. While it is probable that events regress towards the mean, it is not a certainty at all. Just because I have rolled 3 sixes doesn't mean that I definitely won't roll another 6. With a large number of rolls it is inevitable but with only another few it is entirely possible.

4) You have set the criteria that you need to positively identify the period of the SSP, which you base on clearly observing a sudden rise and fall. You have then gone on to exclude all those sites where you can not positively do this. The problem with this is that you have actually cherry picked the data which you are going to accept. You have been working under the false impression that the data within the SSP will automatically be above average and be bounded by a sudden rise and fall as well as the notion that the data in both the pre SSP and the ASBiC periods will be representative of the mean. If the pre SSP and ASBiC periods is a large enough dataset then you can work on the assumption that it will be average, but is this case is that genuinely the case? How many years of data did you actually have? I have had a look at some of the links but can't find this data. I may have missed it however, it's just not clear where it is.
It could actually be the case that the sites where you were able to observe this sudden change at either end of a period you have assumed to be the SSP are actually showing a lower than average accident rate over the pre SSP and ASBiC. This means that you are adjusting for the RTM based on an incorrect mean.

5) Also don't forget that the years you are stating are the SSP due to being above average are actually still part of the average. By removing them from the data you could be reducing the average value. Again, with a big enough data set for pre SSP and ASBiC this will not be the case, or at least it will not be significant. But imagine having data on 20 rolls of a dice then removing a run of 3 sixes from the data. The average number from rolling a dice is 3.5. So assuming over 20 rolls you get the average, then the total will be 70. Remove 18 from this to account for 3 sixes then divide by 17 and you get 3. You are now working under the assumption that 3 is the average from rolling a dice, but it isn't.

6) I do accept that the best way to determine the effectiveness of speed cameras would be to do a randomised trial, I just don't accept that your method tells us anything due to the fact that you have selelcted the data to which you are going to pay attention based on false assumptions.

I am impressed. You have clearly put some thought into this.

I've numbered your paragraphs above:

1) yes, the Oxbube example is only to demonstrate the concept

2) yes.

3a) there isn't any assumption that a camera site is only selected following an above average number of accidents. If the data shows all of the characteristics of the Oxbube example, then we can deduce that the sites within the analysis were "often" chosen following an above average number of accidents. In the case of the mobile speed camera sites, many (12 I think) were chosen following no accidents at all in the SSP. Even including those sites, the data still shows the SSP clearly.

3b) It is true that "Just because I have rolled 3 sixes doesn't mean that I definitely won't roll another 6" but, if you choose 75 dice that have each just rolled 3 sixes, it is extremely unlikely that all of them will roll 3 sixes again at the same time (equally likely they will all roll 3 ones at the same time).

4) If I had done what you say you would be right. "Cherry picking" the data in the way you describe would likely produce unreliable results, but that's not what I did.
Find figure 8.1 here: http://speedcamerareport.co.uk/08_mobile.htm
Figure 8.1 shows the data at ALL qualifying sites, none were excluded (qualifying sites are all mobile speed camera sites that were active in Thames Valley at the start of 2009 and that had been operating for 3 or more years).
The SSP was determined by examining the data for all sites together (not each individually).

5) yes that's true, but what that does is introduce another selection effect. The FTP method removes the effect of site-selection. You're right that the average may be higher than the "mean", that's what the site-selection process can do. The "mean average" can be different to the "mean (normal or expected value of a random variable)". Who was it that decided to use the same words for different things!?!

6) Mathematicians have told me that my FTP method is nothing new, it's been basic mathematics for decades. I get both sides, one side says it's obvious and nothing new, the other says it doesn't work. In the end there are 2 conclusions:

a) the entire KSI reduction at these sites had already occurred a full year before the mobile speed cameras were deployed.
b) whatever opinions people have, all doubts could be settled by running simple scientific trials

[Devil2575]

Have you got a link to the actual data you used? Is it all contained within an individual excel spreadsheet for example?

[Dave Finney]

Devil2575 said:

Have you got a link to the actual data you used? Is it all contained within an individual excel spreadsheet for example?

You're asking all the right questions, you must be an engineer! Before believing a report's conclusions, readers should evaluate the data source and understand the methods used.

Yes, an excel spread sheet (called "mobile cameras TVSRP.xls") containing the data used in the report is in the "Supplementary information" at the end of this page:
http://speedcamerareport.co.uk/08_mobile_refs.htm

The "Supplementary information" also has a sample extract from the full database that was created and supplied by TVSRP (Thames Valley Safer Roads Partnership).

Interestingly, I asked a similar question when the RAC Foundation report that you mentioned was published. The evaluation of RTM in that report starts with data from 6 partnerships in Table 5 (p31/58, labelled p20). I asked which partnerships supplied that data but was told that the data source must remain secret because the partnerships only released the data on the understanding that they would not be identified.