Common before-after accident study on a road site: a low-informative Bayesian method
© European Conference of Transport Research Institutes (ECTRI) 2009
Received: 17 March 2009
Accepted: 8 October 2009
Published: 21 October 2009
This note aims at providing a Bayesian methodological basis for routine before-after accident studies, often applied to a single road site, and in conditions of limited resources in terms of time and expertise.
A low-informative Bayesian method is proposed for before-after accident studies using a comparison site or group of sites. As compared to conventional statistics, the Bayesian approach is less subject to misuse and misinterpretation by practitioners. The low-informative framework seems appropriate in situations of limited expertise. The proposed approach gives the possibility of correcting for regression to the mean. Examples illustrate the application of this method.
Results and conclusions
It is shown that a relatively simple method, based on the Jeffreys’s rule prior considered as a “reasonable standard”, can be implemented without major difficulties. Posterior distributions are proper. The numerical calculation of posterior probabilities can be done without using Monte-Carlo simulations nor specialised software tools.
It is common that road sites are modified in order to achieve improvements from various points of view (traffic conditions, better integration of various uses and users of the road and public space, reduction of noise and air pollution, traffic safety, etc.). A few years after a site has been modified, local engineers generally have to study the effects of this road change, regarding various aspects including road safety. Thus, a retrospective before-after accident study is often needed.
In such routine situations, resources are limited in terms of time and expertise, and the risk of misuse of conventional statistical methods is increased. Even among people who are more experienced in statistics, like researchers, erroneous uses of conventional methods are common: misuse of tests of significance, erroneous understanding of p-values, misinterpretation of confidence intervals (as pointed out by many authors [15, 18, 19, 24, 27, 32]; see also [5, 11, 28]). For example, the p-value is often erroneously regarded as the probability that the null hypothesis is true, and the 95% confidence interval obtained is wrongly assumed to contain the true parameter with a 95% chance. The Bayesian approach to statistics is more in accordance with the expectations and intuitions of non-specialists. In particular, the posterior distribution can be legitimately used to give the probabilities that the parameter of interest is contained in various regions of the parameter space (a 95% credible interval, for example), or exceeds a particular value, given the data observed and prior knowledge. Some authors consider that teaching Bayesian statistics is easier than teaching frequentist statistics [10, 31]. Nevertheless, aids to practitioners are necessary to implement Bayesian methods, since the calculations in these approaches are sometimes complex.
In this paper, we will not deal with studies based on large samples of sites and using multivariate modelling, for which Bayesian approaches were proposed in the recent period [4, 30, 35, 37]. Bayesian methods adapted to meta-analyses or to overviews of several studies (see, for example, ) will not be considered here. We will focus on methods applicable to a single site and transferable to engineers for common practice.
In the case we deal with here (routine evaluation, single site), the methods currently used and recommended are conventional statistical methods (see, for example, ), even though they sometimes make use of empirical Bayes estimates of the expected accident number on the treated site in order to cope with ‘regression to the mean’ bias. The principle of a ‘full’ Bayesian approach was described by Hauer [21, 22] for studying the index of effectiveness θ of a road measure: the prior probability density function of the parameter θ, reflecting the prior knowledge concerning this parameter, is combined with the likelihood function (probability of the data given the parameter) to obtain the posterior probability density function. The posterior probabilities reflect the revised knowledge about the parameter, given previous knowledge and the data analysed. The method proposed by Hauer, however, is an informative (subjective) Bayes method and presupposes expertise or previously formalised knowledge: the prior probabilities are based on the “elicitation of prevailing opinion about the effectiveness of a treatment” (, p. 289), or possibly on the results of previous studies or meta-analyses. Road safety expertise is limited, however, in the routine situations we consider here, since the study is often carried out by a local road engineer, and not by a road safety specialist. Moreover, the site modification is often singular and not generic (it may combine several treatments, for example: redesigning of islands, resurfacing and marking at a junction site). Therefore, it may be difficult to make use of results from previous meta-analyses. A method coping with this problem was described by Al Masaied et al. : prior probabilities were estimated using a part of the accident data, for both the before and the after periods. In the case of a single site, however, this may lead to very small accident numbers for each data subset. Another way is to use the ‘objective’ or ‘low-informative’ Bayesian framework [6, 7, 17, 25, 26] where the prior probabilities are chosen in order to be neutral in some way as regards the possible parameter values, reflecting the lack of previous knowledge. Besides, it can be argued that results based on low-informative approaches are generally easier to communicate to a diverse or uninitiated audience, since, as mentioned by Box and Tiao , they represent “what someone who a priori knew very little about an unknown parameter should believe in light of the data” (p. 22).
In before-after accident studies, it is important to be able to control for regression to the mean bias, which can be done by incorporating some limited information into the prior distribution concerning one component of the vector of parameters (see Section 4). Besides, although such studies are retrospective and not experimental, one should seek to control for the confusing influence of factors other than the road change. To this end, it can be useful to take into consideration a comparison group of similar sites, for example. The method described by Hauer  uses a comparison sample, but the calculations are based on approximations which presuppose that the accident counts in the comparison sample are large. The method proposed by Al-Masaied et al.  is a simple before-after method without comparison sites.
In this methodological note, we describe a low-informative Bayesian method adapted to the current practice of before-after accident studies concerning a single treated site (or a group of sites considered as a whole). A comparison site (or group of sites) is used in order to control for factors other than the road modification. Practical means of calculation, for a commonly available spreadsheet software package, will also be provided on the author’s webpage (http://www.inrets.fr/ur/ma/Brenac.html).
2 Data structure and parameters for the before-after study with comparison sites
Usual form of the basic accident data
Comparison site (or group of sites)
Period I (Before)
Period II (After)
3 The Bayesian framework
This cumulative distribution function makes it possible to calculate credible intervals and the probability that the effect studied is lower or higher than a particular value, given the data and prior probabilities.
4 Low-informative prior distributions
In this paper we assume a lack of previous knowledge or sufficient expertise regarding the parameters. Thus, the prior distributions should be low-informative or neutral as regards these parameters. This choice also tends to “let the data speak for themselves”, giving a higher importance to the likelihood function in the calculation of posterior probabilities. Two situations should be distinguished, however, according to whether regression to the mean bias is likely or not. Regression to the mean (see, for example, ) occurs when the site was chosen for treatment in consideration of a high accident record. In this case, the count x1 gives only biased information on the expected value μ1, and a low-informative prior distribution for μ1 would lead to biased results, overestimating the treatment effect. In this situation, other data or information are needed and should be taken into account in the prior distribution of μ1 (see point 4.2).
4.1 Case where regression to the mean bias is unlikely
Like many non-informative priors, this prior is improper since it does not integrate to a finite value over the parameter space. In Bayesian statistics, however, this is not regarded as a problem, provided that the posterior distribution is proper (i.e., the integral in the denominator of Eq. 3 converges to a finite value).
4.2 Case where regression to the mean bias is likely
In this situation, conventional methods correct for regression to the mean by considering that the site is taken from a population of comparable sites and extracting complementary information from a sample of such sites3. Each of the accident counts x1j at these sites, during period I, is considered as an observation from a Poisson variable with mean μ1j . The μ1j are assumed to be distributed like a Gamma variable with shape parameter α and scale parameter λ (some empirical justifications can be found in the literature [1, 34]). This Poisson-Gamma structure leads to a negative binomial distribution of the counts x1j among this sample of sites. Based on the mean m and variance s2 of this distribution, estimated from the x1j, it is possible to estimate4α and λ: α = m2/(s2–m) and λ = m/(s2−m). Conventional evaluation methods then replace x1, the usual estimate of μ1, by the empirical Bayes estimate μ1* = m2/s2 + x1(s2−m)/s2 = (α+x1)/(1+λ) for the calculation of the odds ratio [16, 23, 36]. This technique has proved to be effective for correcting for regression to the mean bias .
5 Posterior probabilities
5.1 Case where regression to the mean bias is unlikely
The calculation of this integral is generally not possible by analytical means. We describe in the appendix of this paper a way of calculating it numerically.
5.2 Case where regression to the mean bias is likely
5.3 Practical uses of the posterior cumulative distribution function of Θ
From a practical point of view, various useful results can be obtained using the function F Θ (t | x). For example, the lower limit θ LL and upper limit θ UL of a 95% symmetrical credible interval are defined by F Θ (θ LL | x) = 0.025 and F Θ (θ UL | x) = 0.975; the probability, given the data, that θ is contained in this interval is 95%. The median θ med defined by F Θ (θ med | x) = 0.5 gives a point estimate of the odds ratio for which the posterior risks of overestimation and underestimation are equal. The value F Θ (1 | x) represents the posterior probability that θ is lower than 1, i.e. the probability that the treatment is beneficial to safety, given the data and initial assumptions (see Section 2).
6 Particular cases
6.1 Group of comparison sites instead of a single comparison site
In this situation, the group of q comparison sites is considered as a whole, with x3 = Σ x3k and x4 = Σ x4k (where x3k and x4k are the accident counts during periods I and II on each comparison site k, with k = 1 to q). The aggregated counts x3 and x4 are observations from random variables X3 and X4 which are Poisson variables (since they are obtained by summing the independent Poisson variables X3k and X4k) with means μ3 = Σ μ3k and μ4 = Σ μ4k. The calculations described in Sections 3 to 5 are then applied by simply using the aggregated counts x3 and x4 and the aggregated means μ3 and μ4 . The low informative joint prior is given by Eq. 8 or 10. The posterior cumulative distribution function of Θ is then given by Eq. 13 or 14 (with x3 = Σ x3k and x4 = Σ x4k ).
6.2 Multiple treated sites
The general case of several treated sites, considered independently, with possibly different odds ratios θ i due to heterogeneity in the treatment effect is beyond the purpose of this paper and will be the subject of further publications. Nevertheless, in the simpler situation where a group of treated sites is considered as a whole (with a focus on the overall effect of treatment), the methods described above can be easily adapted.
Let us consider n treated sites with accident counts x1i and x2i (i = 1 to n) during periods I and II, with corresponding means μ1i and μ2i, and q comparison sites with accidents counts x3k and x4k (k = 1 to q) during periods I and II, with corresponding means μ3k and μ4k.
When regression to the mean bias is unlikely, and if we consider the treated sites as a whole (and the comparison sites as a whole), the calculations and results described in Sections 3 to 5 can be applied by simply using the aggregated counts x1 = Σ x1i, x2 = Σ x2i, x3 = Σ x3k, x4 = Σ x4k and the corresponding aggregated means, with the prior given in Eq. 8. In this case, the parameter θ represents the overall effect of the programme of treatment. The posterior probabilities are given by Eq. 13.
7 Examples of application
7.1 Example 1: Safety effect of redesigning an urban road section
We describe here the case of an urban section of road where the infrastructure was largely modified in order to enhance the quality of local urban life. Raised median islands, small roundabouts, speed humps and raised tables were implemented in 2000 on this section of a main urban road in a town of 40,000 inhabitants (length of the treated section: 700 m). All the unmodified sections of the main roads in this town were taken as a comparison group of sites. The comparability between the treated site and the comparison group of sites was verified by comparing the yearly injury accident counts for the 1989–1999 period. The ‘before’ period is the five-year period from 1995 to 1999. The ‘after’ period is the five-year period from 2001 to 2005. The presence of regression to the mean bias was considered to be unlikely for the following reasons: this project was not decided for safety reasons, and the proportion of accidents during the 1995–1999 period relative to 1989–1999 was not higher on the treated site as compared to all the unmodified sections of main roads in this town. For the ‘before’ period, 16 injury accidents occurred on the treated site and 61 injury accidents occurred on the comparison group of sites. For the ‘after’ period, 3 injury accidents occurred on the treated site, and 46 injury accidents occurred on the comparison group of sites.
95% symmetrical credible interval:
0.062 to 0.815
Posterior probability that θ < 1:
θ ML * = 0.249
Woolf 95% confidence interval:
0.068 to 0.904
In this example, a practitioner would probably conclude in favour of a positive effect on safety, from both these Bayesian and non-Bayesian results.
7.2 Example 2: Safety effect of a rural crossroads modification
This example deals with a priority intersection on a main rural two-lane road. This crossroads was modified in 1986 (installation of median raised islands, marking) for safety reasons. Therefore, regression to the mean is likely to occur. At this junction, 14 injury accidents occurred during the three-year period before the treatment. During the three-year period following the treatment, 4 injury accidents occurred.
This evolution was compared to the evolution observed at a set of 11 similar intersections on main rural two-lane roads in the same region, used as a comparison group of sites. At these sites, considered as a whole, 33 injury accidents occurred during the before period and 22 injury accidents occurred during the after period.
95% symmetrical credible interval:
0.117 to 1.389
Posterior probability that θ < 1:
95% symmetrical credible interval:
0.151 to 1.789
Posterior probability that θ < 1:
These results show that, in this case, the safety effect is in reality smaller than indicated by the biased results obtained with the low-informative prior given by Eq. 8. The median of the posterior distribution (0.566) can be used as a point estimate of the odds ratio (where the posterior probabilities of overestimation and underestimation are equal). This value corresponds to an accident reduction of approximately 43%. The 95% credible interval, however, is large and the beneficial effect of the treatment remains uncertain.
Using the same data, a more conventional approach would lead, for example, to the maximum likelihood estimate θ ML * = 0.429 (without controlling for regression to the mean), or to a corrected estimate of 0.515 based on the empirical Bayes estimate of μ1 [16, 36].
7.3 Example 3: Safety effect of resurfacing on main roads
95% symmetrical credible interval:
0.794 to 1.537
Posterior probability that θ < 1:
θ ML * = 1.105
Woolf 95% confidence interval:
0.794 to 1.537
This proximity is not surprising: posterior credible intervals based on the Jeffreys’s rule prior are frequently close to frequentist confidence intervals in large sample conditions [17, 40] although they do not have the same meaning.
Based on these results, the posterior median estimate of θ would suggest a slight detrimental effect on safety (increase of accidentality of approximately 11%), but no certain conclusion can be drawn since the 95% credible interval is large. Based on the posterior probability that θ < 1 (approximately 28%), however, one could say that the probability that the treatment increases the accidentality, given the data and assumptions, is 72%. No equivalent result from a conventional statistical analysis could lead to this kind of interpretation, except if one wrongly interprets a p-value as a posterior probability. A possible increase of accidentality could be explained by the fact that resurfacing tends to increase the average speeds, at least when the road is dry, as shown by Leden et al. .
8 Discussion and conclusion
In this note, we described a low-informative Bayesian method for before-after accident studies, using a comparison site or group of sites, and giving the possibility of correcting for regression to the mean bias. The aim was to provide a methodological basis for routine evaluation studies, often applied to a single treated site, and in conditions of limited resources in terms of time and expertise. As compared to conventional statistics, the Bayesian approach is less subject to misuse and misinterpretation by practitioners with limited statistical experience. The low-informative or objective Bayesian methods seem appropriate in routine evaluation studies, where expertise or previous knowledge are often limited or hard to formalise. As shown in Sections 2 to 6, a relatively simple method, based on the Jeffreys’s rule prior considered as a “reasonable standard”, can be implemented without major difficulties. Posterior distributions are proper. The numerical calculation of posterior probabilities can be done without using Monte-Carlo methods nor specialised software tools. The examples given in Section 7 show that the results can be analysed in a direct way, without the high risk of misinterpretation involved in the analysis of frequentist results.
Further developments, however, are still needed. Although this method seems to be transferable to engineers for common practice, further work is necessary in order to provide a simple, didactic description of the Bayesian line of reasoning, with minimal use of mathematical formalisms, appropriate for communicating this approach to practitioners. Concerning the practical means for calculating the posterior probabilities, the spreadsheet mentioned in the appendix (for a common spreadsheet software package) will be made available on our website.
The proposed method has limitations, of course. Retrospective before-after studies are not randomised experiments and the validity of their results is based on the assumption that the treated and comparison sites are similar. Before-after studies based on multivariate generalised linear models make it possible to better control for the influence of differences between treated and comparison sites. But such methodologies would generally involve a thorough data collection and analysis on a large sample of sites, which seems hard to implement by practitioners in the routine situations we considered in this paper. The comparability of treated and comparison sites, however, can be checked by examining their accident history, when accident data are available for a long period before the treatment (see ). A Bayesian approach to this subject could be studied. Besides, other developments could contribute to extending the field of application of the proposed method: in this paper, we only dealt with the case of a single treated site (or a group of sites treated as a whole, with a focus on the overall effect of the programme of treatment), with a comparison site or group of sites. The case of multiple treated sites considered independently and with possibly different odds ratios remains to be dealt with. However, this would involve an increased complexity and more difficulties for practitioners.
We hope this methodological note will contribute to an increased use of the Bayesian approach, which is more in accordance with the expectations and intuitions of non-statisticians, in the current practice of before-after accident studies.
This notation means: probability of the data x = (x1, x2, x3, x4) given the parameter values θ, η , μ1, μ3.
This rule can be justified from several points of view, in particular: invariance by re-parameterisation, uniformity, in the sense of equiprobability of regions of same size in the parameter space with a Riemannian metric, and minimisation of information (the Jeffreys's rule prior can be considered as a special case of the Bernardo-Berger prior). For developments of these arguments, see for example Ghosh et al.  and Kass and Wassermann .
Alternatively, when a general accident model is available (see, for example, the models published by the Transport Research Laboratory in the United Kingdom), it can be applied for obtaining complementary information, instead of using a sample of the population of comparable sites [23, 38].
Instead, if an accident model is available, it can give the mean m and variance s2 of the accident counts on a virtual population of sites with the same characteristics as the site of interest [23,38]. The parameters α and λ are then also obtained by calculating α = m2/(s2−m) and λ = m/(s2−m).
In the expression of K, the term in θ is proportional to a three-parameter Beta-prime distribution, which makes it possible to integrate with respect to θ over [0,+∞). The integration with respect to η is then possible, over [0,+∞).
A correct interpretation of a classical (non-Bayesian) 95% confidence interval is: if we could indefinitely repeat the same “experiment” with the same parameter value, 95% of the confidence intervals thus obtained would contain this value.
The author would like to thank Sylvie Després (Université Paris-Nord) and three anonymous referees for their helpful comments.
- Abbess C, Jarrett D, Wright CC (1981) Accidents at blackspots: estimating the effect of remedial treatment, with special reference to the ‘regression-to-mean’ effect. Traffic Eng Control 22:535–542Google Scholar
- Agresti A, Hitchcock DB (2005) Bayesian inference for categorical data analysis. Stat Methods Appl 14:297–330MATHMathSciNetView ArticleGoogle Scholar
- Al-Masaied HR, Sinha KC, Kuczek T (1993) Evaluation of safety impact of highway projects. Transp Res Rec 1401:9–16Google Scholar
- Aul N, Davis G (2006) Use of propensity score matching method and hybrid Bayesian method to estimate crash modification factors of signal installation. Transp Res Rec 1950:17–23View ArticleGoogle Scholar
- Belia S, Fidler F, Williams J, Cumming G (2005) Researchers misunderstand confidence intervals and standard errors. Psychol Methods 10:389–396View ArticleGoogle Scholar
- Berger J (1985) Statistical decision theory and Bayesian analysis. Springer, New YorkMATHView ArticleGoogle Scholar
- Berger J (2006) The case for objective Bayesian analysis. Bayesian Anal 1:385–402MathSciNetView ArticleGoogle Scholar
- Berger JO, Bernardo JM (1992) On the development of the reference prior method. In: Bernardo JM, Berger JO, Dawid AP, Smith AFM (eds) Bayesian Statistics 4: Proceedings of the Fourth Valencia International Meeting. Clarendon Press, Oxford, pp 35–60Google Scholar
- Bernardo JM (1979) Reference posterior distributions for Bayesian inference. J R Stat Soc Series B Stat Methodol 41:113–147MATHMathSciNetGoogle Scholar
- Berry DA (1995) Decision analysis and Bayesian methods in clinical trials. Cancer Treat Res 75:125–154View ArticleGoogle Scholar
- Berry DA (1997) Teaching elementary Bayesian statistics with real applications in science. Am Stat 51:241–246Google Scholar
- Bin Ibrahim K, Metcalfe AV (1993) Bayesian overview for evaluation of mini-roundabouts as a road safety measure. Statistician 42:525–540View ArticleGoogle Scholar
- Box GEP, Tiao GC (1973) Bayesian inference in statistical analysis. Addison-Wesley, ReadingMATHGoogle Scholar
- Brenac T (1994) Accidents en carrefour sur routes nationales, modélisation du nombre d’accidents prédictible sur un carrefour et exemples d’applications. INRETS report 185. INRETS, Arcueil (France)Google Scholar
- Cohen J (1994) The earth is round (p < 0.05). Am Psychol 49:997–1003View ArticleGoogle Scholar
- De Brabander B, Nuyts E, Vereeck L (2005) Road safety effects of roundabouts in Flanders. J Saf Res 36:289–296View ArticleGoogle Scholar
- Ghosh JK, Delampady M, Samanta T (2006) An introduction to Bayesian analysis, theory and methods. Springer, New YorkMATHGoogle Scholar
- Goodman SN (2005) Introduction to Bayesian methods, I: measuring the strength of evidence. Clin Trials 2:282–290View ArticleGoogle Scholar
- Haller H, Krauss S (2002) Misinterpretations of significance: A problem students share with their teachers? Methods Psychol Res 7:1–20Google Scholar
- Hasofer AM (1970) On the representation of ignorance in Poisson processes. J R Stat Soc Series B Stat Methodol 32:268–271MATHGoogle Scholar
- Hauer E (1983) Reflections on methods of statistical inference in research on the effect of safety countermeasures. Accid Anal Prev 15:275–285View ArticleGoogle Scholar
- Hauer E (1983) An application of the likelihood/Bayes approach to the estimation of safety countermeasure effectiveness. Accid Anal Prev 15:287–298View ArticleGoogle Scholar
- Hauer E (1997) Observational before-after studies in road safety. Pergamon, OxfordGoogle Scholar
- Hauer E (2004) The harm done by tests of significance. Accid Anal Prev 36:495–500View ArticleGoogle Scholar
- Jaynes ET (1968) Prior probabilities. IEEE Trans Syst Sci Cybern 4:227–241MATHView ArticleGoogle Scholar
- Jeffreys H (1961) Theory of probability, 3rd edn. Oxford University Press, LondonMATHGoogle Scholar
- Johnson DH (1999) The insignificance of statistical significance testing. J Wildl Manag 63:763–772View ArticleGoogle Scholar
- Kadane JB (1995) Prime time for Bayes. Control Clin Trials 16:313–318View ArticleGoogle Scholar
- Kass RE, Wasserman L (1996) The selection of prior distribution by formal rules. J Am Stat Assoc 91:1343–1370MATHView ArticleGoogle Scholar
- Lan B, Persaud B, Lyon C, Bhim R (2008) Validation of a full Bayes methodology for observational before-after road safety studies and application to evaluation of rural signal conversions. Transportation Research Board annual meeting, WashingtonGoogle Scholar
- Lecoutre B (1999) Beyond the significance test controversy: Prime time for Bayes? Bull Int Stat Inst LVIII(2):205–208Google Scholar
- Lecoutre B, Lecoutre MP, Poitevineau J (2001) Uses, abuses and misuses of significance tests in the scientific community: Won’t the Bayesian choice be unavoidable? Int Stat Rev 69:399–418MATHView ArticleGoogle Scholar
- Leden L, Hämäläinen O, Manninen E (1998) The effect of resurfacing on friction, speeds and safety on main roads in Finland. Accid Anal Prev 30:75–85View ArticleGoogle Scholar
- Maher MJ (1987) Fitting probability distributions to accident frequency data. Traffic Eng Control 28:356–357Google Scholar
- Miranda-Moreno LF (2007) Fu L (2007) Traffic safety study: empirical Bayes or full Bayes? Transportation Research Board annual meeting, WashingtonGoogle Scholar
- Mountain L, Fawaz B, Sineng L (1992) The assessment of changes in accident frequencies on link segments: a comparison of four methods. Traffic Eng Control 33:429–431Google Scholar
- Pawlovich MD, Li W, Carriquiry A, Welch T (2006) Iowa’s experience with road diet measures, use of Bayesian approach to assess impacts on crash frequencies and crash rates. Transp Res Rec 1953:163–171View ArticleGoogle Scholar
- Persaud B, Lyon C (2007) Empirical-Bayes before-after safety studies: Lessons learned from two decades of experience and future directions. Accid Anal Prev 39:546–555View ArticleGoogle Scholar
- Robert CP (2007) The Bayesian choice, from decision-theoretic foundations to computational implementation, 2nd edn. Springer, New YorkMATHGoogle Scholar
- Welch BL, Peers HW (1963) On formulae for confidence points based on intervals of weighted likelihoods. J R Stat Soc Series B Stat Methodol 25:318–329MATHMathSciNetGoogle Scholar