Estimated crash avoidance with the hypothetical introduction of automated vehicles: a simulation based on experts’ assessment from French in-depth data

Abstract

Objective

We aimed to quantify, through simulations using real crash data, the number of potentially avoided crashes following different replacement levels of light vehicles by level-5 automated light vehicles (AVs).

Methods

Since level-5 AVs are not on the road yet, or are too rare, we simulated their introduction into traffic using a national database of all fatal crashes and 5% of injury crashes observed in France in 2011. We fictitiously replaced a certain proportion of light vehicles (LVs) involved in crashes by level-5 AVs, and applied crash avoidance probabilities estimated by a number of experts regarding the capabilities of AVs depending on specific configurations. Estimates of the percentage of avoided crashes per user configuration and according to three selected (10%, 50%, 100%) replacement levels were made, as well as estimates taking into account the relative weight of these crash configurations, and considering fatal and injury crashes separately.

Results

Our simulation suggests that a reduction of almost half of fatal crashes (56%) and injury crashes (46%) could be expected by replacing all LVs on the road with level-5 AVs. The introduction of AVs would be the least effective for crashes involving a vulnerable road user, especially motorcyclists.

Conclusion

This result represents encouraging prospects for the introduction of automated vehicles into traffic, while making it clear that, even with all light vehicles replaced with level 5-AVs, all issues would not be solved, especially for crashes involving motorcyclists, cyclists and pedestrians.

1 Introduction

Worldwide, automated vehicles (AVs) bring high expectations in terms of road safety. The European Commission considered the development of AVs as one of its priorities for road safety in the Malta Declaration (Ministerial Conference on Road Safety in 2017). In France, the New Industrial plan (NFI plan), which states French industrial priorities, notably concerns AVs, shows growing national interest. This plan foresees the circulation of AVs in the context of regular journeys and automatic valet parking by 2030. After 2030, fully-automated vehicles would be allowed to circulate on public grounds within the framework of European regulations. Although AVs spark growing interest, the angles of approach, contexts, technologies and methodologies are challenging, and the overall road safety potential impact of their introduction has not yet been studied much.

Due to the absence of fully-automated circulation, most researchers, for reasons of data availability, conducted their studies on low levels of automation already available (e.g., driving assistance and connected-vehicle technology), while only a few focused on high automation levels (levels of 4 and 5, with the level of automation ranging from 0, i.e., no automation to 5 or full automation, according to the Society of Automotive Engineers, SAE). Yue et al. [17] reviewed the literature on vehicles with low levels of automation (levels 2 to 3). They showed that studies on road safety impact did not always include all technologies. The results of the reviewed studies were also highly heterogeneous, with a proportion of avoided crashes ranging from 2 to 64%, according to the number of technologies or crash scenarios included. A more recent study evaluated the potential effect of the development of driver support systems (low automation systems) on the variation in the number of crashes in Poland by 2030 [11]. Based on crashes that occurred in 2018, the authors expect a decrease of 15.8% according to the current knowledge of driver support systems and the rate of their diffusion.

Regarding high levels of automation, data do not yet exist, and studies can only try to predict the introduction of AVs in real traffic and their potential effect on safety.

Many methodologies have been used to evaluate the impact of AV introduction. Some studies used observational data of AVs running in licenced areas (e.g., Texas, California) [5] or in isolated experimental environments, which are truly limited and do not represent real situations. Others used traffic simulations, for example, through software to run fictitious level-5 automated vehicles [16], driving simulators [13], or by integrating assumptions in statistical models to simulate AV introduction into actual standard traffic data and estimate the impact on the number of crashes [1]. Fahrenkrog et al. [3] used a combination of these methods. They provided results for each studied driving scenario according to three categories: crashes for which highly-automated vehicles will have a positive impact, i.e., no difficulty,crashes for which highly-automated vehicles could have either positive or negative impacts, those cases requiring in-depth data; and new crash scenarios that could occur, or when highly-automated vehicles could have a negative impact [3]. Unfortunately, they did not summarize the different scenarios and categories to provide an estimate of the impact on the crash rate at a national level.

In the end, few studies attempted an overall assessment of the impact of AVs on the occurrence of crashes on an entire national road network, and none considered both injuries and fatalities stratified by the type of road users involved, including vulnerable road users. Additionally, there is no study evaluating the impact of AV introduction on the crash rate in France.

Our objective was to quantify the overall impact of the replacement of conventional light vehicles (LVs) by vehicles with full automation, i.e., level-5 AVs [12] on the number of fatal crashes and injury crashes. In the absence of real traffic data, we used experts’ evaluations of AV behaviours. Then, based on a French national database including all deadly crashes and 5% of the injury crashes that occurred on all types of road infrastructures over 1 year, we fictitiously substituted different proportions of LVs involved in crashes with level-5 AVs. This is the first study relying on an entire road network at a national level.

2 Methods

2.1 Crash data

We extracted crash data from the VOIESUR project (Véhicule Occupant Infrastructure Études de la Sécurité des Usagers de la Route, Vehicle Occupant Infrastructure Road User Safety Study). Police reports of all fatal road crashes (n = 3702) and of a 1/20 random sample of nonfatal injury crashes (n = 4839) reported in France in 2011 were collected. Every crash and the role played by each road user involved are described precisely elsewhere [15]. In particular, we used 168 predefined pictograms, adapted from pictograms developed for road crash research by the German Insurance Association (GDV) and by the Initiative for the Global harmonization of Accident Data [7, 8], and developed to describe crash circumstances (examples provided in the Additional file 1: Appendix, Figure A1). Each crash was allocated one pictogram. First, a group of pictograms describing a category of accidents (e.g., crashes involving a pedestrian, crashes at a road intersection where the intersection is of importance, or crashes involving a vehicle leaving its parking space) was selected based on the circumstances of the crash. Second, the pictogram, which schematically summarizes the best circumstances and sequence of events for the crash, was chosen to represent the crash. The same pictogram could be used for different types of road users. In the SURCA project (Sécurité des Usagers de la Route et Conduite Automatisée, Road User Safety and Automated Driving), a project dedicated to the introduction of AVs and their possible effect on road safety, a consortium of road crash and AV experts specifically selected the most relevant pictograms for AV-related analyses and the most representative of all the crashes (86/168 pictograms, representing 89% of all VOIESUR crashes, single-user vehicle accidents not included). Based on those pictograms, specific road crash and AV experts (their qualification can be found in the Additional file 1: Appendix, Table A1) evaluated the effect of AV behaviour, i.e., the possibility of avoiding the crash or not,

2.2 Simulation assumptions

To evaluate how level-5 AV introduction could have modified the outcomes of previous VOIESUR-observed situations, we made several hypotheses. First, we hypothesized a level-5 AV as an “ideal” vehicle that would have no failure and would not cause any crash by itself [3] but that might not be able to handle a situation perfectly if the problem arises because of another user. Additionally, we chose to assume level-5 AVs to be fully independent (no communication with another AV and similar behaviour in front of another AV or a conventional vehicle). All level-5 AVs were considered equivalent in terms of technology, and an LV could only be either a level-5 AV or a conventional LV. Second, because we relied on real crash data, we assumed comparability before and after the introduction of an AV, which means that a given situation, if similar to one of those in the dataset, would always lead to a crash without the use of an AV. It also implies that road infrastructure and traffic-related behaviours would stay the same even after introduction of an AV. Third, we assumed that all the road crashes under study were entirely independent, i.e., we did not consider possible updates to AV systems to change the behaviour of a vehicle over the course of the study. Fourth, to be able to use crash pictograms for specific configurations, we considered that those pictograms fully described crash circumstances, i.e., that all elements of context not described in the pictograms, such as meteorological and lighting conditions, geographic area, and the age of the driver, remained constant.

2.3 Crash configurations included in the simulation

In the following, we use the term “active road user” (ARU) for drivers of LVs, cyclists, motorized two-wheeler users (M2W) or pedestrians actively involved in the crash and exclude vehicle passengers. We then categorize VOIESUR crashes into three types of crash configurations depending on the involvement and number of ARUs:

• 1. Single LV crashes (e.g., single vehicle involved with left lane departure, single vehicle involved with lane departure to the right after an encroachment on the left shoulder, n = 1600);

• 2. Two active road users crashes, involving at least one LV (LV versus pedestrian/cyclist/M2W/another LV/truck, n = 4953);

• 3. All other cases: for example, single crashes without an LV, crashes involving two ARUs but no LV drivers, crashes involving more than two ARUs (n = 1988).

In the present study, we focused on crash types 1 and 2 only, i.e., configurations involving one or two ARUs including at least one LV. Crash configuration 3 was not evaluated because of (1) the complexity of assessing the role of each ARU in these crashes; (2) the representativeness of only one pictogram to describe such complicated situations; and (3) the difficulty for experts to evaluate the AV behaviour and provide a probability of a crash for those cases. Crashes included in our study corresponded to more than 75% of crashes in the VOIESUR study in total (74.5% of fatal crashes and 77.1% of injury crashes).

2.4 Simulation: crash avoidance estimation

We used a two-step method to estimate the number of avoided crashes. First, we simulated a replacement level of LVs by AVs by drawing a binomial distribution of a random variable 100 times, whose possible values were {AV, NAV}, with AV being a level-5 AV replacing a conventional vehicle and NAV (nonautomated vehicle) otherwise, and with parameters of p, the replacement level, and n, the number of crashes. We considered replacement levels of 10, 50 and 100% to evaluate the effects of a vehicle fleet with only a small fraction of level-5 AVs; with equal proportions of LVs and level-5 AVs; and with level-5 AVs only. Second, we addressed the two types of configurations using two distinct decision trees (Additional file 1: Figures A2 and A3):

For type 1 configurations ("single-LV" crashes), because we assumed perfect level-5 AVs, we considered that the crash would not have occurred if the vehicle was an AV. The crash would still occur otherwise (Additional file 1: Figure A2).

For type 2 configurations (two ARU crashes, with at least one LV), the probability of a crash relied on the pictograms describing the crash circumstances. Eight experts were asked to give the likelihood (from 0 to 1) of an AV managing the described situation and avoiding the crash. For each crash situation described by a pictogram and each vehicle in the pictogram, the experts assigned a value of 0 when they thought an AV could do nothing more than a conventional vehicle (i.e., it could not prevent the crash) and a value of 1 if an AV would always prevent the crash. In all other situations, the experts assigned a value of n/10 (with 0 < n < 10) to express the probability of an AV managing the situation (see an example in Additional file 1: Figure A4). The eighty-six pictograms covered most of the included cases (91%). Regarding the LV versus LV crashes (LV/LV configuration), we ran 100 random draws for each pictogram and each LV identified in the pictogram to determine the outcome of the scenario described by the pictograms, according to the probabilities given by the experts. Then, we used a “crash situation” variable, which describes the situation of the vehicle and its driver before the collision (e.g., “driver believed to be responsible for a loss of control”), to determine the LV localization in the pictogram and then to determine the corresponding probability of avoiding a crash. In some cases, LV localization in the pictogram was not possible based on the data only. Those cases were considered uncovered. When two AVs were involved, because we assumed no communication between vehicles, we calculated the probability of avoiding a crash as the probability of a crash for the first vehicle times that for the second vehicle (see Additional file 1: Figure A3). Finally, we used the relative weight of each configuration in the database (see Additional file 1: Table A3) to determine a national estimate based on all VOIESUR crashes.

Experts’ opinions could differ for the same pictogram. We used the average of probabilities provided by the experts as follows: When experts’ probabilities were in the same range ([0–0.5] or [0.5–1]), we averaged the different values to compute the overall probability of avoiding a crash. When experts disagreed, i.e., probabilities were either in [0, 0.5], or [0.5, 1], we separately averaged “low” and “high” probabilities, providing lower (unfavourable experts’ response, UF) and upper (favourable experts’ response, F) boundaries to create an interval of possible avoided crash percentages. We then estimated the mean UF and F percentages of avoiding a crash by averaging the 100 draws. We used the weighted Fleiss’ kappa to present the agreement between experts, with a value close to zero indicating poor agreement and a value close to one indicating high agreement [6].

Finally, we tested the sensitivity of our results as applied to AV positioning and situation management by calculating the standard deviations (SD) of the 200 draws (100 for the replacement level and 100 for the experts’ probabilities).

Analyses were carried out with the SAS.9.4 software.

3 Results

3.1 Percentage of avoided crashes by configuration

The percentages of avoided crashes are presented by configuration for injury and fatality crashes in Tables 1 and 2, respectively.

Table 1 Average percentage* of avoided injury crashes by configuration

Table 2 Average percentage* of avoided fatal crashes by configuration

3.1.1 Crashes involving vulnerable road users

For the LV/pedestrian configuration, the number of injury crashes decreases by 6.6–6.8% (interval given by the UF mean and F mean) for a 10% AV replacement level and by 66.2–68.5% for a fully-autonomous fleet. For fatal crashes, the decrease ranges from 6.4 to 7.0% for a 10% AV replacement level and by 63.3–63.9% for a 100% AV replacement level.

For the LV/cyclist configuration, AV replacement reduces the number of injury crashes by 3.2–4.8% and 31.2–47.7% and the number of fatal crashes by 3.4–4.2% and 30.8–39.6% for 10% and 100% replacement levels, respectively.

Regarding the LV/M2W configuration, injury crashes were reduced by 2.7–7.3% and 27.5–75.0%, and fatal crashes were reduced by 2.8–7.4% and 27.8–74% for 10% and 100% replacement levels, respectively. This configuration is particularly sensitive to experts' opinions (kappa coefficient of 0.36), corresponding to a favourable/unfavourable ratio > 2, regardless of the type of crash (injury or fatal) and the replacement level considered. Compared to all other configurations, AV introduction has the lowest impact on LV/M2W fatal crashes, regardless of the replacement level.

3.1.2 Crashes involving only cars or trucks

For the LV/LV configuration, injury crashes were reduced by 8.5–10.9% and 72.6–82.6%, with an AV introduction of 10% and 100%, respectively, and fatal crashes were reduced by 10.4–11.7% and 85.2–92.1%. Experts were in agreement for this configuration, with the lowest kappa coefficient (k = 0.73), corresponding to a maximum favourable/unfavourable difference of approximately 10%. AV introduction has the highest impact on injury crashes for this configuration compared to others, with the largest percent of avoided crashes.

For the LV/Truck configuration, the impact of AV replacement varies from 4.3% to 43.7% for injury crashes and from 6.4% to 63.8% for fatal crashes at 10% and 100% replacement levels, respectively.

Based on our methodology, 10%, 50% and 100% of single LV crashes were avoided with replacement levels of 10%, 50%, and 100%, respectively.

The results were very stable across draws (SD range: 0.057–2.062, Additional file 1: Table A4).

3.2 Percentage of avoided crashes by severity

The impact of the gradual replacement of LVs by AVs in traffic is described separately in Table 3 (injury crashes) and Table 4 (fatal crashes). The weights of these configurations in the VOIESUR dataset are presented in the Additional file 1: Appendix (Table A3).

Table 3 Average percentage* of avoided crashes for injury crashes

Table 4 Average percentage* of avoided crashes for fatal crashes

3.2.1 Injury crashes

All single LV crashes disappear based on our hypothesis of perfect level-5 AVs. For crashes involving two ARUs including one LV, the introduction of AVs only results in a reduction of 3.6–5.2% of injury crashes when the replacement level is 10%; of 17.5–24.8% when the replacement level is 50%; and of 33.7–47.1% with a 100% replacement level. For this configuration, uncovered cases represent 9.6% to 10.4% (UF, F cases, respectively) of the crashes.

Overall, the replacement of LVs with AVs would decrease the French injury crashes by a minimum of 4.8% (most unfavourable case and 10% replacement level) to up to 58.9% (most favourable case and full replacement of LVs by AVs). Uncovered crashes (due to the non-studied configurations or failure of localization of the vehicle) represent 22.9% of the injury crashes.

3.2.2 Fatal crashes

For fatal crashes, single LV crashes are fully-avoided by assuming a perfect AV. For crashes involving two ARUs, the impact of introducing an AV is lower than for injury crashes. An AV replacement level of 10% would avoid 3.2% of fatal crashes in the most unfavourable case. This proportion would increase to 35.6% in the most favourable case for a fully-autonomous fleet. A few of these configurations remain uncovered (4.6–5.2%).

Regarding all VOIESUR fatal crashes, single LV crashes accounted for a large proportion (27.1%). Considering that they would no longer occur, our hypothesis of perfect AVs leads to an overall AV performance almost identical to that of the injury crashes. The percentage of avoided crashes varies from 5.9% for the worst case and a replacement level of 10% to 62.8% with the most favourable experts’ opinions and a fully-autonomous fleet. Uncovered crashes represent 25.5% of all the fatal crashes.

4 Discussion

This original simulation study provides the first estimate of the impact of the hypothetical level-5 AV introduction on French road crashes over 1 year. We estimate that highly-automated vehicles could almost halve the number of crashes (up to 46% of injury crashes and 56% of fatal crashes). The lowest impact of AV introduction on crash reduction is for LV/M2W crashes and the highest is for LV/LV crashes, excluding the single LV user configuration.

In this simulation work, even after replacing all LVs with AVs, we do not obtain the upper bound of the 93–94% crash avoidance-rate announced by Fagnant and Kockelman [2] or Mueller et al. [10]. At best, we found that the number of injury crashes and fatal crashes was barely halved, which corresponds better to the minimum of 34% to 40% announced by the same authors [2, 10]. Contrary to these studies, we did not consider AV replacement for types of vehicles other than LVs, which may explain in part the difference. However, our scenario is likely more realistic for the future because the main AV technologies are currently mostly developed for LVs [4]. Moreover, due to the difficulty in assessing the contribution of an LV in crash configurations involving more than two ARUs, we were not able to include them in our study, which artificially decreases the potential positive impact of AV introduction. However, the contribution of those configurations to the total number of crashes was smaller than those included in our study. Therefore, although all crashes were not considered in our study, we included the majority of crash configurations, which occurred over a whole year at a national level (77.1% for the injury crashes and 74.5% for the fatal ones). Additionally, even though one could assume that all uncovered cases (uncovered pictograms, failure of localization and more than 2 ARUs) could be avoided by AVs, at least one quarter of crashes would still occur. We hypothesized the independence of AVs and crashes, developed necessary hypotheses to reduce the complexity of the simulation and the experts’ estimations while retaining some rigidity, and assumed that communication between AVs and system updates would probably decrease the risk of a crash. Experts consider that, in some situations, current AV technology is not good enough to correctly handle vehicle behaviour and avoid a crash. This is especially the case for crashes involving an M2W or a cyclist, for which the number of avoided crashes is lower. This is in line with a recent work that stated that AVs could avoid human errors of sensing/perceiving and incapacitation, representing 34% of crashes in the United States, but could also reproduce human errors [10]. Other studies also found that the type of automation technology used could result in important variations in the potential impact of AVs on safety. For instance, for a crash configuration of a pedestrian vs. a vehicle, only 70% of crashes would be avoided if traffic flow is prioritized in AV programming, compared to 93% for the prioritization of safety [14]. For Lubbe et al. [9], the introduction of automation technologies would only have a low impact on fatalities with passive safety (12–13%), while it would be up to 45–63% with more advanced technologies, and 33–41% for vulnerable users. These discrepancies show the complexity of evaluating the potential impact of AV introduction on road safety. However, although for some crash configurations we predict a lower positive impact of AVs, we can reasonably hypothesize that when crashes are not avoided with AVs, the impact speed would be reduced, lowering the severity of those crashes. Hence, we could imagine that some current situations triggering fatal crashes would turn into nonfatal situations, and some nonfatal crashes would turn into property damage-only crashes, increasing the relevance of AV introduction.

The use of real crash data and the “perfect” AVs hypothesis do not allow us to anticipate new behaviours or crashes specifically created by AV introduction, for example because of new dangerous behaviours of other road users towards AVs, or specific to AV behaviour on the road. Hence, we may have overestimated the beneficial effect of AVs, but our hypotheses are very strict, and because we considered AV replacement for one type of vehicle only, our estimates are probably in the lower range for the decrease of crashes we would truly observe with AV introduction. Additionally, without real-life data, which do not yet exist, the possible negative effect of AV introduction seems very difficult to predict.

One of our strengths is that contrary to studies relying on licenced areas or isolated experimental environments, we used real traffic data. The VOIESUR database is very rich, with 5% of injury crashes and all fatal crashes observed over 1 year in France. As a result, it includes any type of infrastructure, which allowed us to calculate a relevant synthesis at a national level. More importantly, we were able to evaluate the impact on crashes of AV introduction for various configurations and different severities (injury/fatal crashes) separately. For fatalities, our results (56–62.8% of crashes avoided) are in the range of a decrease of 45–63% of fatal crashes proposed by Lubbe et al. [9] for advanced technologies. Although the suppression of single-LV crashes through AV introduction can be considered a strong hypothesis, it is also retained by other authors, such as Fahrenkrog et al. [3], and may therefore seem reasonable. In the context of our study, we assumed no change in the distribution of crash configurations since 2011. This is justified because the distribution of crash configurations in the French police crash data remains stable today (Additional file 1: Appendix Table A5).

Another strength is the questioning of experts specializing in vehicle technology and in specific types of road users. The difference in crash probabilities given by experts allowed us to provide a range of estimated avoided crashes. Unfortunately, for trucks, we had only one expert’s response, which may limit our trust in the result. However, there was good consistency between responses in general, except for the LV/M2W configuration (kappa = 0.36). For the LV/pedestrian crash configuration, our results (62.2–69.3% of crashes avoided within the configuration) were in line with those of previous studies, such as Utrainien and Pollanen [14], who found a decrease of 55–73%. Our work might have benefited from the consultation of a greater number of experts and more contextual information (e.g., weather, day/night information, urban/rural environment) to describe each crash sequence and increase the precision of our estimates. Taking into account the decrease in crash severity would also improve the quantification of the real impact on fatal crashes but would require additional hypotheses, for example, about speed reduction.

5 Conclusion

In conclusion, our simulation suggests that a reduction of approximately 60% of injury crashes and fatal crashes could be expected by replacing all LVs on the roads in France with AVs. Differences can be noted according to the ARU involved: the effect was lower for LV/M2W crashes and higher for LV/LV crashes.

This result represents encouraging prospects for the introduction of AVs into traffic. This deployment must be carried out cautiously, as even "ideally" operating level-5 AVs would not prevent all incidents.