Safety evaluation via conflict classification during automated shuttle bus service operations

Abstract

The widespread adoption of Connected and Automated Vehicles (CAVs) is being propelled, not only in the realm of private vehicles but also within transit systems. This development serves to enhance urban transport activities, rendering transportation more appealing to passengers. The present study aims to identify and examine the safety effects of testing different operational speed shuttle bus services in various future mobility conditions. To investigate impacts of autonomous shuttle bus services and to further examine their operational speed, the microscopic simulation method was performed. Specifically, four sets of simulation scenarios were comprised: a baseline scenario representing the current conditions and three operational speed scenarios (15 km/h, 30 km/h and 45 km/h) for an autonomous shuttle service. Each one of these sets included eleven CAV market penetration rates (MPRs) of CAVs of the general traffic (ranging from 0 to 100% in 10% increments). By analyzing the trajectory data extracted from microsimulation, traffic conflicts were identified and further analyzed by developing Mixed-Effects Multinomial Logit Regression models (ME-MLMs) in order to associate conflict type taking into account network characteristics as well as traffic conditions. Several aspects were determined as statistical significant parameters influencing type of conflict. The analysis yielded several significant findings that provide quantitative measurements and assessments of the effects observed, enabling a better understanding of the safety implications associated with the widespread adoption of automated services.

1 Introduction

In the coming decades, it is anticipated that Connected and Autonomous Vehicles (CAVs) will become increasingly common on urban road networks. CAVs have the potential to bring about significant changes in how transportation and road systems function. Specifically, CAVs are expected to enhance road capacity, improve fuel efficiency, and reduce harmful environmental emissions, as noted in several studies [9, 10, 16, 36].

In terms of road safety, the dominance of CAVs is expected to lead to a significant reduction in crash numbers. Since there is lack of reliable and generalized crash data, especially for high market penetration rates (MPRs) of CAVs, the microscopic traffic simulation method is considered as a very promising technique for studying automated mobility aspects including road safety. In particular, a microsimulation study conducted by Elawady et al. [7] investigated the impact of CAVs on intersection traffic safety under different MPRs. Similarly, several simulation studies have explored safety considerations in the context of automated mobility (e.g., [3, 6, 13, 15, 28, 30,31,32, 37]).

Several studies have explored the safety implications of the advent of automation, particularly regarding network-wide conflicts, and some have delved into the impact of increasing MPRs of CAVs in the overall traffic composition and hence mixed traffic conditions [3, 13, 21, 31]. Focusing on MPR, the steadily rising MPRs of CAVs appear poised to reduce travel times, as presented in a study by Ziakopoulos et al. [38]. Moreover, fewer traffic conflicts are observed for higher MPR of CAVs and mixed traffic conditions (conventional vehicles and autonomous shuttles and passenger cars) as highlighted in a study conducted by Oikonomou et al. [19]. Notably, Papadoulis et al. [21] highlighted MPR impacts and specifically found that as the MPR of CAVs rise, significant decreases in road conflicts could occur.

Focusing on public transport, automation is being expected to rapidly advance, not only in the realm of private vehicles but also within transit systems. Automated shuttle bus services, are expected to be among the first to line up with their large-scale business cases, aiming to enhance urban mobility and make public transit options more attractive to commuters. Findings from a research conducted by Ziakopoulos et al. [38] indicate that an autonomous shuttle bus service operation has a significant effect on cumulative travel time per segment as well as CO2 emissions per road segment. Additionally, point-to-point shuttle services utilizing dedicated lanes experience fewer delays when compared to mixed traffic situations, as indicated by Oikonomou et al. [19].

It is crucial to note that, outside of simulations, fully independent CAVs have not yet been deployed for unhindered operation in real traffic conditions, and thus, analysts must turn towards simulated environments to conduct related research. Based on recent literature, it noticeable that traffic simulation methodology has been widely used in transportation engineering, albeit not purely aiming to analyze complex transportation aspects in terms of traffic, as it is already known, but in terms of road safety as well. One of the most common way to study safety using microscopic models is to identify traffic conflicts by using the Surrogate Safety Assessment Model (SSAM) software, a model developed by Federal Highway Administration [24]. The software analyzes the vehicle trajectory data and identifies conflicts. Specifically, a conflict is identified when the Time-To-Collision (TTC) and Post-Encroachment Time (PET) are lower from preset thresholds, as identified in early studies exploring the possibility of using microscopic simulation for road safety assessments [4].

A variety of microsimulation studies identified conflicts to evaluate consequences on traffic safety of different transportation planning [23], control policies [14, 26, 29], road configurations [5, 11, 12] as well as transportation innovations [7, 17, 35]. Another simulation study also examined different conflict types exclusively on intersections (crossing, rear-end, and lane change) and created a probabilistic crash propensity model, incorporating reaction time and maximum braking rate distributions [33], however it was conducted significantly earlier. A recent study revealed that lane change conflicts lead to higher crash rates compared to rear-end conflicts [20].

Consequently, this is in line with the increasing popularity of Surrogate Safety Measures, and the increased utility they provide in proactive road safety analyses [18]. Consequently, using microsimulation the road safety assessment is feasible, as several approaches used suitable methodological frameworks and tools. In addition, it can be conclude that the most common technique is the conflict-based approach that enables the investigation of safety without the need of field crash data.

Despite the significant progress achieved, there is still serious concern regarding road safety assessments when applying traffic simulation, due to the absence of suitable analyses for investigating various road safety aspects. Only a few studies have attempted to overcome this issue and therefore, further investigation of past modelling approaches for road safety assessment is essential. Additionally, even fewer studies investigated automated urban mobility with regards to road safety. This research gap is the primary motivation behind the current study, with a particular focus on conflict types. The estimation of surrogate safety measures is deemed a dependable approach to assess safety of network traffic [34]. In addition, this study was also inspired by research conducted within the EU H2020 SHOW project, which aims at shared automation operating models development for worldwide adoption.

Therefore, the current study focuses on evaluating the factors influencing various types of traffic conflicts for different autonomous shuttle bus services as well as MPR of CAVs of the general traffic (i.e. regardless of shuttle service) taking into account network characteristics. To achieve this research aim, a dense urban traffic network located in Madrid, Spain was employed. Realistic data from the network and traffic were integrated into the Aimsun Next software; the used simulation tool. Vehicle trajectories were extracted from the microscopic simulation, and these trajectories were subsequently analyzed using the SSAM software. SSAM software was instrumental in identifying conflicts and categorizing them into three different conflict types, namely crossing, rear-end, and lane change. Following the extraction of traffic conflicts and their respective types, statistical models were developed with an aim to pinpoint the factors that contribute to the specific conflict types within the network traffic.

This study is structured as follows; after the current introduction to the study topic and aim, the method follows, including four main subsections. The first subsection relates to the simulation aim, preparation, and network. The second one introduces the surrogate safety analysis used and the third one relates to the data analyzed by this study and their descriptive statistics. The fourth one presents the theoretical background of mixed-effects multinomial logit regression that was used for the statistical analysis. Afterwards, results are presented by including the deployed model and main outcomes derived from the analysis data along with a comprehensive discussion of the key outcomes. Finally, overall conclusions are presented.

2 Methods

2.1 Microscopic simulation

To investigate safety impacts of automated shuttle bus services that differentiate in operational speed, the microscopic simulation method was performed. Within this context, various scenarios were tested using the Aimsun Next mobility software simulating the Villaverde district of the city of Madrid, Spain. The simulated network consisted of 668 road segments with a total length of 23 km and 365 nodes reaching approximately 2km² as shown in Fig. 1. The network geometry was exported from the OpenStreetMap digital map platform. In addition, the network was calibrated according to real traffic data. In specific, the model included traffic volume data for the morning peak hour that were collected in 2018 from 80 detectors and were provided by the Empresa Municipal de Transportes de Madrid (EMT Madrid) company. The detectors recorded traffic volume in vehicles per time. Those data were used in order the network travel demand for the morning peak hour to be simulated. The resulted from calibration Origin–Destination (OD) matrices of passenger cars and trucks included 30 × 30 centroids and corresponded to a travel demand of 5,784 and 716 trips for passenger cars and trucks, respectively. The existing conventional public transport of the study area was also included in the simulated network and specifically, 23 conventional bus lines along with 39 public transport stops, frequencies and waiting times at stops were considered.

Fig. 1

The simulated network

In the aforementioned network, an autonomous shuttle bus line was implemented as depicted in Fig. 2. This line was designed to operate in parallel with the existing public transport (the 23 bus lines) and connected the “La Nave”, a public facility that encompasses numerous activities, with the “Villaverde Bajo Cruce” subway station. The route of this line was circular with two bus stops in total and its length was 1.6 km. The fleet composed of one electric autonomous shuttle bus: Irizar SAE J3016 [27] level 4, which is shown in Fig. 2, operating with a frequency of 15 min, resulting in four departures in the simulated peak hour. The shuttle bus dimensions were 12 m in length and 2.55 m in width and had a total capacity of 60 passengers and 25 passengers seating. Its maximum desired speed was 60 km/h, maximum acceleration 1.36 m/s², maximum deceleration 10 m/s² and weight 15,845 kg.

Fig. 2

a The route and b the autonomous shuttle bus of the implemented service

Within the present research, three services differentiated in operational speed (15 km/h, 30 km/h, and 45 km/h) are investigated and hence three different sets of simulations were considered as well as one set representing the current conditions (baseline) without the shuttle bus operation. Each set represented the corresponding service, including eleven Market Penetration Rates (MPR) of CAVs scenarios (from 0 to 100% with 10% increments). The CAV MPR concerned both passenger cars and trucks and replaced the respective conventional vehicle percentages. Consequently, forty-four microscopic simulation scenarios were formulated and for each one ten different replications with random seeds were simulated as well. From the simulation of these scenarios, traffic data was recorded every 10 simulation minutes. Furthermore, the vehicle trajectories were also extracted per 0.4 s, equal to the simulation time step.

The CAV driving profile of passenger cars was simulated based on parameters provided in a study by Oikonomou et al. [20]. In that study, two driving profiles have been presented: 1st and 2nd generation CAVs, characterized as cautious and aggressive(in comparison to each other). For the present study, the second generation of CAVs is selected to model CAVs because it is expected to be more advanced and thus more representative of future networks.

For modelling autonomous trucks and the shuttle buses of the three services, it was assumed that their driving profile was more cautious than both CAVs and conventional human-driven vehicles due to their reduced values on maximum acceleration and deceleration. These driving profiles were defined setting various parameters in the Aimsun software as shown in Table 1, i.e. acceleration and deceleration, reaction time, lane changing model parameters and overtaking behaviour.

Table 1 Microsimulation vehicle parameters

2.2 Surrogate safety analysis

The vehicle trajectories extracted from simulation were analyzed using the Surrogate Safety Assessment Model (SSAM) software, a model developed by Federal Highway Administration [24], in order for traffic conflicts to be identified. Within the software, a conflict is identified when the time-to-collision (TTC) and post-encroachment time (PET) are lower from preset thresholds, with 1.5 s and 5.0 s default values, respectively. In the present study, the TTC threshold value was different in case of CAVs due to their smaller standstill distance and was set to 0.5 s instead of 1.5 s, based on the framework conducted through the a recent study [20].

Through the surrogate safety analysis, a dataset for each scenario set was extracted. These datasets included information regarding all conflicts occurred during the simulation time and specifically each row represented one conflict. Each row of the data represented one conflict by offering measures regarding the conditions that the conflict occurred such as its type, involved vehicle IDs, road segment ID where the conflict occurred and multiple surrogate safety measures (i.e. TTC, PET, speed, heading, deceleration, etc.). Afterwards, the vehicle IDs were matched with the corresponding vehicle types by using a relevant Application Programming Interface (API) in Aimsun software. More information regarding functions related to vehicle information in Aimsun Next can be found at Aimsun Next Users Manual (22.0.1) [2]. Similarly, the road segment IDs were matched with multiple characteristics derived from the network through the Aimsun software.

2.3 Data and descriptive statistics

The conflict database (each row representing one conflict: 638,163 rows in total) was structured in order to be analyzed and consequently investigate the relationship of traffic conflict type with regards to CAV MPR, traffic and network characteristics as well as several safety measures. Specifically, minimum PET observed during the conflict, CAV MPR (as a percentage, i.e., 0–100%), shuttle bus operational speed scenario, maximum difference in vehicle speeds of the involved vehicles in the occurred conflict, conflict angle, number of lanes, number of public transport lines, type, lane, length and width of the leading and following-vehicles, number of lane changes, speed difference of the involved vehicles, speed limit, conflict type (i.e., rear-end, lane change, and crossing), road type and traffic control type (i.e., give way, stop sign, traffic light and none) were included in the final dataset.

The numerical and integer as well as factor variable descriptive statistics are presented in Table 2 and 2, respectively. In Table 2, the data source (Aimsun or SSAM software), type of measurement, a short description as well as units, and descriptive statistics i.e. sample size (N), minimum value (min), median, mean, maximum value (max), and standard deviation (Std.) are given.

Table 2 Descriptive statistics of numeric and integer variables

Similarly, in Table 3 the data origin (Aimsun or SSAM software), variable type, a short description, the levels of the variables and descriptive statistics i.e. sample size (N) and percentage (%) are provided.

Table 3 Descriptive statistics of factor variables

2.4 Mixed-effects multinomial logit regression

The aim of the present study entails the classification of a dependent (or response) variable, i.e. conflict types while taking into account network characteristics, which would be independent (or explanatory) variables. However, it was necessary to account for differences in the various scenarios, such as increases of MPR of general traffic CAVs or increases in the adopted speed profile of the automated shuttle. Thus, a classification model was needed which would allow for flexibility and variation in its coefficients based on groups of the explanatory variables.

Therefore, the selected models for implementation fitting the above description were the Mixed-Effects Multinomial Logit Regression models (ME-MLMs), i.e. multinomial regression models containing random effects in the form of random intercepts. The multinomial logit regression link is well-established in the literature, therefore a brief outline is provided here solely based on more extensive works [1, 22]. The main linear predictor function is:

$$logit\left(Pr\left({Y}_{i}=c\right)\right)={\beta }_{c}{X}_{i}+{u}_{i}{Z}_{i}$$

(1)

Where $Pr\left({Y}_{i}=c\right)$ denotes the probability of ${Y}_{i}$, the dependent variable, belonging to category c, one of the C categories present in the sample overall. The fixed-effects part of the model is expressed by the independent variables ${X}_{i}$, which are regulated by the fixed-effects coefficients ${\beta }_{c}$, associated with each category c. The random-effects part of the model is expressed by the random predictor variables ${Z}_{i}$, regulated by the random-effects coefficients ${u}_{i}$ which follow a normal multivariate distribution (governed by within-group correlations).

For computational reasons during the ME-MLM fitting, the simulated data underwent z-score scaling, a common standardization process which does not affect the obtained coefficients. Mathematically, for every parameter $x$ with a mean $\overline{x }$ and a standard deviation $S$ a scaled value is obtained:

$${x}_{scaled}=(x-\overline{x })/S$$

(2)

The best-fitting model which contains the more informative variable combination and explains the highest degree of variance per given dataset is selected as the one with the smallest residual deviance and larger differences in deviance when comparing consecutive models, as this indicates an improvement in model fit. This is determined by ANOVA (log-likelihood test) between the fixed effects baseline and the various configurations of the model. Within this study, R-studio has been used [25] for the analyses, and specifically ME-MLM models are applied using the mclogit package by Elff [8].

3 Results

Traffic conflicts can be characterized as maneuvers, constituting parameters describing physical movement of the vehicles. The target of the present analysis is to classify the three conflict types (rear-end, lane change and crossing conflicts) of the present research based on an array of independent variables. To achieve this target, as the SHOW project provided a wealth of data, a series of ME-MLMs were fitted with varying configurations. After a trial phase, it was determined that a model featuring a series of geometrical, network and automated traffic characteristics, while including variables describing the first and second vehicle involved in each conflict, displayed the optimal performance.

The random effects constitute additional mathematical terms in the model that serve to better adapt the classification algorithm to the specific dataset, expressed in this case with random intercepts per specific variables. In other words, the constant of the model is allowed to vary across groups of a designated variable. The random part of the optimal model comprised random intercepts for each MPR level of CAVs in the network. Τhe comparison is shown in Table 4 below for a baseline fixed-effects model and a competitor model that comprised random intercepts per shuttle bus speed scenario; various other configurations were tested as well but showed poorer performance. In Table 4, the model family and configuration, along with residual degree of freedom (df), residual deviance, degree of freedom (df) and difference of deviance are included.

Table 4 ANOVA Log-likelihood comparison of MLM models

As evident, the third variant has a lower residual deviance, and a larger difference of deviance than its competitors, thus it is selected as the optimal model from the analysis. In this model, crossing conflicts are used as reference category, and the results of lane change and rear-end conflicts are compared and interpreted against this category. Model results, i.e., Coefficient, Standard Error (SE), Odds Ratio (OR), Confidence Interval (CI) and p value (p), are shown in Table 5, for the optimal model including random intercepts for MPR.

Table 5 ME-MLM results with crossing conflicts as reference category (statistically significant predictors appear in bold)

Moreover, ORs can also be visualized by contribution in a logarithmic scale, as shown in Fig. 3.

Fig. 3

Odds ratio contributions of each variable in the model (blue ≥ 1, red < 1)

The interpretation of the results against the crossing conflict category is quite straightforward, and it is presented in the following Discussion section. For significant variables, an OR higher than 1 denotes a variable that contributes to each observation falling into the examined category compared to crossing conflicts through a multiplication by a factor of ${e}^{OR}$, all else remaining constant.

Furthermore, the random effects of the model were found to be statistically significant, expressed as random intercepts, based on Table 5. In other words, each MPR value in the examined range provides a unique constant term to the model apart from the universal constant term of the regression. The values of these extra terms can be visualized in Fig. 4. Specifically, each random intercept is shown in the chart, colored by conflict type (lane change random effects are shown in orange, while rear-end effects are shown in pink). In addition, the dot size represents the substrata size, i.e. the frequency of the subsample where MPR has the corresponding percentage value, and in which the random effect is applied.

Fig. 4

Random intercepts per MPR for each conflict type

It can be deduced that the random effects fluctuate more in lower MPR values for rear-end conflicts, while random effects fluctuate more in higher MPR values for lane change conflicts. Thus, MPR levels are considered to meaningfully contribute towards explaining the variance of the conflict type response variable. In other words, these random effects show the manner in which each MPR percentage contributes towards a specific conflict type generation compared to others.

The distributions of the three probability density curves (one for each conflict category) are shown on Fig. 5. Each probability density curve represents the distribution of predicted probabilities for each conflict type generated by the model. The x-axis shows the probability score of each category given the model predictions, while the y-axis represents the density of those probabilities, which indicating how frequently different probability values occur within the sample. The plot aims to illustrate how the model’s predictions are distributed across different conflict categories.

Fig. 5

Probability allocation per conflict category from the ME-MLM

4 Discussion

At this stage, the interpretation of the model results can be conducted, after examining the previous Tables and Figures. Critical inputs are derived from the coefficients of Table 5 have been visually represented in Fig. 6 to facilitate comparative evaluation.

Fig. 6

Graphical representation of ME-MLM coefficients

Indicatively, if PET increases by one unit while all other variables remain constant, the odds of a conflict observation belonging to the lane change conflict increase by a factor of ${e}^{0.329}=1.39$, while the odds of a conflict observation belonging to the rear-end conflict increase by a factor of ${e}^{0.656}=1.93$. These results are intuitive, as PET increases are more closely related to reduced lane changing margins, while they are absolutely critical to the creation of rear-end conflicts and crashes compared to crossing conflicts, hence the much higher OR.

In a similar manner, it can be surmised that higher MPR and higher maximum speed difference (MaxDeltaV) between vehicles lead to reduced probabilities that a conflict will be of the lane change or rear-end types compared to the crossing type. In other words, more CAVs in the network, or vehicles with higher speed differences lead to more crossing conflicts. Moreover, different control types and no control type generally increase the probability of lane change or rear-end types compared to crossing conflicts, relatively to the ‘Give way’ control type. The only exception is the ‘Stop’ control type which reduces lane change probability only compared to crossing conflicts, while increasing rear-ending probability.

Compared to primary roads, circulation in any other road type leads to reduced probabilities that a conflict will be of the lane change or rear-end types compared to crossing conflicts. Higher speed limits lead to more rear-end conflicts, but less lane changing conflicts compared to crossing conflicts.

The shuttle bus operational speed for Irizar buses led to more rear-end conflicts compared to crossing conflicts when it was 15 km/h and 30 km/h, which can be interpreted as a ‘moving disruption’ that simulated vehicles encounter while moving at higher speeds and then suddenly braking behind the automated shuttle. Increased numbers of overall lanes on the segment of circulation constitute lane changing and rear-end conflicts more likely compared to crossing conflicts, however, increased numbers of public transport lanes inverse these effects, making crossing conflicts more likely.

Regarding first (leading) and second (following) vehicle parameters, i.e. first heading (i.e. headway), width, length and first vehicle type (compared to conventional buses), are mostly found to reduce lane change or rear-end conflicts compared to crossing conflicts overall, with some non-statistical significant effects present. On the other side, second heading increases lane change or rear-end conflict chances of appearance compared to crossing conflicts overall.

Second (following) vehicle types other than conventional buses generate more lane change conflicts but less rear-end conflicts compared to crossing conflicts, apart from shuttle buses which generate both more lane change conflicts and rear-end conflicts. This appears sensible due to lack of agility characterizing buses, and the fact that they have to comply with lower operational speeds as a results. The particular lane of movement for first vehicles increases likelihood of lane change and rear-end conflicts compared to crossing conflicts. For second vehicles, the likelihood of rear-end conflicts similarly decreases, while lane change conflicts increase instead.

Lastly, in multiclass classification models, sharper curves denote more concentrated density around the correct categories, indicating higher certainty in predictions. Based on Fig. 5, the model shows a satisfactory certainty performance judging by density sharpness.

The present research effort naturally includes some limitations. A considerable part of the limitations pertains to the use of traffic microsimulation. In particular, there are no pedestrians integrated in the models, and there is no illegal behavior encoded therein, in terms of adherence to speed limits for any vehicle or impaired driving (for the conventional vehicle drivers). On a related note, due to coding restrictions, crash conditions are excluded from occurring in the microscopic simulation environment. Some assumptions in the CAV profiles always exist, as it is anticipated that different manufacturers will not use the exact same settings in their Artificial Intelligence pilots. Regarding the applied ME-MLM model, the obtained results ought to remain valid as effects, however, more efforts would be needed for a broader, more universal sample to achieve higher transferability of results. Random effects are typically harder to transfer due to their mathematical nature, however, the high-level conclusion that different traffic mixes of varying MPRs impact how conflicts are generated can be anticipated in other study areas as well.

Notably, conflicts are not necessarily harsh events or near misses, and certainly not crashes. Therefore, steps should be taken to even more solid safety indicators. A related future research direction would be to examine the impact of harsh braking events on safety within automated transit systems, and the extent to these can serve as surrogate safety measures, potentially supplying statistical inferences of simulated crashes [20], extending the existing research and offering insights into potential mitigative measures.

5 Conclusions

The analysis yielded several significant findings that quantified safety impacts of automated services in various levels of CAV market penetration. These findings provide quantitative measurements and assessments of the effects observed, enabling a better understanding of the safety implications associated with the widespread adoption of automated services. The quantification of safety impacts is considered as highly important as it enables stakeholders to make informed decisions regarding the deployment and operation of automated services.

It is evident that a large array of variables influence conflict type classification. Road type, infrastructure elements (such as total and public transport lane number), first and second vehicle characteristics and lanes of movement all affect classification outcomes between crossing, lane change and rear-end conflicts. More macroscopically, results indicate new and unexplored possibilities of novel types of road safety assessments, many of which can be proactive, and as such they can be applied in uncharted study areas before crashes occur. The combination of traffic simulation and statistical/econometric models provides undeniably promising venues, which can be better materialized if the respective data analyses is conducted across sites in a standardized manner, enabling better validation and forecast capabilities.

The analysis of safety impacts of automated services, such as these provided by the present study, highlights the need for informed policymaking. Quantifying these impacts provides crucial data for developing regulatory frameworks tailored to autonomous vehicle technologies. As per the aforementioned, it can be deduced that varying MPRs impact how and what types of conflicts are generated, to a degree. Therefore, policymakers and related stakeholders must be mindful during all stages of AV integration into their transport systems, as fluctuations of safety levels may occur. These findings also can be important in specific parts of a wider transport network, where, due to socioeconomic, geographical, practical or other factors MPR may change drastically compared to the average, with the different types of conflicts manifesting there.