- Original Paper
- Open access
- Published:
Detection of anomalies in cycling behavior with convolutional neural network and deep learning
European Transport Research Review volume 15, Article number: 9 (2023)
Abstract
Background
Cycling has always been considered a sustainable and healthy mode of transport. With the increasing concerns of greenhouse gases and pollution, policy makers are intended to support cycling as commuter mode of transport. Moreover, during Covid-19 period, cycling was further appreciated by citizens as an individual opportunity of mobility. Unfortunately, bicyclist safety has become a challenge with growing number of bicyclists in the 21st century. When compared to the traditional road safety network screening, availability of suitable data for bicycle based crashes is more difficult. In such framework, new technologies based smart cities may require new opportunities of data collection and analysis.
Methods
This research presents bicycle data requirements and treatment to get suitable information by using GPS device. Mainly, this paper proposed a deep learning-based approach “BeST-DAD” to detect anomalies and spot dangerous points on map for bicyclist to avoid a critical safety event (CSE). BeST-DAD follows Convolutional Neural Network and Autoencoder (AE) for anomaly detection. Proposed model optimization is carried out by testing different data features and BeST-DAD parameter settings, while another comparison performance is carried out between BeST-DAD and Principal Component Analysis (PCA).
Result
BeST-DAD over perform than traditional PCA statistical approaches for anomaly detection by achieving 77% of the F-score. When the trained model is tested with data from different users, 100% recall is recorded for individual user’s trained models.
Conclusion
The research results support the notion that proper GPS trajectory data and deep learning classification can be applied to identify anomalies in cycling behavior.
1 Introduction
Cycling is a key component of any sustainable urban mobility in terms of environment and public health and as an alternative to driving a car [42]. The Netherlands is leading the ranking in Europe with 27% of trips done by bicycle with other countries (e.g. Denmark, Belgium, and Germany) already beyond the 10% threshold. Below 5% we find countries like Norway (4, 3%), Italy (3, 3%), France (2, 7%) and Luxembourg (2%) [45]. Anyway, all of them report considerable increases in bicycle usage further pushed due to the Corona crises in 2020 [26].
Unfortunately, as bicycle use increases, at the same time the rate of bicycles involved in road crashes has increased, as well. Due to the vulnerability of bicyclists to serious injuries, it has been estimated that riding a bike is seven times more unsafe than travelling by car [21]. Data coming from European statistics shows that the rate of fatal accidents for cyclists in urban roads has increased from 2010 to 2018 by + 6% in contrast to the decrease of all the other modes of transport (Fig. 1) [18].
Moreover, the high risk of crashes when perceived by the users acts as strong barriers, dissuading people from using bicycles as a form of transport [46, 54]. A critical area for cycling safety research is the underreporting of cyclist crashes [29],Evelien [20] and lack of reliable data about cycling travelled distance. Even in highly cycling countries, 50% of bicycle involved traffic accidents are not reported in police statistics [22, 58]. Consequently, the crash statistics are biased in the magnitude and exposure, and less feasible than for motorized vehicles [53].
Accordingly, Traffic Conflicts, near miss or Critical Safety Events (CSEs) are spreading as crash surrogate in safety studies [2]. CSE is defined as “A traffic event that requires a rapid evasive maneuver by the subject vehicle, or any other vehicle, pedestrian, cyclist, or animal to avoid a crash. A rapid, evasive maneuver can be steering, braking, accelerating, or combination of control inputs” [57]. Bivariate extreme value models showed that pairs of temporal and speed-related indicators should be combined in order to properly predict severity of surrogate measures of safety [7].
Loss of control, turning, breaking and overtaking are recurrent maneuvers in CSE involving bicyclists [17, 51]. These “evasive maneuvers” are “anomalies” in the normal ride behavior whose identification is a complex task due to high dimensionality and heterogeneity trajectory data (e.g. speed, acceleration, direction) for which deep learning models for anomaly detection [61] may be more efficient than traditional statistical methods [3].
In this framework, the main contribution of this study is the development of an experimental framework where convolutional neural network (CNN) deep learning is originally applied to integrate multiple GPS data streams of bicycle kinematics to detect anomalies in the cycling behavior which are associated to evasive maneuvers in the occurrence of CSEs. Validation of the results with real data and higher performance than traditional threshold based, and statistical techniques makes the proposed approach promising in order to identify location with potential hazard for cyclists by using mobility data that can be easily collected in smart cities and communities. A case study in the city of Catania, is presented as well.
The paper is organized in the following sections:
Section 2 includes an overview of observational studies about bicyclist safety and application of CNN to road mobility and safety. Section 3 the overall method is presented in its different conceptual and operative modules which includes Dataset preparation (GPS data collection (typology and frequency), signal smoothing and cycling parameters calculation, data labeling into normal and abnormal) and Neural Network Model Synthesis (training of the convolutional autoencoder by defining architecture and setting the model parameters to perform anomaly detection). Section 4 the detection performance methodology is validated with real SCEs; different model settings are compared, and superior performance is observed over traditional detection techniques is shown. Validation through Case study is also carried out in Sect. 4 to demonstrate how results can be used in practical application. Section 5 is about proposed method, results and future recommendations are reported at the end of the paper.
2 Related background
Section 2.1 we will provide an overview of observational studies in cyclist safety of, while in Sect. 2.2 we will focus on the research activities related to the use of deep learning and CNNs for road safety and bicyclist mobility.
2.1 Observational studies
Literature is extensive about safety assessment using observational studies, but in comparison a limited number of studies are applied to bicyclists [2]. In the InDeV project [38], the Safe VRU app was developed for self-reporting of accidents and near-accidents and has been used by more than 400 participants [37]. The target of the UDRIVE project [49] was to identify factors in CSEs involving a bicyclist; CSEs were identified manually and correlated to the features of the infrastructure [52]. In the project BIKEALYZE, data was collected by a mobile eye tracking, a GPS-based motion data acquisition complemented with acceleration and steering direction data; CSEs (e.g. collision avoidance, way-giving violations, abrupt braking, abrupt turnout) have been identified by video-based analysis and elicitation interviews [13].
In several studies participants had an active role to indicate any CSE they experienced via a push-button on the vehicle [60] or in a smartphone app [37] or by online questionnaires [24]. A study in Sweden [16] collected movement data of 20 bicyclists with an Inertial Measurement Unit (IMU) and GPS installed on an instrumented bicycle and analyzed cycling kinematics. Longitudinal and lateral accelerations have been considered relevant for analyzing cycling behavior. Another study showed that GPS data must be collected at least with a 1 Hz frequency to provide suitable speed profiles and to detect hard breaking of cyclists [41], while vertical accelerations acquired at least at 50 Hz by accelerometer sensors are required to analyze cyclist comfort and safety due to pavement unevenness [8]. Overtaking behaviour of motorized vehicles by measuring the lateral distance between the bike and passing vehicle and a statistical model was developed to predict the probability of an unsafe critical maneuver [40] and cyclists’ safety perception [48]. In [10], authors developed an algorithm to detect a cyclist downfall by combining acceleration and rotation thresholds. Strauss & Miranda-Moreno [55] proposed a procedure to use cyclist GPS data taken by a smartphone to calculate decelerations and correlate thresholds with the number of injuries. Despite the promising results, they concluded that more granular data and validation work needs to be done to improve the reliability of the correlation. Perceived risk resulted consistently with frequency of CSEs in bicycle paths classified by a panel of experts analyzing video recording and speed and heading GPS data [9]. All these research mainly rely on the identification of safety–critical events via self-reporting, manual review of video footage pre-selected thresholds and statistical methods to analyze data.
An extensive review [5] reported that the existing solutions for trajectory outlier detection were “algorithm based” (e.g. distance-based; density-based; pattern mining–based) with emerging machine learning–based schemes that learn the outlier detection from the training trajectories to identify anomalies in the newly inserted trajectories. Moreover, the research focused more on vehicle mobility [50, 59] and not to micro-mobility, such as bicycle, which also suffer of a “digital divide” when compared to the increasing opportunities for data collection through connected and automated vehicles. More specifically, no studies are reported for cycling trajectories [5].
2.2 Deep learning in road safety and bicyclist mobility
Recently, deep learning and Convolutional Neural Networks (CNN) have been applied in road safety studies [25, 30, 31, 39] and driving style analysis [6, 15]. The convolutional autoencoders (CAE) allowed the extraction of valuable information from large quantities of complex and heterogeneous data, showed fast convergence due to the convolutional layers, and provided better performance with multi-dimensional data compression and feature learning making the procedure well suitable for managing the mobility data characterized by high volume, variability and velocity (i.e. big Data) [36].
Dong et al. [15] made the first attempt of adopting a deep neural architecture, based on Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), to extract features on the driving style directly from GPS data. More recently, Bichicchi et al. [6] applied an unsupervised Denoising Stacked Autoencoder (SDAE) to provide output layers from kinematic measures tracked with an in-vehicle 10 Hz GPS device. The RGB colors of the outcomes were associated with different path geometries encountered during the driving.
When applied to cyclist mobility, deep learning and CNN have been used in the bike-sharing prediction modeling, because the use of shared bicycles is susceptible to time dependence and external factors [44], such as weather [4, 47], bike rebalancing and land use characteristics. In [14], authors applied the Self Organizing Map artificial neural network to identify atypical trajectories from video sequences at fixed locations. More recently in [1], authors used video records from fixed cameras and trajectory data extracted by means of computer vision algorithms and Advanced Artificial-Intelligent (AI) techniques to model cyclists’ behavior and their interactions with pedestrians in a shared space.
The limitations of the existing works for classification of abnormal cycling behavior are summarized as follows: observational studies applied on bicyclist safety mainly relay on traffic conflict techniques applied to video tracking from fixed positions. Few studies used trajectory data to identify SCE, but by self-reporting or handled classifications. In [33], authors uses text mining analytics and an Artificial Neural Network (ANN) to extract information from near-miss and collision event descriptions, acquired from BikeMaps.org, a global tool for mapping collision and near-miss events. Deep learning is becoming widely applied to transport and road safety studies, but application to cycling are mainly focused on mobility choice. Results from previous studies about CNN for anomaly detections or modeling of motorized driver behavior, cannot directly be transferred to cycling because of its specific kinematic features and limited availability of advanced equipment for data collection as in standard naturalistic studies.
To the best of our knowledge, this is the first work extending the use of deep learning CNN to extract features of the riding style of bicyclists from GPS data and to detect anomaly events in cycling behavior.
3 Methodology
The overall methodology consists of different tools that comprise data preparation and convolutional Neural Network training and testing as illustrated in Fig. 2.
3.1 Dataset preparation
Dataset preparation includes both collection and treatment of GPS data to extract features optimized to train the CNN model.
3.1.1 Data acquisition
The data source is an instrumented bicycle with GPS (Global Positioning System) and HD video system (Video Vbox Lite). The Video Vbox Lite (VVL) records an extended GPS NMEA dataset not limited at only latitude, longitude, but including also speed and heading at 10 HZ frequency synchronized with a video recording of 2 HD cameras. Data accuracy and resolution are reported in Table 1. Worthily to mention that GPS in the standard acquisition without augmentation has limited position accuracy, but good data quality in speed and heading derived from the Doppler method and Carrier Phase observations [63].
Data was collected from 10 cyclists, named from ID-1 to ID-10, who participated in controlled test rides. The ten cyclists included eight males and two females. Participants were between 27 and 65 years of age, whereas 40% was over 40 years old. On average, the cycling experience of users was uniform with a weekly cycling use. Only ID-4 was a highly experienced commuter cyclist with daily use of bicycle. Participants were instructed to ride the instrumented bicycle following their normal behavior. The test was carried out in normal weather, daylight, and traffic hours. The ride path was long around 4 km, traveling different road infrastructures, to provide different traffic and road environment conditions that include cycle track, bicycle/bus shared line, cycle track termini and one roundabout [41]. Dataset collected for each rider contains around 9000 samples at 10 Hz acquisition frequency. Regardless of the limited number of participants such dataset is appropriate for training purposes as we will discuss in the conclusions.
CSEs occurred during the test were identified by the research team reviewing the videos together with the test rider who explained about the occurrence of an actual CSE. A total of 41 CSEs have been detected and classified over about 2.5 h of tests. As an example, Fig. 3 shows a test path section with bicyclist GPS positions and time of NMEA data. The two blue dots mark the position of one CSE, while the blue boxes show the time interval of the CSE in the speed and heading profiles. Right corner of the map includes screenshot of recorded video.
3.1.2 Data preprocessing and feature derivation
Once data was recorded, different Python routines were applied to (1) Improve the data quality, (2) Interpolate for smoothing, (3) Calculate derived parameters and (4) Create the data set for training and testing the CNN. In the present application, speed (S) and heading (H) define the recorded time data in the NMEA string, while longitudinal acceleration (LA), traveled Distance (D), heading rate (HR), transversal acceleration (TA) and combined acceleration (CA) are derived by as shown in Eqs. (1–5).
where ΔT = 0.1 s, S speed in m/s, H heading in radiant, HR in rad/s, LA, LT and CA in m/s2.
Speed and heading data from GPS have a good standard accuracy as reported in Table 1. Anyway, environmental factors such as satellite view, signal blockage, atmospheric conditions can affect precision. Moreover, pedaling produces riding oscillation with frequencies around 2.5 Hz in the longitudinal speed and 1.2 Hz in the lateral direction [16] which can be considered as noise in the S and H signals, emphasized by the high frequency rate.
Therefore, we applied a Savitzky-Golay smoothing filter (SGF) to the speed and heading profiles, before calculating their derivatives (i.e., LA, HR, TA, CA). SGF is a digital filter [11, 32], well applied in GPS trajectory data of urban bus [34, 35], that we adapted to our time series dataset of speed and heading to increase the data precision without deforming the actual signal frequencies and shape, reducing noise and determining a smoothed trend line for deriving the other parameters.
Figure 4 illustrates the original data parameters speed and direction (heading) and their derivate LA and HR before (black line) and after (red line) applying SGF. It is evident the improvement in the signal smoothness especially for the derivate of LA and HR.
3.2 Neural network model synthesis
In this section, we propose a methodology that exploits a convolutional autoencoder for anomaly cycling detection in time series. Therefore, we will first provide CNN background and then, discussion on convolutional autoencoders (Sect. 3.2.1). Then, we will present the proposed methodology for anomaly detection in the application scenario (Sect. 3.2.2).
3.2.1 Preliminaries: convolutional neural network (CNN)
Deep learning is a recent technology used in several scenarios including the identification of anomalous points [12, 28]. Convolutional Neural Networks (CNNs) are types of deep learning algorithms, introduced to process images efficiently and are quite popular for anomaly detection as well [61, 62].
CNNs automatically extract features from the data that are used for classification purposes [34, 35]. The architecture of a CNN includes several layers that are classified in convolutional layers, pooling layers, and fully connected layers (Fig. 5). The convolutional layers are the first layers of a CNN, which contains filters in the form of weighted matrix (C1) and recognize patterns efficiently by reducing the variables dimension. Convolutional layers are followed by pooling layers (S) which can be repeated several times to summarize features. The last layer is the fully connected layer whose neurons (NN) take the extracted features as their input as shown in Fig. 5.
In our work we use Convolutional Autoencoders and thus, we will provide initial information of Autoencoder (AE). An AE is a type of artificial neural network and popular for anomaly detection, AE consists of two main modules: the encoder and the decoder (Fig. 6). The encoder maps the input data into a latent vector while the decoder tries to reconstruct the input from the latent vector.
The encoder is a neural network which is specified by a set of parameters which we call \({w}_{E}\). As we already said the encoder takes as input a n-tuple X and gives as output a m-tuple Z, which we call latent vector, with \(m\ll n\). Obviously, the latent vector Z is a function of the parameters \({w}_{E}\) and the input X as shown in Eq. (6).
The decoder is a neural network, specified by the parameters \({w}_{D}\) which takes as input the latent vector Z and gives as output a n-tuple \(\widehat{X}\), i.e. Eq. (7). The AE training minimizes the difference between the input X and the model \(\widehat{X}\) as shown in Eq. (8):
where \(\left\| {\hat{X},X} \right\|\) represents a measure of the difference between \(\hat{X}\), and X as shown in Eq. (9). Several ways of measuring such difference can be applied. Notable examples include the Mean Absolute Error (MAE), the Mean Squared Error (MSE), and the Root Mean Squared Error (RMSE). In our work we applied the MAE because it gave the best performance. This was expected; in fact, MSE and RMSE square errors before averaging, and therefore, they give higher weight to large errors. We opt MAE over Mean Square Error and Root Mean Square Error (RMSE) due to data distribution and error size suitability. Therefore,
Note that convolutional autoencoders (CAE)s are capable of learning the most useful feature patterns in the input data [27] and anomaly detection [61].
3.2.2 BeST-DAD model: the proposed CNN application for anomaly detection
We call the complete scheme proposed for anomaly detection in the scenario of interest: ‘Bicycle Safety through Deep learning-based Anomaly Detection’ (BeSt-DAD). Best-DAD employs a 1-D CAE as depicted in (Fig. 6). The input consists of a sequence of time-data samples \({, X}_{1},{X}_{2}, \dots , {X}_{6}\) generated at a frequency of 10 Hz. The generic \({X}_{i}\) is a 6-tuple of values, i.e., \({X}_{i}=\left[{x}_{i1},{x}_{i2},{x}_{i3},{x}_{i4}, {x}_{i5},{x}_{i6}\right]\) which represent the speed, heading, heading-rate, longitudinal acceleration, transversal acceleration, combined acceleration, calculated as discussed in the previous Sect. 3.1. Thus, the input data is 2-dimensional matrix in nature, as shown in Fig. 6, that we flatten as a 1-dimensional input sequence of type as shown in Eq. (10).
The number of samples j composing the input sequence as window size. Experiments show that a good value for \(j\) is \(j=40\). Therefore, the input size of the encoder is \(n=40\times 6=240\).
Figure 6 shows the overall architecture of BeSt-DAD where encoder consists of 2 convolutional layers. We applied Stride as an advance convolutional parameter which is capable to replace max pooling with less computation. Padding is used to maintain output dimension as input while activation function is responsible for neuron activation. In our case, each of convolutional layer reduces the input dimension of a factor equal to the stride, i.e., four. As a result, the output of the first convolutional layer has a dimension equal to 60, whereas the output of the second layer, that is, the latent vector Z, has dimension equal to \(m=15.\)
The decoder consists of two de-convolutional layers and a dropout layer which avoids model over fitting. The output of the decoder will have dimension again equal to 240 and compared to the input by calculating the mean absolute error (MAE). If this MAE is higher than a given threshold an anomaly warning is issued. Observe that the value of the such threshold is a critical parameter. We will discuss how to select it in the next section.
4 Results
In the following Sect. 4, the proposed scheme will be validated by comparing the actual CSE detected as discussed in Sect. 3.1.1, which we call “real positives”, to the anomalies in cycling behavior detected by BeST-DAD, which we call “CNN positives” (Sect. 4.2). For comparison analogous validations have been carried out by using Principal Component Analysis as robust standard statistical approach for feature reduction and anomaly detection, and the more widespread method based on setting a threshold in the longitudinal acceleration to identify hard breaking. Results are presented and analysis is done in Sects. 4.3 and 4.4, based on quantitative performance metrics presented in Sect. 4.1.
4.1 Performance metrics
In a binary classification, ‘’Positive’’ and ‘’Negative’’ assignments refer to the classifier’s prediction, and the terms ‘’True’’ and ‘’False’’ refer to whether that prediction corresponds to the real observation. Given these definitions, the confusion matrix (CM) describes the performance of the classification model as shown in Table 2.
CM is useful for calculating two metrics of classification performance called Recall and Precision. Precision (P) measures the rate of true positive (TP) over the total predicted positive (TP + FP). The Recall (R) computes model’s ability to detect TP over the total number of real positives (TP + FN). For our classification with unbalanced data due to the small number of real Positive, the F-measure (Fβ) is an effective quantitative metric to select the model setting which minimizes the errors [23]. Fβ score, in Eq. (11), is the weighted harmonic mean of precision and recall, that ranges between 0 and 1.
As commonly used to emphasize Precision against Recall, we applied a weight β = 2, because we are more interested in limiting FN (i.e., missing detection of CSEs) rather than FP.
4.2 Criteria for classification of CNN-positive
Our event detection criteria are illustrated in (Fig. 7) and defined as follows. Considering the observed time extension of a real CSE in the range of 0.8–3.1 s with an average of 1.4 s and the high time variability of the kinematic parameters in the cyclist riding, events with MAE > threshold of less than one second were not classified as “CNN positive” and detection sequences within 5 s have been classified as one CNN positives.
4.3 Model testing
The model performance measured by R, P and F2 is affected by several factors related to the data and CNN setting and varies by changing the classification threshold. Therefore, to evaluate the results, two major comparisons are carried out: first comparison case is carried out by using various model settings and features, while other comparison case is done between BeST-DAD and alternative detection approaches like Principal Component Analysis (PCA) and breaking acceleration threshold.
To maximize the performance of the model and to learn new insights about cycling behavior modeling, model setting and features relate to the following comparisons: CNN setting by changing (A) Training and testing, (B) Thresholds and (C) Size of the time windows; input dataset by varying (D) Use of SGF filter and (E) Use of only speed related variables: speed and longitudinal acceleration or (F) Use of only heading related variables: heading and Heading rate.
Furthermore, two scenarios have been selected for training and testing of the model:
(A1) Training and Testing CNN for each user, by using 80% of data for training and 20% of data for testing. The average training time in scenario A1 is 30.142 s for each user.
(A2) Training with the data collected by considering the entire dataset related to only one user and testing with 100% of the data from all the other users. The average training time in scenario A2 is 26.642 s at all. Results and values of Recall, Precision and F2 are reported in Table 3.
For the first scenario (A1), where each user adopts its own model, we reached the absolute best performance with an average F2 = 0.77. In the second scenario (A2), the model is trained with the full dataset data of one-by-one user and tested with 100% of data from the other users with an average F2 = 0.72. Anyway, training with ID-4 as reference user and testing with all other users gave the best F2 score of 0.77. While comparing the two different training approaches, we applied the best CNN settings in Table 3, with SGF, 40 TS-window, and all input parameters.
Results in Table 3 for scenario A1 show that training tailored CNN models for each user returned the best results. Anyway, in practical application, scenario A1 means the need to train BeST-DAD model for each user sharing his/her cycling data. While in scenario A2, transferring the model trained on one user returned slightly worse performances, but an overall average F2 comparable with the previous scenario, as well.
The previous result illustrates the reason of “ID-4” selection as 100% training dataset for the further comparisons, given the good performance and availability of a larger dataset for testing purposes (i.e., 100% of data for user ID-1, …, ID-9) by using the full set of 41 real-positives for testing.
Results for the other validation scenarios (B, C, D, E, F) are presented in Table 4. First, we tested CNN setting for different thresholds (T) and time window sizes (TW) (scenarios B, C). Results in Table 4 confirm the best performance when T = 0.18 and TW = 40. Best values of TW and T also have meaning in explaining the cycling behavior. MAE = 0.18 is equal to the 88th percentile of the overall Loss values, while MAE = 0.21 is the 93rd percentile, showing that anomaly cycling behaviors for evasive maneuvers are quite rare (12% of the cycling time), but not exceptional events in the riding path especially in shared lanes as will be highlighted in Sect. 5.
Without the application of the SGF filter the high-frequency time-variability of the cycling data returned many CNN-positive with high Recall, but also many FPs returning a very low Precision (P = 0.36). It is worth noting that merging speed and heading parameters had the most important impact in improving the model performance. Results showed that without including heading derived parameters (i.e., HR, TA, CA), the model performance decreases significantly by missing several detections with more FNs and FPs (low R and P in Table 4). That is expected for cyclists, rather than other road users, because they apply both braking and swerving as evasive maneuver. Analyzing results, when Precision and Recall are compared it is noteworthy that Recall is always higher than Precision. This is significant because, in our application scenario missing CSE (FN) is of higher concern than False Positive. Moreover, FP may not be wrong detection of cycling anomalies, but often have been identified as changes of cycling behavior related to other events not classified as CSE (e.g., hard braking at traffic lights, avoiding pavement bumps, potholes etc.).
Finally, to further evaluate the effectiveness of deep learning in detecting CSEs, the performance metrics have been calculated also by applying a robust PCA statistic and the traditional empirical approach based on breaking acceleration threshold [56]. Results in Table 4 confirm the higher performance of deep learning for event classification.
4.4 Validation through case study and risk assessment
In order to show the practical results of BeST-DAD for risk assessment and ranking, the procedure was applied to the trajectory data and anomaly detections located with the GPS coordinates in the different roadway sections composing a mixed cycling path (i.e. cycle track, roundabout, cycle track termini and bicycle/bus shared lane) (Fig. 8.).
The exposure to the occurrence of a conflict rises with the time the bicyclist spends in the road section and therefore cycling time was considered as an exposure metric to rate the number of anomalies among different roadway components to make the results comparable.
Risk Rate = N. of anomalies/cycling time.
The mean cycling time is reported in Fig. 9 which also shows the total number of BeST-DAD anomaly detections in the different road sections travelled during the test and the comparison between observed and CNN risk rates in a normalized scale (0–1).
Figure 9 shows cycle track termini at the highest risk rank followed by the roundabout. The normalized Risk Rate are also calculated to allow for comparisons with the observed risk rate reported in previous study [9]. It should be noted that in Fig. 9, the very good consistency between the two-risk ratings for cycling track and shared lane. Also consistent are the higher rating for lane termini and roundabout, even if BeST-DAD has classified a higher risk at the Cycle track termini than the one observed by actual CSE. The cycle termini end with a sharp curve before the lane crossing (Fig. 9) where cyclists were required to steer and often stop showing as anomalies in riding behavior (e.g. hard braking and steering) that have been correctly detected by BeST-DAD although not specifically related to traffic conflicts and therefore classified as FPs.
5 Conclusion
Cyclists are vulnerable road users, and their safety is a serious issue to be addressed with increasing number of fatalities among cyclists. Due to the lack of reliable data for crash analysis and the opportunities to collect new data in the smart cities and bicyclist communities, innovative observational studies can offer new approaches for a network-wide safety assessment of VRUs consistent with the EU directive [19].
Auto-encoders are mainly utilized for dimensionality reduction, feature extraction, image denoising, anomaly detection and image compression [6]. In the authors- knowledge that is the first attempt to use both speed and direction GPS data with customized Convolutional Autoencoder to automatically detect anomalies in cycling behavior that can be associated to critical safety events (CSE) and plotted on map as risky points.
Performance of the classification was very good considering the low rate of FN with Recall of 100% in 6 out of 9 tests after individual training of the model (Table 3). Furthermore, in scenario A2 (Sect. 5), we have seen that also a model trained using the dataset for one selected rider can be effectively transferred to the other riders with R = 0.78 and F2 = 0.77. This result is interesting because, in large scale applications, the use of a pre-trained model results in the reduction of communication and energy resources and more suitable to protect the user privacy.
Performance evaluation of BeST-DAD for different model settings (Table 4) demonstrates that adding direction information (heading, heading rate, transversal acceleration) to the more traditional only speed parameters (speed and longitudinal acceleration), improved remarkably the capability of the model to detect anomalies in cycling. Data filtering by using SGF played a positive role in reducing the FPs, although CNN showed good capability to handle noise and extract features from raw input data as we observed weighted average of scenario A2 of Table 3. The advantageous application of CNN was also proven by the best performance of proposed BeST-DAD over other traditional statistical techniques like PCA or heuristic threshold-based method applied to the cyclist braking rate. A case study showed the practical application and consistency of risk assessment and ranking.
5.1 Lessons learned and future needs
Despite the good performance of the CNN trained on the reference cyclist, we can expect larger deviance increasing the number of users. Because the CNN model depends on both the user and the specific road environment, transfer learning and cooperative learning [43] can be applied in real time to the model trained and transfer its knowledge to the specific user and road environment. To model the cycling behavior, our study used extended GPS NMEA contents (i.e. Speed, Heading) at high 10 Hz acquisition frequency which are not common in present smartphone and data mobility-data providers (e.g. Strava) following mainly the targets of profiling users' destinations and flows or fitting activity which need low frequency data acquisition (e.g. positions at 0.2 Hz). Therefore, frequencies up to 1 Hz are not yet available in standard smartphones whose development is moving mainly in the direction of improving the localization accuracy while already appropriate are speed and heading accuracy. To achieve the suitable higher frequency, an alternative to be evaluated is the capability of using SGF also for sub-sampling the GNSS signals at higher frequencies than the actual sampling rate [41].
Availability of data and materials
Not applicable.
References
Alsaleh, R., & Sayed, T. (2021). Markov-game modeling of cyclist-pedestrian interactions in shared spaces: A multi-agent adversarial inverse reinforcement learning approach. Transportation Research Part C: Emerging Technologies, 128(November 2020), 103191. https://doi.org/10.1016/j.trc.2021.103191
Arun, A., Haque, M. M., Bhaskar, A., Washington, S., & Sayed, T. (2021). A systematic mapping review of surrogate safety assessment using traffic conflict techniques. Accident Analysis and Prevention, 153(February), 106016. https://doi.org/10.1016/j.aap.2021.106016
Arvin, R., Khattak, A. J., & Qi, H. (2021). Safety critical event prediction through unified analysis of driver and vehicle volatilities: Application of deep learning methods. Accident Analysis and Prevention, 151(November 2020), 105949. https://doi.org/10.1016/j.aap.2020.105949
Babic, M., Ster, B., Povh, J., & Rodrigues, J. J. P. C. (2021). A new composite method of modeling bicycle traffic using convolutional neural networks and genetic programming. 2016 6th International Conference on Smart and Sustainable Technologies, SpliTech. https://doi.org/10.23919/SpliTech52315.2021.9566405
Belhadi, A., Djenouri, Y., Lin, J. C. W., & Cano, A. (2020). Trajectory outlier detection: algorithms, taxonomies, evaluation, and open challenges. ACM Transactions on Management Information Systems. https://doi.org/10.1145/3399631
Bichicchi, A., Belaroussi, R., Simone, A., Vignali, V., Lantieri, C., & Li, X. (2020). Analysis of road-user interaction by extraction of driver behavior features using deep learning. IEEE Access, 8, 19638–19645. https://doi.org/10.1109/ACCESS.2020.2965940
Borsos, A. (2021). Application of Bivariate Extreme Value models to describe the joint behavior of temporal and speed related surrogate measures of safety. Accident Analysis and Prevention, 159, 106274. https://doi.org/10.1016/j.aap.2021.106274
Cafiso, S., Di Graziano, A., Marchetta, V., & Pappalardo, G. (2022). Urban road pavements monitoring and assessment using bike and e-scooter as probe vehicles. Case Studies in Construction Materials, 16(January), e00889. https://doi.org/10.1016/j.cscm.2022.e00889
Cafiso, S., Pappalardo, G., & Stamatiadis, N. (2021). Observed risk and user perception of road infrastructure safety assessment for cycling mobility. In MDPI Infrastructures.
Candefjord, S., Sandsjö, L., Andersson, R., Carlborg, N., Szakal, A., Westlund, J., & Sjöqvist, B. A. (2014). Using smartphones to monitor cycling and automatically detect accidents-towards ecall functionality for cyclists. In Proceedings, International Cycling Safety Conference, November.
Cao, R., Chen, Y., Shen, M., Chen, J., Zhou, J., Wang, C., & Yang, W. (2018). A simple method to improve the quality of NDVI time-series data by integrating spatiotemporal information with the Savitzky-Golay filter. Remote Sensing of Environment, 217(August), 244–257. https://doi.org/10.1016/j.rse.2018.08.022
Chadha, G. S., Islam, I., Schwung, A., & Ding, S. X. (2021). Deep convolutional clustering-based time series anomaly detection. Sensors. https://doi.org/10.3390/s21165488
Chaloupka-Risser, C., & Füssl, E. (2017). The importance of communication between cyclists and other traffic participants and its potential in reducing traffic safety-critical events. Transactions on Transport Sciences, 8(1), 24–30. https://doi.org/10.5507/tots.2017.004
Detzer, S., Junghans, M., Kozempel, K., & Saul, H. (2014). Analysis of traffic safety for cyclists: The automatic detection of critical traffic situations for cyclists. WIT Transactions on the Built Environment, 138(70), 491–502. https://doi.org/10.2495/UT140411
Dong, W., Li, J., Yao, R., Li, C., Yuan, T., & Wang, L. (2016). Characterizing Driving Styles with Deep Learning. http://arxiv.org/abs/1607.03611
Dozza, M., & Fernandez, A. (2014). Understanding bicycle dynamics and cyclist behavior from naturalistic field data (November 2012). IEEE Transactions on Intelligent Transportation Systems, 15(1), 376–384. https://doi.org/10.1109/TITS.2013.2279687
Dozza, M., & Werneke, J. (2014). Introducing naturalistic cycling data: What factors influence bicyclists’ safety in the real world? Transportation Research Part F: Traffic Psychology and Behaviour, 24, 83–91. https://doi.org/10.1016/j.trf.2014.04.001
EU Commision Road Safety–Key figures. (2020). EU Commision. https://ec.europa.eu/transport/road_safety/index_en
DIRECTIVE (EU). (2019). 2019/1936 of the European Parliament and of the council of 23 October 2019 amending Directive 2008/96/EC on road infrastructure safety management https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32019L1936&from=LV
Evelien Polders. (2018). How to analyse accident causation? A handbook with focus on vulnerable road users (2nd ed.). Hasselt University Martelarenlaan 42, 3500 Hasselt, Belgium. https://www.intrasl.net/downloads/publicaciones/Handbook-InDeV.pdf
Feleke, R., Scholes, S., Wardlaw, M., & Mindell, J. S. (2018). Comparative fatality risk for different travel modes by age, sex, and deprivation. Journal of Transport and Health, 8(April 2017), 307–320. https://doi.org/10.1016/j.jth.2017.08.007
Ferenchak, N. N., & Marshall, W. E. (2019). Advancing healthy cities through safer cycling: An examination of shared lane markings. International Journal of Transportation Science and Technology, 8(2), 136–145. https://doi.org/10.1016/j.ijtst.2018.12.003
Ferri, C., Hernández-Orallo, J., & Modroiu, R. (2009). An experimental comparison of performance measures for classification. Pattern Recognition Letters, 30(1), 27–38. https://doi.org/10.1016/j.patrec.2008.08.010
Fonseca-Cabrera, A. S., Llopis-Castelló, D., Pérez-Zuriaga, A. M., Alonso-Troyano, C., & García, A. (2021). Micromobility users’ behaviour and perceived risk during meeting manoeuvres. International Journal of Environmental Research and Public Health. https://doi.org/10.3390/ijerph182312465
Formosa, N., Quddus, M., Ison, S., Abdel-Aty, M., & Yuan, J. (2020). Predicting real-time traffic conflicts using deep learning. Accident Analysis and Prevention. https://doi.org/10.1016/j.aap.2019.105429
Fuller, G., McGuinness, K., Waitt, G., Buchanan, I., & Lea, T. (2021). The reactivated bike: Self-reported cycling activity during the 2020 COVID-19 pandemic in Australia. Transportation Research Interdisciplinary Perspectives, 10(April), 100377. https://doi.org/10.1016/j.trip.2021.100377
Guo, X., Liu, X., Zhu, E., & Yin, J. (2017). Deep clustering with convolutional autoencoders. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10635, 373–382. https://doi.org/10.1007/978-3-319-70096-0_39
Hatcher, W. G., & Yu, W. (2018). A survey of deep learning: platforms, applications and emerging research trends. IEEE Access, 6, 24411–24432. https://doi.org/10.1109/ACCESS.2018.2830661
Hosseinpour, M., Madsen, T. K. O., Olesen, A. V., & Lahrmann, H. (2021). An in-depth analysis of self-reported cycling injuries in single and multiparty bicycle crashes in Denmark. Journal of Safety Research, 77, 114–124. https://doi.org/10.1016/j.jsr.2021.02.009
Huang, T., Wang, S., & Sharma, A. (2020). Highway crash detection and risk estimation using deep learning. Accident Analysis and Prevention, 135(December 2019), 105392. https://doi.org/10.1016/j.aap.2019.105392
Islam, Z., Abdel-Aty, M., Cai, Q., & Yuan, J. (2021). Crash data augmentation using variational autoencoder. Accident Analysis and Prevention, 151(July 2020), 105950. https://doi.org/10.1016/j.aap.2020.105950
Kordestani, H., & Zhang, C. (2020). Direct use of the savitzky–golay filter to develop an output-only trend line-based damage detection method. Sensors (Switzerland). https://doi.org/10.3390/s20071983
Kwayu, K. M., Kwigizile, V., Lee, K., Oh, J. S., & Nelson, T. (2022). Automatic topics extraction from crowdsourced cyclists near-miss and collision reports using text mining and Artificial Neural Networks. International Journal of Transportation Science and Technology. https://doi.org/10.1016/j.ijtst.2021.10.005
Li, P., Abdel-Aty, M., & Yuan, J. (2021). Using bus critical driving events as surrogate safety measures for pedestrian and bicycle crashes based on GPS trajectory data. Accident Analysis and Prevention, 150(November 2020), 105924. https://doi.org/10.1016/j.aap.2020.105924
Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2021). A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects. IEEE Transactions on Neural Networks and Learning Systems. https://doi.org/10.1109/tnnls.2021.3084827
Lore, K. G., Akintayo, A., & Sarkar, S. (2017). LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognition, 61, 650–662. https://doi.org/10.1016/j.patcog.2016.06.008
Lund University. (2015). Project InDeV: In-Depth understanding of accident causation for Vulnerable road users. https://portal.research.lu.se/en/projects/in-depth-understanding-of-accident-causation-for-vulnerable-road-/publications/
Madsen, T.; Andersen, C.; Kamaluddin, A. V., & András; Lahrmann, H. S. (2016). Review of current study methods for VRU safety. https://www.bast.de/EN/Traffic_Safety/Subjects/InDeV/Documents/pdf/2-1-4.pdf?__blob=publicationFile&v=1
Mammeri, A., Siddiqui, A. J., Zhao, Y., & Pekilis, B. (2020). Vulnerable road users detection based on convolutional neural networks. 2020 International Symposium on Networks, Computers and Communications, ISNCC, 2020. https://doi.org/10.1109/ISNCC49221.2020.9297332
Mehta, K., Mehran, B., & Hellinga, B. (2019). A methodology to estimate the number of unsafe vehicle-cyclist passing events on urban arterials. Accident Analysis and Prevention, 124(March 2018), 92–103. https://doi.org/10.1016/j.aap.2019.01.005
Murgano, E., Caponetto, R., Pappalardo, G., Cafiso, S. D., & Severino, A. (2021). A novel acceleration signal processing procedure for cycling safety assessment. Sensors, 21(12), 1–19. https://doi.org/10.3390/s21124183
Nikitas, A., Tsigdinos, S., Karolemeas, C., Kourmpa, E., & Bakogiannis, E. (2021). Cycling in the era of covid-19: Lessons learnt and best practice policy recommendations for a more bike-centric future. Sustainability (Switzerland). https://doi.org/10.3390/su13094620
Panigrahi, S., Nanda, A., & Swarnkar, T. (2021). A survey on transfer learning. Smart Innovation, Systems and Technologies, 194(10), 781–789. https://doi.org/10.1007/978-981-15-5971-6_83
Peng, Y., Liang, T., Hao, X., Chen, Y., Li, S., & Yi, Y. (2021). CNN-GRU-AM for shared bicycles demand forecasting. Computational Intelligence and Neuroscience. https://doi.org/10.1155/2021/5486328
PEP. (2018). Pan-European master plan for cycling promotion 3rd Draft. 2018(May), 1–28. https://thepep.unece.org/sites/default/files/2018-07/Informationnote 01 Draft THE PEP Pan-European Masterplan for Cycling Promotion.pdf
Pontus Wallgren, C. S. (2021). Cycling that makes sense-A qualitative exploration of cyclists ’ perception of bicycle infrastructure. In 9th International Cycling Safety Conference.
Qiao, S., Han, N., Huang, J., Yue, K., Mao, R., Shu, H., He, Q., & Wu, X. (2021). A dynamic convolutional neural network based shared-bike demand forecasting model. ACM Transactions on Intelligent Systems and Technology, 12(6), 1–24. https://doi.org/10.1145/3447988
Rasch, A., Moll, S., López, G., García, A., & Dozza, M. (2022). Drivers’ and cyclists’ safety perceptions in overtaking maneuvers. Transportation Research Part F: Traffic Psychology and Behaviour, 84(July 2021), 165–176. https://doi.org/10.1016/j.trf.2021.11.014
Jansen, Reinier J., & Tsippy Lotan, M. W. (2017). Interactions with vulnerable road users. European Commission Seventh Framework Programme FP7-SST-2012.4.1–3 GA No. 314050. https://doi.org/10.26323/UDRIVE
Sabour, S., Rao, S., & Ghaderi, M. (2021). DeepFlow: Abnormal Traffic Flow Detection Using Siamese Networks. In 2021 IEEE International Smart Cities Conference, ISC2 2021 https://doi.org/10.1109/ISC253183.2021.9562915
Salmon, P. M., Naughton, M., Hulme, A., & McLean, S. (2022). Bicycle crash contributory factors: A systematic review. Safety Science, 145(September 2021), 105511. https://doi.org/10.1016/j.ssci.2021.105511
Schleinitz, K., Petzoldt, T., Franke-Bartholdt, L., Krems, J. F., & Gehlert, T. (2015). Conflict partners and infrastructure use in safety critical events in cycling - Results from a naturalistic cycling study. Transportation Research Part F: Traffic Psychology and Behaviour, 31, 99–111. https://doi.org/10.1016/j.trf.2015.04.002
Shinar, D., Valero-Mora, P., van Strijp-Houtenbos, M., Haworth, N., Schramm, A., De Bruyne, G., Cavallo, V., Chliaoutakis, J., Dias, J., Ferraro, O. E., Fyhri, A., Sajatovic, A. H., Kuklane, K., Ledesma, R., Mascarell, O., Morandi, A., Muser, M., Otte, D., Papadakaki, M., … Tzamalouka, G. (2018). Under-reporting bicycle accidents to police in the COST TU1101 international survey: Cross-country comparisons and associated factors. Accident Analysis and Prevention, 110(October 2017), 177–186. https://doi.org/10.1016/j.aap.2017.09.018
Stamatiadis, N., Cafiso, S., & Pappalardo, G. (2019). A comparison of bicyclist attitudes in two urban areas in USA and Italy. Advances in Intelligent Systems and Computing, 879, 272–279. https://doi.org/10.1007/978-3-030-02305-8_33
Strauss, J., & Miranda-Moreno, L. F. (2017). Speed, travel time and delay for intersections and road segments in the Montreal network using cyclist Smartphone GPS data. Transportation Research Part D: Transport and Environment, 57(September), 155–171. https://doi.org/10.1016/j.trd.2017.09.001
Strauss, J., Zangenehpour, S., Miranda-Moreno, L. F., & Saunier, N. (2017). Cyclist deceleration rate as surrogate safety measure in Montreal using smartphone GPS data. Accident Analysis and Prevention, 99, 287–296. https://doi.org/10.1016/j.aap.2016.11.019
Tarko, A., Davis, G., Saunier, N., Sayed, T., & Washington, S. (2009). White paper surrogate measures of safety ANB20(3) subcommitee on surrogate measures of safety. Safety Data Evaluation and Analysis–ANB20, 20(3), 1–13.
Vandenbulcke, G., Panis, L. I., & Thomas, I. (2017). On the location of reported and unreported cycling accidents: A spatial network analysis for Brussels. CyberGeo. https://doi.org/10.4000/cybergeo.28073
Wang, Q., Gan, S., Chen, W., Li, Q., & Nie, B. (2021). A data-driven, kinematic feature-based, near real-time algorithm for injury severity prediction of vehicle occupants. Accident Analysis and Prevention, 156(March), 106149. https://doi.org/10.1016/j.aap.2021.106149
Werneke, J., Dozza, M., & Karlsson, M. (2015). Safety-critical events in everyday cycling - Interviews with bicyclists and video annotation of safety-critical events in a naturalistic cycling study. Transportation Research Part F: Traffic Psychology and Behaviour, 35, 199–212. https://doi.org/10.1016/j.trf.2015.10.004
Yin, C., Zhang, S., Wang, J., & Xiong, N. N. (2020). Anomaly detection based on convolutional recurrent autoencoder for IoT time series. IEEE Transactions on Systems, Man, and Cybernetics: Systems. https://doi.org/10.1109/tsmc.2020.2968516
Zhang, H., Luo, Y., Yu, Q., Sun, L., Li, X., & Sun, Z. (2020). A framework of abnormal behavior detection and classification based on big trajectory data for mobile networks. Security and Communication Networks, 2020. https://doi.org/10.1155/2020/8858444
Zhou, H., Li, Z., Liu, C., Xu, J., Li, S., & Zhou, K. (2020). Assessment of the performance of carrier-phase and Doppler smoothing code for low-cost GNSS receiver positioning. Results in Physics, 19(February), 103574. https://doi.org/10.1016/j.rinp.2020.103574
Acknowledgements
Not applicable
Funding
This research is supported by the funding of “SAFE DEMON SAFE Driving by E-health MONitoring” CUP G29J18000720007, PON FESR 2014/2020—Action 1.1.5, Sicilian Region, Italy.
Author information
Authors and Affiliations
Contributions
All the authors have contributed to the work described, read and approved the contents for publication in this journal.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Yaqoob, S., Cafiso, S., Morabito, G. et al. Detection of anomalies in cycling behavior with convolutional neural network and deep learning. Eur. Transp. Res. Rev. 15, 9 (2023). https://doi.org/10.1186/s12544-023-00583-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s12544-023-00583-4