Driving simulator-based training to improve self-rating ability of driving performance in older adults – a pilot study

Abstract

Objective

The aim was to investigate the potential of using simulator-based training (SBT) to improve older drivers' self-rating ability and to compare two forms of feedback; corrective versus corrective and rewarding feedback.

Method

The study was designed to study the possibility of training for self-rated driving ability in a simulator, and the impact of corrective (errors made) feedback versus corrective (errors made) and rewarding (correct behaviour) feedback during training. In total, 21 older drivers (mean age 78.5, SD=3.9 years) were trained and assessed in the driving simulator. Driving performance was assessed by penalty scores as well as self and expert ratings.

Results

The average deviation from correctly rated ability (own vs. expert) changed from -0.7 (under-rating) to 0.1 at the final training and assessment occasion; i.e., drivers ratings became more like the expert's rating or, in other terms, better calibrated. The individuals with the largest deviations from the expert's rating initially improved their self-rating ability the most. There were no differences between the two feedback groups in terms of their ability to self-rate, but rewarding feedback had a positive effect on penalty scores. The SBT showed positive training effects on the ability to self-rate one's driving ability, and rewarding feedback contributed to lower penalty scores. However, simulator sickness was a shortcoming that needs to be adressed, and the optimal form of feedback should be further investigated.

1 Introduction

Maintaining safe and independent transportation mobility is important for all persons, and even more so for older adults. Older drivers may choose to restrict their driving due to a lack of confidence or lack of alternative transportation options, but for many older adults, driving is seen as the primary mode of transportation [1, 2]. However, declining abilities due to normal ageing are not always compensated by the cars, driving environments, or driving behaviours [3,4,5].

When controlling for mileage, older drivers are not in general overrepresented in accident statistics, according to previous research [6]. However, as a part of the total population of Sweden, older drivers are over-represented among fatal injuries on the road. During 2014–2016, older drivers represented 32% of fatalities (STRADA data), while they represented 20% of the population [7]. Several factors can explain this over-representation such as the lower level of violence of the collision needed to injure or kill an older driver than younger people. Most older drivers tend to self-regulate their driving correctly; i.e., limiting their exposure to challenging situations such as avoiding driving at night, in the rain, or during rush hour [8, 9]. However, not all older drivers are able to self-regulate their driving behaviour [10]. Self-regulation alone is therefore not sufficient to ensure safe driving among older drivers.

Awareness and insight into one’s own driving abilities can determine the choices one makes [11]. The interactions between self-monitoring, beliefs about driving, and factors supporting driving abilities enable and influence safe driving behaviour [11]. However, it is unknown if older drivers have insight into their own driving abilities or whether external factors and changes in lifestyles result in self-regulation [4, 5]. It is often said that older drivers adjust their driving behaviour and habits to their age-related changes [5]. The reason may be that they restrict their driving due to awareness of specific impairments such as physical or visual abilities [12, 13]. However, not all drivers are capable of adjusting their driving due to declines in cognitive abilities, medications taken, and the aging process itself [11, 13, 14]. The key question is to balance task demand and capability; i.e., calibration [15].

Results from previous studies of self-regulation vary as to what extent older drivers restrict their driving or what type of self-regulation they apply [4, 16, 17]. Most studies have simply asked older drivers if they modify their driving. Drivers with a lack of insight are less likely to self-regulate and may overestimate their driving ability [16, 18]. It is a challenge to estimate safe driving correctly, both objectively and by the drivers themselves. Some drivers over-rate their driving ability, while others under-rate it, and some drivers make correct estimations [19]. People who over-rate their ability are more likely to be risky drivers, not only for themselves but also for others using the road [20]. These drivers might turn up in crash statistics, but usually there is no one single cause of an accident. Those who under-rate their ability are probably more reluctant to drive and might be more difficult to identify unless one is searching for drivers who have given up driving. Several attempts have been made to find a realistic way to classify drivers in to correct, under-, and over-raters of their own ability [21, 22]. However, a general principle has been to compare a person’s self-assessment with some more or less objective measure of ability [19, 22].

As crashes and near crashes are relatively rare, we usually have to rely on driving behaviour data such as time-based safety margins like Time-To-Collision (TTC) as well as expert judgement based on observations of behaviour [21]. Educational interventions, such as updated knowledge on traffic laws or other strategies such as transmission type, may help to ensure the competency of older drivers [14]. Another alternative strategy is to encourage and raise awareness among older drivers to self-rate their driving skills and compare this rating with that of an independent assessor, and thereby be able to modify and adjust one’s driving appropriately [20, 23]. Thus, as some older drivers tend to overate their own driving abilities [20, 21], there is a need to investigate drivers’ insight and to be able to utilize more objective measures. The purpose of the present study was to investigate the potential of using simulator-based training (SBT) to improve older drivers’ self-rated driving ability. The hypothesis was that SBT could contribute to improving people’s self-rated performance. Moreover, providing feedback, both corrective (errors made) and rewarding (correct behaviour) feedback, may result in driving modifications.

2 Method

The study was designed to study the possibility of training for self-rated driving ability in a simulator, and the impact of corrective feedback versus corrective and rewarding feedback during training.

2.1 Participants

The first 27 volunteers were recruited from a previous on-road study. They were found through a local senior’s organization and the Swedish driving license register [24]. Another 9 were recruited with help from the other participants; i.e., husband/wife or friends. A total 36 drivers fulfilled the inclusion criteria; i.e., over 70 years old and an active driver. However, one participant terminated due to an acute illness and another 14 could not complete the training due to simulator sickness. Hence, 21 (58%) participants were included in the final analysis (7 women and 14 men). The mean age of the group was 78.5 (standard deviation [SD] =3.9, range, 72–88) years. No medical or other risks related to participation in the study were identified. All participants were informed before signing an informed consent. The participants were told that they could stop whenever they wanted without giving any justification. All data forms were coded, and no individual could be identified by the researchers. All procedures were in accordance with the ethical standards of the Declaration of Helsinki.

2.2 Procedure

Data were collected in a driving assessment unit in Gothenburg, Sweden. The test leader was a driving assessor (specially trained occupational therapist). The participants came three times, and each simulator training session lasted approximately 1 h. The participants were randomly allocated to two experimental conditions based on different kinds of feedback. One group was given feedback on incorrect driving behaviour or errors made (corrective feedback). The other group also learned about their incorrect behaviour but also received rewarding feedback (corrective and rewarding feedback).

Initially, participants were acquainted with the simulator by driving a 5-min route (country road). The drive was also used to detect any signs of simulator sickness. Thereafter, the actual data collection started with a “pre-training assessment drive”, which was followed by three sessions (Fig. 1). During each training session, the difficulty for each of the six lessons was increased. The third session included a final training and “post-training assessment drive” (identical to the initial pre-training baseline drive, from the first occasion). After each lesson, the participants were asked “How well do you think you performed on the driving task?” to self-rate their driving performance.

Fig. 1

Flowchart of the simulator training

2.3 The simulator program

The simulator hardware was built by components (steering wheel, pedals, dashboard, etc.) from a Ford Focus, three 40-in. LCD screens, and a surround sound system (Fig. 2). A longer drive (12 min), with a mixed traffic environment including more or less critical situations, was used for pre- and post-assessment drives. The program included 6 different training lessons and each of the lessons had three versions with increasing difficulties:

• 1. Traffic light in an urban environment

• 2. Pedestrian crossings in the city

• 3. Two consecutive left turns (city) with oncoming traffic (increasing gap)

• 4. Entering a motorway with traffic

• 5. Passing another car on a motorway

• 6. Navigation

Fig. 2

The simulator used in the study (left) and pedestrian crossing in city scenario (right)

2.4 Data collection

Three types of data were collected; objective data on driving performance, participant self-rating, and expert rating. The objective data on driving performance were operationalised by using penalty scores representing errors made; e.g., regarding speed, vehicle position, harsh braking, etc. Lower penalty scores indicate better driving performance. Participants’ self-rating regarding their driving performance was operationalised by asking them to answer the following question after each training lesson “How well do you think you performed on the driving task?”. The question was displayed on the central screen. The participant used a rating scale from 1 (very bad) to 5 (very well). Experts’ rating on the participants driving performance was also collected by answering the same question before the participants gave their answer. Thereafter, the participant was given the feedback on the central screen. Furthermore, if the participants explained in words, qualitatively, the reason for a specific score, the test leader noted the comments.

2.5 Statistical analyses

Statistical analyses (analysis of variance [ANOVA]) were done with SPSS® (version 22.0). A p value of < 0.05 was considered statistically significant. ANOVA was used for all analyses between the two groups of feedback.

3 Results

The results were based on the two feedback-groups; 11 participants (7 men and 4 women) received corrective feedback, and 10 participants (7 men and 3 women) received corrective and rewarding feedback.

3.1 Training effect

The results are presented for all three types of data. First, concerning the penalty scores, a small deterioration in driving performance was observed after the post-training assessment drive (Table 1). The mean penalty scores for the pre-training assessment drive was 22.81 and 25.35 for the post-training assessment drive; however, the difference in scores was not significant (p = 0.60). Concerning the self-rating, participants gave themselves a better rating after the post-training (3.43 vs. 3.95); however, the difference in scores was not significant (p = 0.77). The expert ratings were similar after pre-training and post-training (p = 0.42).

Table 1 Results (mean and standard deviation) of the pre-training assessment and post-training assessment drive; penalty scores, expert’s rating, and self-rated performances

3.2 Feedback effect

There were no significant differences between the two feedback groups concerning penalty score (p = 0.16), expert-rated (p = 0.69), or self-rated (p = 0.87) performance (Table 1). However, a mixed model ANOVA revealed a significant interaction (F [1, 18] =5.00, p < 0.05) showing that the group who received only corrective feedback increased their penalty scores, while the group that also received rewarding feedback deceased their penalty scores.

3.3 Self-rating ability and calibration

During the first training session (i.e., session 1; level 1; lessons 1–6, Fig. 1), the mean differences in scores between self- and expert ratings was − 0.7. During the third training session, the difference was 0.1. Thus, from being under-raters after the first session, the total group was better able to calibrate themselves. This improvement in ability to self-rate was significant (F [1, 19]=36.92,p < 0.01) for the total group. However, Fig. 3 shows individual differences. Differences in scores closer to zero (0) indicate that the expert’s and participant’s own ratings agreed. Thus, negative differences between the expert and participant indicate under-rating and positive differences indicate over-rating.

Fig. 3

Mean differences (score) between self- and expert ratings on the first and third training sessions (n = 21)

3.4 Participants’ self-perceived experience about scoring

As previously mentioned, notes were taken to capture the participant’s thoughts about their scores; i.e., self-rating. Some were particularly prominent for some of our participants and illustrate different kind of estimators, such as an over-rater. For one participant (participant number 10, Fig. 3), the difference between the self- and expert rating was − 0.7, but had changed to 0 after the third session (i.e., no difference between participant and expert). A quote by this participant was: “I probably drove too fast, but otherwise it went ok. I think I followed all the rules”. This participant changed from under-rating to correct-rating.

One participant (participant number 17, Fig. 3) had a mean self-assessment score of 3.8 after the first session, but the expert had rated the performance as 4.7 (mean); i.e., the difference was − 0.9 (i.e., under-rating). However, after the third session the differences had changed to (+) 1.0 (i.e., over-rating). A quote from this person was: “I was a bit too close to the cyclist, but in reality it is not like that…I crossed the lane line, but it was the cyclist’s fault”. This participant changed from under- to over-rating.

For another participant (participant number 18, Fig. 3), the mean self-rating score from the first session was 3.3 versus the expert-rating of 4.7, resulting in a rather large difference of − 1.4 (i.e., under-rating). After the third session, there was only a smaller change; the difference was then − 1.0 (i.e., under-rating). An example of a quote from this participant: “I saw the yellow car behind me, but when I began to pick up speed, I began to wobble – it was bad! Poor timing when I drove on the highway... I don’t know if I had chosen another speed in real life”. This participant remained an under-rater.

4 Discussion

In this study, the objective was to investigate the potential of using simulator-based training to improve older adults’ self-rated driving ability. The drivers’ average deviation from the expert assessment decreased after training, with those participants who deviated the most initially improving substantially. All drivers with a difference more than − 0.7 showed positive changes (i.e. their ratings came closer to being “correct” in their estimation). This shows the potential of a simulator training programme in helping people to better calibrate their driving abilities.

At the first training session, all participants underestimated their performance. This may be due to their lack of acquaintance with simulator driving, since none of them had used a simulator and older drivers seldom play videogames. As a group, they changed their performance from under-rating to over-rating, but their self-rating became closer to the expert’s rating. This may suggest that this form of training may be more relevant for persons who under-rate their abilities than for those who over-rate their abilities (i.e., the training made them perceive themselves in a more positive and correct way).

Providing feedback is a concept that has been used in previous studies to improve performance and provide insight of one’s own behaviour [25]. A simulator setting offers an opportunity to let drivers experience situations where both overrating and underrating might manifest itself in different behaviours; e.g., making a left turn at an intersection with either too short of a time gap or being too hesitant. Feedback can be an effective means to both improve performance and provide insight into one’s own behaviour. Thus, to help the drivers to a better self-rating, we used both corrective and rewarding feedback during the training. Only the group with rewarding feedback reduced their penalty points despite an escalating degree of difficulties. They did not, however, asses their performance better than the other group.

Rewarding feedback may be more relevant to under-raters but negative to over-raters. The results suggest that training, rather than rewarding feedback, does more to improve accurate self-rating. In order to change under- or over-raters’ assessments, attitudes towards training must change, suggesting that training gives one self-confidence and a better self-rating. Rewarding feedback, on the other hand, gives them a better performance that may convince them that they will drive safer after the training. Many women stop driving too early, possibly due to degraded confidence [2]. Training in a simulator might help them improve their self-rating and, with rewarding feedback, also attain better performance. How to address the person who over-rates themselves might be better achieved by other type of feedback. This needs to be further investigated [26].

Driving is a cognitively demanding task [13, 27] and some older drivers have insight into their abilities and might compensate and change their driving behaviour. However, not all have insight into their declining skills and are not able to adapt their driving behaviour [11]. Even if the deviant self-assessment could be seen as a personal trait that persists into growing older, we might see other causes for an incorrect self-assessment, such as lack of understanding age-related functional declines. Ageism is also a factor that can promote a driving style that is subjectively viewed as very competent (e.g. to choose a higher speed) [28]. General (prejudicial) attitudes in society of older drivers as risky drivers might also contribute to an unrealistic degraded self-assessment. We can see ourselves as both higher and lower performers compared to what we actually are. Defensive driving, often promoted for older drivers, can have side effects such as unrealistic beliefs of being able to master critical situations or loss of self-confidence. Those drivers who do lose confidence in difficult driving situations tend to avoid those situations when possible [16].

SBT has previously focused on technology, but much remains on educational and didactic issues [29]. Questions such as: “How should the training be designed?”, “How much and for how long?”, “Should feedback be used and how?”, “Direct augmented feedback?”, or “Can several sense modalities enforce impact?”. In summary, SBT can be used to improve older drivers’ ability to assess their own performance, and feedback plays an important role, but should be further investigated. A limitation with the current study is that it did not include a control group. Thus, we cannot rule out the influence of extraneous factors as the actual cause of the observed change. However, as the study was a before/after study the participants serves as their own controls. Furthermore, the main purpose was to investigate improvements in self-assessment not the actual driving performance per se and the drivers did not get any feedback on their assessment, i.e., how they compared to expert assessment only on their performance. The difficulty of the lessons increased for each session and thus, assessment could be more difficult. In summary, a change from being less confident to over confident and specifically closer to the expert assessment could be seen as a real improvement even if it needs to be further investigated by the inclusion of a control group. Moreover, simulator sickness, as observed in this study, might hamper the use of simulators, specifically with older drivers, and further work is needed to decrease this problem. A high proportion of our participants could not continue due to simulator sickness, although a driving simulator appears to be a safe and cost-effective method for assessing driving performance. Several studies have reported similar results and also showed a significant difference between older and younger adults [30]. A problem with the simulator, which may increase simulator sickness, was that many of the participants experienced difficulties with manoeuvring (i.e., the steering wheel) and thought it was too “sensitive”. This is a limitation and needs to be examined and adjusted. Another important observation made by the test leader was the participants speed adaptation during the first training. They were all inexperienced with simulator driving, which made them drive very slowly, and might have caused fewer driving errors.

5 Conclusions

Being able to rate your own performance can be vital for maintaining the safe mobility of older drivers. Incorrect self-rating can lead to an increased risk of being involved in a crash or unjustifiable restriction in mobility. SBT showed positive training effects on the ability to self-rate one’s driving ability, and rewarding feedback contributed to less penalty scores. However, simulator sickness is a shortcoming that needs to be addressed, and the optimal form of feedback should be further investigated.