- Original Paper
- Open Access
Cognitive workload and visual behavior in elderly drivers with hearing loss
© The Author(s) 2014
- Received: 13 January 2014
- Accepted: 16 May 2014
- Published: 29 May 2014
To examine eye tracking data and compare visual behavior in individuals with normal hearing (NH) and with moderate hearing loss (HL) during two types of driving conditions: normal driving and driving while performing a secondary task.
24 participants with HL and 24 with NH were exposed to normal driving and to driving with a secondary task (observation and recall of 4 visually displayed letters). Eye movement behavior was assessed during normal driving by the following performance indicators: number of glances away from the road; mean duration of glances away from the road; maximum duration of glances away from the road; and percentage of time looking at the road. During driving with the secondary task, eye movement data were assessed in terms of number of glances to the secondary task display, mean duration of glances to the secondary task display, and maximum duration of glances to the secondary task display. The secondary task performance was assessed as well, counting the number of correct letters, the number of skipped letters, and the number of correct letters ignoring order.
While driving with the secondary task, drivers with HL looked twice as often in the rear-view mirror than during normal driving and twice as often as drivers with NH regardless of condition. During secondary task, the HL group looked away from the road more frequently but for shorter durations than the NH group. Drivers with HL had fewer correct letters and more skipped letters than drivers with NH.
Differences in visual behavior between drivers with NH and with HL are bound to the driving condition. Driving with a secondary task, drivers with HL spend as much time looking away from the road as drivers with NH, however with more frequent and shorter glances away. Secondary task performance is lower for the HL group, suggesting this group is less willing to perform this task. The results also indicate that drivers with HL use fewer but more focused glances away than drivers with NH, they also perform a visual scan of the surrounding traffic environment before looking away towards the secondary task display.
- Hearing loss
- Driving simulator
- Visual behavior
- Cognitive workload
Drivers with moderate to severe hearing loss (HL) show worse driving performance in the presence of distracters than drivers with normal hearing (NH) or mild HL . This may be a growing traffic safety concern since HL is often age-related and the aging population implies an increased number of road users with HL. At the same time the in-vehicle environment is becoming increasingly more complicated, for example with navigation systems and communication devices.
The prevalence of HL in Europe is roughly 30 % for men and 20 % for women at the age of 70 years, and 55 % for men and 45 % for women at the age of 80 years , and similar percentages have been reported in other countries outside Europe . This number is increasing, due both to prolonged life expectancy and to increased exposure to noise in the environment . The prevalence of HL increases for all ages, although the most common category of HL is age-related presbycusis .
Relatively few studies have been carried out to investigate the effect of HL on traffic safety. Some researchers have suggested that HL is associated with a higher risk of accidents [5–7],(Barreto, Swerdlow et al. 1997; Ivers, Mitchell et al. 1999; Picard, Girard et al. 2008) while others have found no such relation [8–10] – though McCloskey et al.  do find that hearing aid users are more at risk of accidents.
Glad  suggested that hearing impaired drivers could compensate for their sensory loss by adding extra mirrors to the vehicle to offset their difficulty in detecting fast vehicles approaching from behind. Whether this suggestion is correct or not, Magnet  indicated that when compared with drivers with NH, drivers with HL do not use compensatory eye movements. Thorslund et al.  demonstrated that increased driving complexity affects drivers with HL more than those with NH, and that as driving complexity increases, drivers with HL show more cautious driving behavior, with lower driving speeds and higher disregard of any secondary tasks. The purpose of this study is to investigate whether the visual behavior of drivers with HL differs from drivers with NH, both during normal driving and while performing a distracting secondary task.
We have found no literature regarding whether individuals with HL compensate visually for the loss of auditory information; however, some studies show that deaf people perform better on certain perceptual tasks than those with NH . The enhancement of attention resources directed towards the periphery might serve as a compensatory means for detecting auditory events outside the central field of view ; however, how this relates to traffic is unclear.
Andersson  demonstrated that specific aspects of the phonological system deteriorate as a function of poor auditory stimulation in individuals with HL. Specifically, phonological representations decline and this decline also affect the ability to perform rapid phonological operations, such as recognizing and comparing letters . Thus, it is reasonable to assume that a secondary task that includes performing phonological operations while driving would affect drivers with HL more than drivers with NH.
With previous assumptions that driving is mostly a visual task , that deaf people have enhanced peripheral vision , and that drivers with HL show more cautious driving behavior , it is interesting that drivers with HL show worse driving performance during distraction than drivers with NH . The purpose of the present study is to compare visual behavior in older adults with and without HL during normal driving and driving while performing a secondary task.
Mean values of PTA4 (means of 500, 1,000, 2,000 and 4,000 Hz) in best and worst ear for men and women respectively
PTA4 Best ear (dB)
PTA4 Worst ear (dB)
2.2 Simulator scenario and procedure
The experiment was conducted in an advanced moving-base driving simulator (Sim III) at the Swedish Road and Transport Research Institute. The car body consists of the front half of a SAAB 9-3 and the visual display subtended 120° × 30° (horizontal and vertical) from the participant’s position in the simulator. By moving, rotating, or tilting the car and the video screens, acceleration and deceleration forces in either direction can be simulated. A vibration table enables a high-fidelity simulation of road surface contact, making the driving experience very realistic.
There was a short (5 min) training session in the simulator in order to get the participants familiar with the driving situation and the visual distraction task. The driving scenario was a 35 km long rural road with a speed limit of 70 km/h. The traffic density was low with no traffic in the same lane and random oncoming traffic at a mean interval of 22 s.
The secondary task was initiated every 30 s. Drivers were prompted by a vibration in the seat to first look at and then read back a complete sequence of four letters appearing on a display, one letter at a time. The level of cognitive workload was randomly varied by altering the phonological similarity of the letters according to Conrad and Hull . Thus, two sequences consisting of randomized letters that were either phonologically alike (e.g. BDPT) or not phonologically alike (e.g. RKNJ) were used. The reason for choosing this particular task was that drivers with HL are likely to be more affected by workload added to the phonological loop . his is an adaptation of Sternberg’s scanning paradigm , in which a set of 1 to 6 digits were presented sequentially to the subject at the rate of one every 1.2 s. The display time had to be long enough for recognition of the letters, but short enough for the participants to need to keep their eyes on the display. The instruction was to look at all four letters and then repeat the whole sequence in the correct order afterwards.
The experiment also included a number of critical events in which an offset in the steering wheel angle was introduced to gently push the car towards an oncoming vehicle while the driver was looking away. There was also a parked car event in which the drivers had to brake. Analysis of the data surrounding these events as well as driving performance is presented in a separate paper  and thus excluded here.
2.3 Eye movements
Eye movements were acquired with a remote four-camera eye tracking system (Smart Eye Pro 5.7, Smart Eye AB, Gothenburg, Sweden), which measured the driver’s gaze direction in full 3D at a rate of 50 Hz. The eye tracker is connected to a model of the vehicle, and thus allows analyses of the objects in the cockpit attracting the driver’s gaze.
The eye tracking system reflects the contrast between the edge of the iris and the sclera with a confidence measure normalized to the range 0–1, on which 0 corresponds to the 1st percentile of all collected quality values of the current trip, and 1 corresponds to the 99th percentile. Extreme values outside the range are set to 0 or 1 depending upon whether they are low or high. All samples with gaze confidence below 0.2 were set to missing values in order to remove unreliable data. In the current dataset, 2.8 % of the data were set to missing values due to low gaze confidence values. A median filter of 400 ms was also applied to the 3D gaze direction signals in order to remove noise while preserving the inherent structure of fixations and saccades. The filter was implemented to facilitate interpolation of short segments of missing data by ignoring missing values in the sliding median calculation . Consecutive missing value sequences longer than 200 ms were left unchanged.
Objects in the vehicle used in the analyses
Right window, right mirror, and right door
Left window, left mirror, and left door
4. Rear-view mirror
The instrument cluster
6. Secondary task display
Center console and task display
Other objects such as roof, floor, and glove compartment
A glance is defined as a sequence of gaze direction data lasting from the moment at which the direction of gaze moves towards a particular target to the moment it moves away from the target [28, 29]. Here the filtered zone data is used to define glances; as long as the eyes are residing within a certain zone the gaze data belongs to the same glance.
2.4 Performance indicators
Eye movement behavior was assessed during normal driving using the following performance indicators: number of glances away from road, mean duration away from road, max duration away from road, and percentage road center. The definition of “percentage road center” was gaze data residing in a circle with 8° radius, centered on the main mode of the gaze data, according to Victor et al. .
During the secondary task, eye movement data were assessed in terms of number of glances to the secondary task display, mean duration of glances to the secondary task display, and max duration of glances to the secondary task display. The secondary task performance was assessed as well, counting the number of correct letters in order, the number of skipped letters, and the number of correct letters ignoring order. The last performance indicator was included to see whether the participants remembered the letters themselves but not the presentation order.
A new secondary task was given every 30 s. The first 15 s were used partly to display the secondary task letters and partly to prepare the simulator for an occasional critical event (not used in this study). Of these 15 s, eye movement data from the 2.8 s that the letters were displayed were used in the analysis. In addition, the normal driving data set was governed by the remaining 15 s leading up to the next task.
2.5 Design and expectations
A mixed design with the fixed factors of hearing status (NH vs. HL), gender (men vs. women), and condition (normal vs. task); participant (participants 1–48) was nested within genders × hearing status as a random factor. Main effects and interaction effects of hearing status, gender (between-group variables), and condition (within-group variable) were examined for all performance indicators and measures.
First, as a natural effect of divided attention, we expected to see in both groups an effect on all areas during the task, an increased number of glances at the secondary task display, and a decreased number of glances towards the other zones.
Second, with the knowledge of their more cautious driving behavior , we assumed that participants with HL would be generally less willing to take their eyes off the road, in terms of both number and duration of glances. We expected fewer and shorter glances away from the forward roadway in the HL group than in the NH group.
Third, with the indications of compensatory strategies and a cautious driving behavior , we assumed that participants with HL would be less willing to take their eyes off the road while performing the secondary task. We expected that secondary task performance would be lower, frequency of glances to the secondary task display would be higher, and duration of glances to the secondary task display would be shorter in the HL group than in the NH group.
Our strategy for analyzing the distribution of glances was to start with a model as comprehensive as possible, with several variables, interactions, and multidimensional responses. Significant results would lead to the examination of each zone in which they were found and significant interactions would lead to the analysis of each level of the factors involved in the interaction. Factors were tested with Wilk’s MANOVA and with ANOVAs using F-tests.
A MANOVA was performed to examine whether condition, hearing status, gender, or any two-factor interactions of these had an effect on the distribution of glances, where the distribution is governed by a vector representing the 7 target gaze zones. In this model hearing, gender, and condition were included as fix variables, and participant nested within hearing and gender was included as a random variable. The significant interaction effect of condition and hearing lead to the analysis of each condition and each hearing status.
ANOVAs were performed to test hypotheses examining one zone at a time. All effects reported as significant have p < .05 and are presented with an F -value and least squares means.
The results from the MANOVAS of distribution of glances during normal driving and task, respectively, are presented first. These are followed by ANOVAS of fixations in target zones and eye movement behavior. Finally the results of the secondary task performance are presented.
3.1 Distribution of glances
Transition matrices showing the jump probability between different zones are illustrated in Fig. 3. Similar to the 2D histograms, the transition matrices are also very similar between the NH and HL groups during normal driving. As expected, most fixations on peripheral targets are followed by a fixation on the windshield. The pronounced modes in the 2D histograms for the HL group are emphasized in the transition matrix as very few fixations to zone 7 (other).
3.2 Fixations in target zones
The effect of condition per target zone for NH and HL respectively
Least squares means
Secondary task display
The interaction between condition and gender was more apparent in the HL group; however, since the effect of gender was not significant, this was not examined further.
The main result from the analysis of target zones is that during the task, people with HL looked twice as often in the rear-view mirror as they do during normal driving and twice as often as drivers with NH regardless of the driving condition.
3.3 Eye movement behavior
The full model ANOVA showed a significant main effect of condition, p < 0.05, on all 4 eye movement behavior measures and a significant interaction effect of hearing status + condition, p < 0.05, in 3 out of 4. Thus, to follow the analysis strategy one ANOVA was carried out to examine the effect of hearing status + gender per condition, and another was carried out to examine the effect of condition + gender per hearing status.
On number of glances away from road there was a significant interaction effect of condition + hearing status, such that during the task the HL group looked away from road more often than those with NH, while during normal driving there was no between-group difference. Also on mean duration away from road a significant interaction effect emerged with condition + hearing status, such that during normal driving there was no between-group difference and during the task glances away from road were shorter for the HL group than the NH group. There was a significant interaction effect of condition + hearing status on max duration away from road, with the NH group unaffected by condition and those with HL having shorter max duration during task.
The interaction between condition + hearing status on eye movement behavior
Least squares means
Number away from road
Mean away from road
Max away from road
Percent road center
To summarize, the effects of hearing status on eye movement behavior were connected to the condition of task. During task, the HL group looked away from the road more frequently and for shorter durations than the NH group.
3.4 Secondary task performance
Secondary task performance revealed a significant main effect for hearing status, such that participants with NH had a higher percentage of correct answers, F(1, 44) = 3.52, p = 0.05. There was no significant main effect of gender and no interaction effect of gender + hearing status on percentage correct.
A significant main effect of hearing status emerged for the percentage of skipped letters, such that participants with HL skipped more letters than those with NH did, F(1, 44) = 6.10, p = 0.02. A significant main effect also emerged for gender, such that women skipped more letters than men did, F(1, 44) = 4.60, p = 0.04. There was no significant interaction effect between gender and hearing status.
Ignoring the ordering of the correctly repeated letters, a significant main effect emerged for hearing status, such that participants with NH performed better than participants with HL, F(1, 44) = 9.91, p < 0.01. There was no significant effect of gender and neither was there a significant interaction effect between gender and hearing status.
In summary, those with HL identified fewer correct letters and skipped more letters than those with NL.
The purpose of this study was to compare visual behavior in drivers with and without HL. The question was whether the HL group would use more active visual scanning than the NH group, and more frequent glances of shorter duration were expected in the HL group. This was not entirely confirmed by the statistical analysis; however, according to the histogram in Fig. 1, the HL group had a larger number of fixations than the NL group on the speedometer and on gaze targets to the left (left window, left mirror, and left door) in normal driving. The analysis of target zones showed that during task, HL looked twice as often in the rear-view mirror than during normal driving and twice as often as NH drivers in either condition.
As expected, as a natural effect of divided attention during task, the initial MANOVA confirmed an increased number of glances at the secondary task display and a decreased number of glances in the other zones for both groups.
We assumed that participants with HL would generally be less willing to take their eyes off the road, and this would be reflected in their looking away less often and for shorter durations than those in the NL group. This was partly confirmed by the analysis, and bound to the driving condition exceeding normal driving (as in performing the task), which is in line with previous results .
We also assumed that participants with HL would be less willing to take their eyes off the road while performing the secondary task, which was confirmed by the analyses of eye movement behavior and secondary task performance.
The HL group also tended to look more in the center rear-view mirror and further to the right during the secondary task, indicating scanning along the right side of the road (see Fig. 2). The transition matrix in Fig. 3 shows that in the HL group, shifts in gaze towards the secondary task display originated from all other gaze targets. Glances to the secondary task in the HL group were preceded especially more often than in the NH group by glances from the center and right rear-view mirrors. This suggests more cautious driving behavior by those with HL, who tended more often than drivers with NH to check in all directions before looking away from the road.
The advantages of performing simulator studies, including the opportunity to offer a safe convenient alternative to measuring driving performance on-road and the ability to keep driving conditions and environmental conditions constant, also come with limitations [31, 32]. For example, motion, velocity, and acceleration ranges are limited; it is impossible to fully represent a real traffic environment; and simulator sickness . Also, the simulated world does not contain the same level of detail and roughness as the real world. Since there is a limited amount of details present in the simulated world, there is also less to focus on. It is not known to which extent this limitation affects visual behavior. Simulator performance shows medium to strong correlations with many on-road driving performance measures as well as to cognitive and physiological measures, however simulators need to be validated for each new setting . The percentage of fixations on the speedometer was high in both groups. This is probably an effect of the difficulty of maintaining a fixed speed in driving simulators .
Because the study population was limited to older adults, the results are only generalizable to this age group. The main motive for using this age group was that the HL population consists mainly of older adults. It has also been shown that age is strongly related to attitudes towards HL and transportation . For each participant the gaze data was analyzed on 39 sequences each of normal driving and driving with the task. A complicated full model and an irregular non-response forced a simplification in the test for the interaction between hearing and condition such that the result is not fully generalizable to the population and rather should be seen as a result for this specific sample only.
Differences in visual behaviors between drivers with NH and drivers with HL are bound to the driving condition. When driving and performing a secondary task, drivers with HL look away from the road as much as drivers with NH, but with more frequent glances of shorter duration.
Statistical hypothesis-testing showed a significant decrease in secondary task performance for the HL group. The results also indicate that drivers with HL use fewer but more focused glances than NH drivers; they also perform a visual scan of the surrounding traffic environment before looking away towards the secondary task display.
We would like to give our special thanks to Promobilia, who supported this study financially.
Open Access This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.
- Hickson L, Wood J, Chaparro A, Lacherez P, Marszalek R (2010) Hearing impairment affects older people’s ability to drive in the presence of distracters. J Am Geriatr Soc 58(6):1097–1103View ArticleGoogle Scholar
- Roth TN, Hanebuth D, Probst R (2001) Prevalence of age-related hearing loss in Europe: a review. Eur Arch Otorhinolaryngol 268:1101–1107View ArticleGoogle Scholar
- Lin FR, Niparko JK, Ferrucci L (2011) Hearing loss prevalence in the United States. Arch Intern Med 171:1851–1853View ArticleGoogle Scholar
- HRF (Hörselskadades Riksförbund [The Swedish Hard of Hearing Society]) (2009) HRF Rapport [HRF Report]. Hörselskadades Riksförbund, StockholmGoogle Scholar
- Barreto SM, Swerdlow AJ, Smith PG, Higgins CD (1997) A nested case–control study of fatal work related injuries among Brazilian steel workers. Occup Environ Med 54:599–604View ArticleGoogle Scholar
- Ivers R, Mitchell QP, Cumming RG (1999) Sensory impairment and driving: the blue mountains Eye study. Am J Public Health 89:85–87View ArticleGoogle Scholar
- Picard M, Girard SA, Courteau M, Leroux T, Larocque R, Turcotte F, Lavoie M, Simard M (2008) Could driving safety be compromised by noise exposure at work and noise-induced hearing loss? Traffic Inj Prev 9:489–499View ArticleGoogle Scholar
- McCloskey LW, Koepsell TD, Wolf ME, Buchner DM (1994) Motor-vehicle collision injuries and sensory impairments of older drivers. Age Ageing 23:267–273View ArticleGoogle Scholar
- Thorslund B, Peters B, Lyxell B, Lidestam B (2013) The influence of hearing loss on transport safety and mobility. Eur. Transp. Res. Rev. 5:117–127 (Published online: 30 November 2012)Google Scholar
- Green KA, McGwin G, Owsley C (2013) Associations between visual, hearing, and dual sensory impairments and history of motor vehicle collision involvement of older drivers. J Am Geriatr Soc 61:252–7View ArticleGoogle Scholar
- Glad A (1977) Requirements regarding drivers: hearing ability. Transportekonomiskt Institut, Oslo, p 30Google Scholar
- Magnet W (1992) Empirische Untersuchung zur Kompensationsfrage bei Gehöhrlosen Autofahrern. Eine Differentieller Analyse der visuellen Wahrnehmung von Gehöhrlosen Kraftfahrern [Empirical study on the question of compensation in motorists with hearing loss. A differential analysis of the visual perception of drivers with hearing loss]Google Scholar
- Thorslund B, Peters B, Lidestam B, Lyxell B (2013) Cognitive workload and driving behavior in persons with hearing loss. Transp Res Part F 21:113–121View ArticleGoogle Scholar
- Neville HJ, Lawson D (1987) Attention to central and peripheral visual space in a movement detection task: an event-related potential and behavioral study. II. Congenitally deaf adults. Brain Re 405:268–283View ArticleGoogle Scholar
- Dye MW, Hauser PC, Bavelier D (2009). Is visual selective attention in deaf individuals enhanced or deficient? The case of the useful field of view. PLoS One 4): e5640Google Scholar
- Andersson U (2002) Deterioration of the phonological processing skills in adults with an acquired severe hearing loss. Eur J Cogn Psychol 14:335–52View ArticleGoogle Scholar
- Sivak M (1996) The information that drivers use: is it indeed 90 % visual? Perception 25:1081–1089View ArticleGoogle Scholar
- Arlinger S (ed) (2007) Nordisk lärobok i audiologi [Nordic textbook of audiology]. C-A Tegnér AB, StockholmGoogle Scholar
- Conrad R, Hull AJ (1964) Information, acoustic confusion and memory span. Br J Psychol 55:429–437View ArticleGoogle Scholar
- Sternberg S (1966) High-speed scanning in human memory. Science 153:652–654View ArticleGoogle Scholar
- Chartier S, Renaud P (2008) An online noise filter for eye-tracker data recorded in a virtual environment. 2008 Symposium on Eye Tracking Research and Applications, SavannahView ArticleGoogle Scholar
- Salvucci DD, Goldberg JH (2000) Identifying fixations and saccades in eye tracking protocols. Eye tracking research and applications symposion, palm beach gardens. ACM Press, New YorkGoogle Scholar
- Ahlstrom C, Victor T, Wege C, Steinmetz E (2012) Processing of eye/head-tracking data in large-scale naturalistic driving data sets. Intell Transp Syst 13:553–564View ArticleGoogle Scholar
- Savitzky A, Golay MJE (1964) Smoothing and differentiation of data by simplified least squares procedures. Anal Chem 36:627–1639View ArticleGoogle Scholar
- Nystrom M, Holmqvist K (2010) An adaptive algorithm for fixation, saccade, and glissade detection in eyetracking data. Behav Res Methods 42(1):188–204View ArticleGoogle Scholar
- Beinhauer WA (2006) Widget library for gaze-based interaction elements. Proceedings of ETRA: Eye Tracking Research and Applications Symposium, ACM Press, San DiegoView ArticleGoogle Scholar
- Ahlstrom C, Dukic T (2010) Comparison of eye tracking systems with one and three cameras. 7th International Conference on Measuring Behaviour, Eindhoven, NetherlandsGoogle Scholar
- ISO (2002). Road vehicles – measurement of driver visual behaviour with respect to transport information and control systems. Part 1: Definitions and parameters. ISO 15007–1:2002Google Scholar
- ISO (under development). Road vehicles – measurement of driver visual behaviour with respect to transport information and control systems. Part 2: Equipment and procedures. ISO/DTS 15007–2Google Scholar
- Victor TW, Harbluk JL, Engström JA (2005) Sensitivity of eye-movement measures to in-vehicle task difficulty. Transport Res F: Traffic Psychol Behav 8:167–190View ArticleGoogle Scholar
- Nilsson, L. (1993). Contributions and Limitations of Simulator Studies to Driver Behaviour Research. Driving Future Vehicles. A. A M Parkes, Franzen, S., Taylor & Francis: 401–407Google Scholar
- Mullen, N., J. Charlton, et al. (2011). Simulator validity: Behaviours observed on the simulator and on the road. Handbook of Driving Simulation for Engineering, Medicine, and Psychology. D. L. Fisher, M. Rizzo, J. Caird and J. D. Lee. Boca Raton, FL, CRC Press: 12:11–12:13Google Scholar
- Mullen, N., Charlton, J., Devlin., A., Bédard, M. (2001). Simlutaor Validity: Behaviors Observed on the Simulator and on the Road. Driving Simulation for Engineering, Medicine and Psychology. D. L. Fisher, Rizzo, M., Caird, J.K., Lee, J.D., Taylor and Francis GroupGoogle Scholar
- Ferris FL, Kassoff A, Bresnick GH, Bailey I (1982) New visual acuity charts for clinical research. A J of Ophthalmol 94:91–96View ArticleGoogle Scholar