Driver-vehicle interfaces and interaction: where are they going?

Abstract

Car evolution

The car was born around a century ago and its evolution has been incredibly fast, both in technology and in style. We have to move through different social and cultural evolutions to arrive to the present state of the art. The technical and social acceleration of the 20th century is well visible looking at the different worldwide research programs. Nowadays digital content and ubiquitous computing are changing us and our life style. New concepts involving the full society are emerging and the term “personal mobility” becomes more and more used together with “co-operative driving” and “environmental compatibility”.

HMI evolution

Human Machine Interaction (HMI), initially limited only to the primary in-vehicle commands, has been a major issue since the beginning. In which direction is it moving? Which technological efforts will be key factors to face the challenges of the future? We are in the middle of a transition phase where the world has to cope with and to solve big problems as energy and climate change that can strongly influence the future of the automotive industry and not only.

1 Introduction

Every imagined world that is not seated in the past must share its mode of non-being with the future, which by definition does not exist

(Sean Cubit—“Digital Aesthetics” 1998)

Human Machine Interaction (HMI) has been a major issue since the very early phase of car development. Initially it was limited only to the primary commands devoted just to drive the vehicle as steering wheel, accelerator and brake, but after a short while, with the evolution and diffusion of cars, became essential for drivers to know more about the state of the vehicle starting from basic information such as speed and fuel level.

This was the reason why the instrument cluster was conceived: at the beginning it was made of separate instruments, nowadays it includes an ever increasing number of indicators and displays.

The transformation was mainly due to the diffusion of cars to a mass market asking initially for a new concept of mobility and afterwards for a higher comfort.

What is characterizing the car evolution is the enormous effort done by the automotive industry to develop new technological solutions. This effort moved the car industry towards different objectives and related needs: cost reduction, better performances and increased quality have been the driving factors of this process [1].

One of the leading factors of the progress done by car industry is electronic. “Current production cars contain more computing power than was used to send the Apollo spacecraft to the moon” (MOTOROLA). The result is distributed in different areas: from better efficiency power train to passive and active safety, to climate control and multimedia applications.

The technical and social acceleration of the 20th century is well visible looking at the numerous research programs in Europe, USA, Japan and Australia. The various programs dedicated to traffic, mobility and safety have produced results that are now present in many vehicles. Names as ABS (Anti-lock Brake System), ESC (Electronic Stability Control), ACC (Adaptive Cruise Control), Lane support, Side support [2] are today well known by the customer market.

In this scenario of continuous evolution of vehicles we are now in a period where technology is rapidly changing vehicles and the way in which information is exchanged with drivers is now of crucial importance.

2 The driver–vehicle interaction

The main characteristic of the ongoing evolution in this society is the need to communicate anywhere and anytime, namely to be continuously connected.

The increasing amount of information and support functions is today forcing HMI designers to face the problem of contemporary warnings, messages and communications that are given to drivers at the same time or when they are engaged in demanding driving conditions.

Different solutions have been investigated and now, after a number of projects co-funded by the European Commission as CEMVOCAS [3], COMUNICAR [4], AIDE [5], a clear understanding of when and how to provide information to the driver is reached: the integrated management of information based on message prioritisation depending on driving context, together with the use of new technologies like for example haptic devices and voice recognition, are the present concept of a smart, friendly and adaptive HMI, the so called “natural interaction”. These new HMI concepts enable the possibility to maintain driver’s workload at an acceptable level, avoiding distraction and consequently increasing safety, providing the driver with all the useful information coming from the novel driving support and information functions.

If CEMVOCAS project made the first attempt to examine the driver workload while coping with the oncoming phone calls and to apply this concept to an on vehicle speech recognition system, the COMUNICAR project developed the first rule-based information manager and the AIDE integrated project, makes evolve this concept developing an adaptive and integrated driver vehicle interface. The AIDE project, co-funded by the European Commission Information Society and Media and supported by EUCAR, ended in April 2008 and involved nine car and trucks manufactures all over Europe, major automotive OEMs, suppliers and research centres, the results of this project are now in the hands of most of the European automotive industries that are today planning the deployment phase of the AIDE project results.

Using a number of different data from the on board devices and sensors, the AIDE system is aware in real time of the driving context in terms of traffic and environmental conditions, of driver’s status and activity. The AIDE system uses this “awareness” to adapt in the most efficient and safe way the provision of the information to the driver limiting the amount of simultaneous messages and warnings given to the driver not only from car functions but also from his/her personal devices [6, 7] Figs. 1 and 2.

Fig. 1

The AIDE adaptive integrated driver-vehicle interface

Fig. 2

The AIDE demonstrators developed by Centro Ricerche FIAT, Volvo Technologies and SEAT

In the AIDE project in fact also portable devices have been included into the concept of information management, a personal mobile phone can be seamlessly connected and managed by the central intelligence of the system keeping the connectivity always on, as demanded by today drivers, but enabling the full hands free control of the smart phone functions including SMS and MP3 player management.

To address the challenges of connectivity and safe use in cars of nomadic devices the AIDE integrated project established a Nomadic Device Forum that brought together representatives of the key stakeholders involved in this field. The activity of the Forum addressed the most important use cases and requirements to define the perspectives of a common “nomadic device gateway” for the integration of portable devices in the vehicles in compliance with the European Statement of Principles (ESoP2) defined by the European Community about in-vehicle HMI.

The AIDE system is extending the concept of information management from the use of rule based algorithms to the development of a dynamic predictive model conceived to take into account in a more extensive way driver’s behaviour and profile, vehicle dynamics and driving context.

The AIDE concept is thus to create a sort of “real time virtual awareness layer” gathering all relevant information about the context, the vehicle dynamic and the driver’s behaviour and status to close the loop and to put the driver into the loop itself. In this way the subsequent communication channel selection and information prioritisation is a powerful mean to increase driving comfort and safety.

The design of the driver-vehicle dialogue is the core of the AIDE “Interaction and Communication Assistant” (ICA) that defines the communication and data exchange protocol.

ICA is the central intelligence of the AIDE system; it is responsible of managing all the interaction and communication between driver, vehicle and driver’s personal nomadic devices. Starting from the assessment of the Driver-Vehicle-Environment (DVE) status/situation provided by the DVE monitoring modules ICA enables the selection of the presentation modality, the messages prioritisation and scheduling and the global adaptivity of the driver-vehicle interface (e.g. display configuration and function allocation).

The main goal of ICA is to manage all interactions between the driver and the various in-vehicle systems in order to:

• avoid any negative impact of the information sources on the driving task (e.g. distraction and information overload),

• avoid interference between different pieces of information.

The management of the functions includes:

• the definition of which type of information should be delivered, when and how,

• the adaptation to the driver and to the environment,

• the personalization of the adaptive HMI to the individual driver.

In cars of today the elements that can contribute to the interaction with the new vehicle functions are different and distributed all around the driver. For example:

• the steering function is normally assisted and the amount of mechatronics able to face different and also difficult driving conditions can be used at the same time to alert the driver in case of erroneous lane departure;

• the accelerator pedal is rapidly evolving to become a source of feedback linked to functions like ACC (Adaptive Cruise Control) or frontal collision warning;

• the safety belt interacts with the driver for example to remind its use and in some cases its pretensioning is used to prepare the driver’s body in case of crash or as a light warning of dangerous headway;

• the seat generates comfort and is also used as output device for lateral warning in some implementation of the lane support function;

• the climate control could be used to interact with the driver with fresh air or fragrances in case of fatigue;

• the lateral acceleration and more in general the vehicle handling contributes to create the feeling of dangerous driving.

On one hand we are moving to integrate and manage the information provided to the driver, on the other hand we are working to use different communication channels, in all cases these considerations should be taken into account:

• driving pleasure is done by emotions coming primarily by physical feelings;

• drivers frequently have not consciousness of the systems that are active on the vehicle.

The first thought gives us the idea that with affective computing applications, nowadays well represented in pilot projects like FEELIX GROWING [12] and encouraged by the vision where “computers learn to react to our motions and thoughts” (Intel) [13], the drive task will become a real immersive emotional experience not giving the driver the perception of the high level of artificial intelligence distributed around him/her.

The second consideration may result in a risk of overconfidence by drivers, an effect that should be investigated in the early design phases of a new system and should be prevented.

3 Where we are: a short state of the art

Looking at the hints concerning the car interiors and HMI offered by concept vehicles and by cars already on the market we mainly see the implementation of the following criteria:

• increasing of the support to the driver (preventive safety);

• satisfaction of driver’s need for connectivity;

• increasing of entertainment for passengers;

• re-thinking to the interior as a useful functional space.

Concerning the support to the driver, as confirmed by a number of successful research projects of the last decade [8–11], the number of safety related functions available on board is continuously increasing, from the already widely commercially available functions like ABS (Anti-lock Brake System), ESC (Electronic Stability Control), etc. to the more advanced ADAS (Advanced Driver Assistance Systems) functions like collision avoidance, lane departure warning, blind spot monitoring, and others based on on-board integrated sensors like radars and cameras, relying on satellite systems like GPS (Global Positioning System) or taking advantage from wireless communication with infrastructures and remote service providers. All these systems are providing a higher level of support to the driver both in terms of automatic intervention and of additional warnings augmenting driver’s awareness of the external scenario.

Concerning interiors and comfort, interesting examples are related to accessibility (Renault Megane coupè concept car Geneve 2008, Toyota Fine-T concept 2006), to in-vehicle office (Mercedes-V mobile office or the Kia Soul at Genève 2008) or to the needs of families with children (Mercedes F600 Hygenius concept) (Fig. 3).

Fig. 3

Renault Megane coupé, Toyota Fine-T, Kia Soul, Mercedes F600 Hygenious concept cars

Concerning entertainment of passengers on board, the evolution is so rapid that in this area we have not only concepts but also products integrating the most popular multimedia (from DVD to MP3 players and videogames) (Fig. 4).

Fig. 4

Fiat, Lancia, Maserati, BMW, Chrysler, Mercedes, Volvo in-vehicle entertainment solutions

In this scenario FIAT developed the Blue&Me innovative telematic solution for cars as first results of a strategic partnership with Microsoft Business Unit Automotive (Fig. 5). Blue&Me is based on Windows Mobile and uses an open, updatable system with modular contents and an associated offer of telematic services: with the support of Magneti Marelli, a platform has been developed that is compatible with most of mobile phones, music players or other personal electronic devices.

Fig. 5

The FIAT Blue&Me system

Blue&Me is a good example of integration of portable devices in the car environment: the voice command system, which is completely integrated with the steering wheel controls and the information display, allows customers who own a Bluetooth mobile phone to use it even when it is in the pocket of a jacket or in a handbag, without having to move the hands off the wheel.

Thanks to the advanced voice recognition and synthesis system, it displays and reads out aloud incoming SMS messages, and understands the user’s vocal commands without the need for a voice learning phase.

Among the various audio functions it offers, driver can listen to music, stored on his/her mobile phone, MP3-player or pen-drive as a USB port allows to connect any digital device fitted with a USB connection. It also offers a navigation system and the access to a set of services such as traffic and other useful information.

As shown by the Blue&Me example, the wireless technology (Bluetooth, WiFi, etc.) has allowed our personal smart phones, PDA (Personal Digital Assistants), PND (Personal Navigation Devices) entering into the car systems and bringing on board personal contents that can be used for entertainment or work via the displays and commands that are part of the car interior.

The speech recognition technology is mature enough to be used instead of complex manual actions, like menu selections and destination entry to the navigation system, reducing driver’s distraction while synthesized voice is already used from several car functions to communicate with the driver.

It’s a matter of fact that the portable technology gave us the possibility to be always connected, the continuity to communicate without interruptions both outside and inside the vehicle and the opportunity when we change car, to continue to use our mobile phones and portable navigation systems is a reality of today.

If these devices are logically but not physically integrated in the car system they can take advantage of an integrated and safe on board HMI, compatible with the driving task, keeping the continuity and the freedom of their use outside the vehicle.

This scenario is nowadays true for many different and not only luxury cars.

For what it concerns HMI devices this scenario has the consequence of a growing number of displays and new input devices as buttons, rotary knobs and touch screens on the dashboard and the steering wheel but there is a general attempt, confirmed also by the results of the aforementioned research projects COMUNICAR and AIDE, to integrate the information and its management to optimize and rationalize the dialogue between the driver and the vehicle.

All car manufacturers are focused on the very important topic of usability of human-vehicle interaction and the quality of the graphic interfaces is constantly improving thanks to the enhanced capabilities of display technologies, electronic control units and sophisticated software tools now permitting high level graphic and 3D performances. Nowadays the on board systems are becoming “real computers” able to manage the ever increasing amount of data with higher storage capabilities and computational speed.

New emerging materials and technologies are making real in a near future the possibility to spread images and input devices on whichever surface and in the space as the potentiality of 3D virtual images or new technological e-textiles and plastics are suggesting (Fig. 6).

Fig. 6

Transparent, virtual image, 3D, holographic, flexible displays. Touch sensitive tissues, touch screens, projection keyboards

This is reflected also in the vision of the future presented in a number of concept cars (Fig. 7) where big and numerous displays are replacing the traditional instruments making the dashboard becoming lighter and re-configurable. These concepts are supported by the current technologic trends of the major display suppliers, with futuristic solutions as transparent or flexible displays spread on the dashboard and the windshield surface, with a large use of virtual images and special light effects involving the whole car interior.

Fig. 7

HMI solutions in Nissan, Peugeot, Alpine, Ford concept cars

4 Where are we going? Users and society contribute to design a complex scenario

In the document “2006 transport policy review impact assessment” for the Road Sector it is reported that: “Due to the flexibility of its technology and infrastructure, road transport is the only motorized mode being able to provide door-to-door services,… It can provide flexible services regarding departure time and destination, and it is the fastest transport mode for distances up to about 500 km. Thus it is not surprising that this mode remains highly attractive, despite its weaknesses, such as its high toll as regards accidents and fatalities, and the pressure it exerts on the environment.”

In general the scenario that we will have to manage is complex due to different factors influencing the present and the future: the need to increase road safety, the need to reduce CO₂ emissions and fuel consumption, and in general the need to design and implement a sustainable mobility for all citizens in scenarios of growing complexity like in megalopolis.

The idea that for the future society “space will be an essential resource”, as well as time and energy, is already a challenge of today for OEMs that are reducing vehicles dimensions and creating specific models of low emissions or alternative energy city cars (Fig. 8).

Fig. 8

FIAT Philla (solar), Toyota IQ (hybrid), Citroen C-Cactus (low emissions) concept cars

For example, the Smart City Group of Media Lab at MIT [14] is looking for foldable city vehicles easy to be parked and better to be stored (Fig. 9) and the Nissan Pivo2 concept car (Geneve 2008) as well as the Peugeot Moovie concept are proposing similar ideas (Fig. 10).

Fig. 9

MIT foldable concept car

Fig. 10

Nissan Pivo2, Peugeot Moovie and Nissan Mixim concept city cars

It seems that the attention for city cars is growing not only at European but at worldwide level, advanced studies and a number of motor shows and car exhibitions show numerous examples of technical solutions to deal with the request to reduce the space (i.e. Nissan Mixim concept 2007) (Fig. 10).

What emerges from these concept vehicles is a deep modification of the overall idea of the vehicle itself, with radical changes that have the potentiality to affect future vehicle interiors and not only.

New materials and technologies permit to develop new concepts to cope with both space and mobility challenges and new adjectives as “foldable”, “flexible”, “wearable” are starting to be applicable to the automotive world. As MIT is working on the foldability to reduce space, others are working on flexibility to modify shapes. One example is BMW that working on the concept of a flexible car and using the potentiality of the more advanced fabrics conceived the GINA (Geometry and functions in ‘N’ Adaptions) concept car (Fig. 11), based on a lightweight stretchable fabric which acts like a skin. The result of this textile based car frame is a customisable body that can adopt conformations that were not previously possible with metal based cars, and allows the owner to modify a number of features of the car such as the spoiler, wings, bonnet and even the dashboard inside the car.

Fig. 11

BMW GINA flexible concept car

Tissues make us immediately think to clothes, and just the wearability idea is well exemplified by the i-Swing concept proposed by Toyota (Fig. 12) or by the electric exoskeleton motorcycle concept Yamaha Deus Ex Machina (Fig. 13) designed by a student of the Art Center College of Design, a bike that once parked, the rider can strap it on like a suit.

Fig. 12

Toyota i-Swing and other wearable concept vehicles

Fig. 13

Yamaha Deus Ex Machina concept and other examples of personal and wearable vehicles

Is thus the car transforming in a special moving wearable device? Maybe not yet, but the idea to have clothes interacting with the vehicle as a new communication channel is not far to become a reality. Technological and functional tissues already exists and lot of examples have been proposed in the consumer electronic market demonstrating the possibility to integrate portable devices with our garments or in the future to have clothes directly speaking and communicating with us and with other devices through images, sounds or sensations. In the car interiors these tissues could be used as new interaction channels giving us information and haptic feedbacks, distributed all around our body and on the car internal surfaces.

In addition to that, new concepts involving the full society are frequently mentioned and the term “personal mobility” is becoming more and more used together with “co-operative driving” and “environmental compatibility”.

It’s nowadays a reality that in crowded countries as UK experiments are launched to verify and validate the “pay-as-you-drive” concept [15]. The EcoDrive solution already realized by FIAT and Microsoft via the Blue&Me platform [16] enables drivers to minimize their impact on the environment and may evolve into a transformation of our feeling when driving. It operates on a simple principle: it collects all necessary data related to vehicle efficiency and, through an on board USB gate, stores the data into a standard USB key. The driver can then plug the USB key in any PC and the EcoDrive system presents the driver, via a dedicated internet portal, the detailed environmental performance of the car including the CO₂ emission level for each trip. It analyses driver’s style and then provides tips and recommendations on how to modify and improve it to achieve CO₂ reductions—and save money on fuel. This solution demonstrates that the possible measures that OEMs can undertake to reduce the level of CO₂ emissions are not only limited to the design of novel or optimised engines based on existing or on alternative energy sources, but can be extended to telematic applications.

In the near future the introduction of the IPV6 protocol will increase our possibility to have data exchange, communication, and personal information available everywhere in a seamless way on different devices and environments. Nowadays it is already possible to foresee that in the future [17] our personal habits will probably change.

Will our thinking be different? [18] The answer is probably “yes” and the transformation is under our eyes: it’s enough to look at the importance for the young generation to maintain contacts everywhere and at anytime.

5 Conclusions

This paper aims to be a cue for thought on where the human-vehicle interaction is going, looking at the progress done up to now and at the present scenario that is about to bring cars on the market very different from those we are accustomed to use nowadays.

We started wondering in which direction the in car driver–vehicle interfaces and interaction are moving. What we can do now is go back to the starting point and think again to “the road” done by cars up to now. From the beginning of the car industry, until a few years ago, the most used “attributes” of cars were words like: distinctive, elegant, and sportive. From distinction and elegance derived concepts very important as comfort and style while sport has always been associated with high performance cars.

In the second part of the 20th century a number of new attributes like safety, friendliness and environmental compatibility started to become more and more important and finally and finally the 21st century began with the massive introduction of the concept of connectivity [19].

Which will be the car “attributes” of the future? Maybe new, unexpected needs and fashions will arise, but in any case the design and development of new technologies and devices will have to face the challenges opened by the new paradigms.

A brief glance at the state of the art shows us that the current research in the field of human-vehicle interaction is going to the challenge to enable the connectivity demanded by drivers while at the same time increase driving safety.

It is expected that men and women of the future when moving will continue their normal life, leisure and work while the car will take care of their safety. The driver and the other passengers will probably be immersed in a not too huge but comfortable environment, flexible enough to enable also pleasant activities while travelling. Car “inhabitants” will entertain themselves and communicate with the world outside, all useful and interesting information will flow fluently to the driver without distracting him/her from the primary driving task: driver’s attention will remain focused on driving and, depending on car real time awareness of how much the driving task is demanding, the driver will be enabled to manage other information that will be conveyed not only through his/her eyes and ears but also through his/her skin and feelings.

Relaxing conditions will be guaranteed by the fact that the car, thanks to its intelligence and cooperation with the other vehicles and the external world, will take care of making travel fluent and safe.

We are now in the middle of a transition phase where the world has to cope with and solve big problems as energy and climate change that can strongly influence the future of the automotive industry and not only. New markets like China and India are very promising for all vehicle makers, at the same time they represent a challenge, in fact some companies are starting to produce new low cost vehicles for those markets that don’t suit yet American and European regulations. Other debates are open on the new role of electronics in the future vehicles or on the need to adapt to the novel low-cost cars.

Taking care of all these different factors it is reasonable to ask ourselves in which way the new markets and technologies may affect the HMI trends here briefly discussed.