Digibus©: results from the first self-driving shuttle trial on a public road in Austria

Abstract

From April to November 2017, the non-profit research organisation Salzburg Research conducted the “Digibus© 2017” trial, the first trial of a self-driving shuttle on a public road in Austria. The shuttle from the French company Navya Tech has been tested on a 1.4-km long track in the village of Koppl, which is situated approximately ten kilometres east from the City of Salzburg. The trial in Koppl was one of the first trials worldwide on public roads with mixed traffic in a rural area. The focus of this trial was on the real-world evaluation of a self-driving shuttle for bridging the first/last mile in public transport. From April to November 2017, 240 test drives with 874 passengers covering 341 test kilometres have been conducted. Results show that the technology is ready for testing, but there is still a long way to go for driverless operation, especially in mixed traffic scenarios. The work describes the trial setting, the test route, the process of deploying the shuttle, experiences collected during the trial as well as results from a passenger survey. The accompanying passenger survey with 294 participants revealed high acceptance and a good feeling of safety.

1 Introduction

In June 2016, the Austrian Federal Ministry of Transport, Innovation and Technology presented the Austrian Action Plan for Automated Driving [2]. The action plan defines seven automated driving use cases. One use case - called “New Flexibility” - pursues the goal of highly flexible automated and networked vehicles for passengers in an intermodal public mobility system. In 2017, the European Road Transport Advisory Council (ERTRAC) published Version 7 of the European Roadmap for the development of automated driving in Europe [6]. In this roadmap, a development path for automated urban mobility systems is presented alongside development paths for passenger cars and vehicles for freight transport. According to the Austrian Action Plan as well as the ERTRAC Automated Driving Roadmap, “automated passenger shuttles” are supposed to play a major role in future public mobility systems, predominately as feeders of intermodal mobility hubs [1].

During the last years, several tests with self-driving shuttles have been conducted starting with the trials of the European CityMobil2 project in Lausanne, La Rochelle and Trikala [9]. The focus of these trials was on testing the vehicles in mixed traffic environments with cyclists and pedestrians as well as the evaluation of passenger acceptance. Due to the technological progress and the market-entry of suppliers (e.g. French start-up companies Navya Tech and EasyMile), the technology is ready to move from test sites to public roads. So far the shuttles have been used for trials on public roads in Switzerland [14], the Netherlands [20] and Finland [16]. A trial on private roads with a prototype of the “Olli” shuttle from Local Motors has been conducted on the EUREF-Campus in Berlin [10]. Further tests are planned or ongoing in Switzerland [18, 19] and Germany [4, 8, 12].

While most of the previous or current trials were conducted on public roads in city centres, the “Digibus© 2017” trial¹ in Austria was one of the first trials in a rural area. Due to the above-mentioned action plan from the Austrian Ministry of Transport, Innovation and Technology the trial could directly start on a public road facing real traffic conditions. Especially in rural areas closing the first/last mile with public transport services is one of the challenging questions where self-driving vehicles could play a major role in the future. In contrast to previous city trials, the shuttle had to cope with a road mostly lacking road markings, varying inclines, varying mobile network coverage, varying quality of GNSS and correction signals, other road users driving at speeds up to 60 km/h per hour or varying weather conditions (different temperature ranges, rain or fog). Together with the qualitative feedback from passengers, results allow for a realistic estimation of the current state of technology.

This work describes and discusses the aims of the trial, the applied methodology including test permit, test setup test drives and test vehicle, gathered experiences during the test drives with focus on deployment, positioning, automated driving capabilities and interaction with other road users as well as passenger feedback.

2 Methodology

This Section describes the aims of the trial, the process of getting the test permit, the test setup as well as vehicle characteristics.

2.1 Aims

The “Digibus© 2017” trial pursued two aims: The first aim was the real-world evaluation of the autonomous driving capabilities of a self-driving shuttle for bridging the first/last mile public transport scenario in a rural area. As mentioned earlier, previous trials of self-driving shuttles predominately focused on urban areas or aimed at demonstrating the technology in closed areas. The municipality of Koppl, due to its rural environment, topographical layout, poor public transport coverage and the strong support from the municipality, offered an ideal trial setting. The trial was conducted as a black box trial as the operators of the shuttles, despite being trained by the vehicle supplier for operating the shuttle, were not aware of the details of the automated driving software.

Beside the evaluation of autonomous driving capabilities, the second goal was to gather qualitative feedback from passengers, most of them experiencing a self-driving shuttle for the first time. Since the test track in Koppl is a typical first/last mile scenario, we organised test drives for the local population since the shuttle was not operated on a regular basis. Other groups of people testing the shuttle were interested delegations from different organisations (public transport operators, technology suppliers, government officials, etc.), political representatives and press delegates.

It is worth to mention that the “Digibus© 2017” trial is one of the few independent evaluations of a self-driving shuttle carried out by a publically owned, non-profit research organisation being not driven by commercial interests.

2.2 Test permit

Before starting the trial, it was necessary obtaining a permit from the Austrian Ministry of Transport, Innovation and Technology (BMVIT). Since December 2016, a regulation of the Austrian Federal Ministry of Transport, Innovation and Technology, the so-called “AutomatFahrV”, defines basic conditions for testing automated vehicles on public roads in Austria [3]. The regulation allows for test drivers under certain conditions to transfer driving tasks to assistance systems or automated driving systems. This applies to systems which have already been approved and are in series (for example a jam assistant), but are currently not allowed to be used due to existing drivers’ obligations. On the other hand, the regulation allows testing of completely new systems at research and development stage, not complying with existing regulations. As outlined below, Section 2, §7 of AutomatFahrV defines certain rules for testing self-driving shuttles on public roads (excerpt):

• For the purposes of this regulation, a self-driving shuttle is a vehicle of categories M1, M2 and M3 equipped with a system capable of handling all driving tasks at a speed up to 20 km/h.

• This system may be tested by vehicle manufacturers, system developers and research institutes.

• The system may only be used on public roads with mixed traffic if at least 1000 test kilometres have been previously covered by the system.

• The self-driving shuttle may be tested on a predefined test route only.

• As soon as the driver activates the system, all driving tasks are transferred to the machine. In this case, the system must be able to handle all driving situations automatically.

• The vehicle must offer an emergency button to deactivate the system at any time. If a critical situation arises, the driver must immediately press the emergency button.

• The system may be tested up to a maximum speed of 20 km/h.

• During the test period, persons may only be transported on the intended seats and not on a commercial basis.

At the end of 2016, Salzburg Research applied for a test permit for testing a self-driving shuttle on a public road based on the AutomatFahrV. Since the Navya ARMA DL4 shuttle is missing a European type approval, according to the AutomatFahrV regulation (§ 4) the shuttle is considered a vehicle in research and development stage and therefore, may be only tested on public roads with test license plates. The local authorities issued these license plates to Salzburg Research upon the AutomatFahrV regulation and a signed liability insurance contract. On April 20th, as first organisation in Austria, Salzburg Research got the permission to conduct test drives with a self-driving shuttle on public roads. The national contact point for automated driving at AustriaTech was of great help during the application phase.²

According to the AutomatFahrV regulation, a trained operator with at least driving license B has to be present in the shuttle being able to take over control at any time during the test drives. Before starting the test drives, each operator (six persons agreed to take over the role as operators) had to undergo a 2 days’ operator training from Navya Tech. The training included technical vehicle specifications, manual driving, driving in autonomous mode as well as reporting and emergency management.

2.3 Test setup

As test route, a 1.4 km long public road in the village of Koppl - 10 km east from the City of Salzburg - has been chosen (Fig. 1). The route represents a so-called first/last mile scenario. In public transport bridging the first/last mile - i.e. the way from the stop to the destination or to the home - is critical for customer acceptance. In the case of Koppl, the centre of the village is 1.4 km from the major bus line connecting the village with the city centre. Although a public bus connection to the village centre exists, due to economic reasons it is not operated on a regular basis. In the future, bridging this first/last mile with a self-driving shuttle could provide more flexibility and higher acceptance of public transport. Figure 1 gives an overview of the test route as well as the physical and digital infrastructure. The actual driving path along the route was selected in cooperation with the vehicle supplier, the municipality and the road authority (Federal State of Salzburg).

Fig. 1

Map of the test route and overview of the used physical and digital infrastructure

The deployment of the shuttle on the test route was accomplished in two phases: Firstly, the shuttle was deployed on a short route with a total length of about 400 m and a driving time of approximately 5 min. This short route in the village centre guaranteed a fast setup and immediate testing. The long route has a length of 1.4 km (one-way), thus driving the whole route with 2.8 km takes approximately 20 min (at a maximum speed of 16 km/h including stops and slow driving manoeuvres). Beside the start and end stops, the route includes two additional bus stops for each driving direction. The main characteristics of the route are: Some parts lack buildings or landmarks for accurate LIDAR positioning, some parts lack trustworthy GNSS and correction signals, non-signalised intersections, most parts lack road markings, maximal incline of 8%, fast-changing weather conditions, other traffic participants driving at speeds up to 60 km/h.

The deployment of the shuttle on the test route followed a defined deployment procedure from Navya Tech. Firstly, the shuttle has to be manually driven on the planned route for data acquisition. Secondly, the resulting path as well as the 3D LIDAR map are manually edited (e.g. cleaning of the 3D LIDAR map, path adjustments, path attributions like driving speeds or traffic rules, vehicle stops). After getting a first driving path and 3D LIDAR map, the vehicle is ready for the first test drives. The last step of the deployment procedure is to repeat the manual path editing and attribution as long as a satisfying level of quality has been reached. Upon supplier and customer satisfaction, the vehicle is ready for operation.

2.4 Test drives

All test drives have been conducted following a predefined test procedure. Firstly, the shuttle was manually driven from the garage to the starting point in front of the municipal office (public transport stop in the village centre). After initialising the systems, a test drive was conducted with only operators on board. If this initial test drive has been successfully completed, the actual test or demonstration drives (sometimes with external passengers) were started. All reported results and experiences are based on test protocols completed by the operators after each test drive. Since Navya Tech did not provide any access to drive data, it was impossible to do a more comprehensive analysis of the driving performance (beside the analysis of operator protocols).

Directly after each test or demonstration drive with passengers, passenger feedback was gathered via an online survey on a smartphone. The survey covered eighteen questions and dealt with the following topics: prior knowledge of automated driving, prior experiences with self-driving shuttles, test purpose, pleasure in driving, perceived sense of safety during the test drive, conceivable possibilities of use, possession of a private car, conceivable replacement of a private car by a self-driving shuttle and demographic data.

2.5 Test vehicle

The “Digibus© 2017” trial in Koppl has been conducted with the Arma DL4 model from Navya Tech [13] (Fig. 2). Navya Tech is specialised on the design of electrical, autonomous vehicles. The self-driving shuttle Navya Arma DL4 is electrically powered and can theoretically reach a speed of 45 km/h. According to the AutomatFahrV regulation, the maximum allowed speed on public roads for test drives is limited to 20 km/h (the maximum speed during the trial has been limited to 16 km/h due to safety reasons). The shuttle has a capacity of maximum 11 sitting passengers however, due to the regulations of the driving license B in Austria (which is the operators’ driving license) a maximum of nine persons (including the operator) must not be exceeded.

Fig. 2

a Digibus© bus stop in Koppl village centre, (b) Steepest part of the long route

3 Results and discussion

This section presents and discusses results and experiences gathered during test and demonstration drives during a seven-month test period.

3.1 Statistical data

The following figures provide statistical data concerning the test drives. The test protocol consisted of ten questions: name of the operator, name of the attendant, number of passengers, test route (short or long), test purpose, test audience, weather conditions, road conditions, problems during the drive and applied solutions to the problem. During the test period from April 24th to November 22th a total of 240 test drives were conducted, of which 102 were on the long route and 138 on the short route. The test drives covered a distance of almost 341 km. During these test drives 874 persons were transported. The majority of test drives (70%) were conducted in sunny and dry or slight cloudy conditions. For almost 28% of the test drives it was cloudy respectively very cloudy and rainy. 0.7% of the test drives took place during snowfall. However, test drives and a few rides in heavy rain had to be stopped because of weather conditions. The majority of the test drives (45%) was conducted for demonstration purposes for an external audience. These demonstration drives were held either for company delegations, representatives from road or transport authorities, for the press or for private persons. 38% of the drives were used for operator training, test drives for data collection or test drives without external passengers immediately after the commissioning of the self-driving shuttle. Almost 18% of the trips were conducted for technical tests. Here, for example, optimizations were made on the route guidance, the software was updated, brake tests were made, or the functionality and/or range of the sensors were tested.

3.2 Test experiences

In this section, we summarise the gathered experiences considering the aspects deployment, positioning, automated driving capabilities and interaction with other road users.

3.2.1 Deployment

Before a self-driving shuttle is able to drive autonomously from A to B on a pre-defined route, an extensive analysis and assessment of the driving environment and the driving lane have to be conducted. Furthermore, a digital image of the driving environment (3D LIDAR map) and the driving path has to be created, which is part of the deployment process. For automated driving, beside the accurate 3D map, accurate positioning, environmental recognition as well as automated execution of driving manoeuvres are essential.

Recording and editing of the 3D LIDAR map as well as the driving path are currently conducted by the shuttle supplier based on proprietary models and procedures in the context of a complex, largely manual preparation process. The initial data is recorded with manual drives at a speed of 1 metre per second (using SLAM technology) [11]. These test runs result in a 3D LIDAR point cloud as well as a digital trajectory of the path. Both data have to be manually edited afterwards (removing dynamic objects such as vehicles, bicycles or pedestrians from the point cloud and manual smoothing and correcting the trajectory). Additionally, driving rules have to be manually added (e.g. vehicle speed, priorities or stops). As soon as the 3D LIDAR map as well as the driving path have been prepared, the shuttle is ready for driving the test route for the first time in automatic mode. The manual adaptation process of the virtual path as well as test drives have to be continued until a satisfying quality level has been reached. Since standardised quality evaluation procedures are missing so far, the quality level is judged by human experience (e.g. involving people being responsible for the deployment procedure).

Deployment on a new route is currently a very complex and resource-intensive proprietary process, which has to be conducted by the vehicle supplier. Specific infrastructure requirements, legal restrictions, driving situations and local requirements have to be considered case by case [7]. There is a strong need for a standardised, vendor-independent process for analysing, evaluating and digitising the driving environment based on a standardized tool chain for the (partially) automated creation of the digital driving environment or driving lane.

3.2.2 Positioning

The Navya Arma DL4 shuttle uses optical (LIDAR) and satellite-based positioning (Multi-GNSS-RTK) [17]. For LIDAR-positioning two Velodyne VLP-16 multi-layer 360° LIDAR sensors on the front and back rooftop of the shuttle as well as a pre-recorded LIDAR map of the test track are used. The LIDAR map is recorded by manually driving the pre-defined track at slow speed (~ 1 m/s). Afterwards, the recorded LIDAR data of moving objects (e.g. vehicles, pedestrians, cyclists, parking vehicles) are manually removed from the map. The result is a LIDAR map with fixed reference objects from the vicinity, which is used to position the vehicle on the test track. For Multi-GNSS-RTK positioning, in Koppl a local GNSS reference base for generating the GNSS correction signals was set up on the hill opposite to the test route (Fig. 3) so that the correction signals from the base could be received along the entire route (line-of-sight is beneficial for a reliable signal). The correction signals are transmitted to the shuttle using UHF technology. During operation, the positions from LIDAR and Multi-GNSS-RTK positioning are fused in order to get a more accurate position but also to validate positions against each other. Additionally, odometry and inertial data are used for sensor fusion.

Fig. 3

Mounted GNSS reference base on the opposite hill to the test route

Accurate and reliable positioning of the vehicle along the test route is still a challenge since each of the used technologies has advantages and disadvantages. Optical positioning (cameras, stereo cameras, Lidar) requires visible orientation marks (e.g. road markers) or an optical reference map (e.g. LIDAR). In addition to the elaborate recording and maintenance of these reference data, driving environments with (partly) missing or insufficient road markings (along the test route in Koppl) and (partly) missing reference objects (e.g. along meadows or in tunnels) are problematic. Therefore, optical positioning works only for parts of the test route in Koppl (along the short route in the village centre), while it does not work reliably for the parts with missing reference objects. Multi-GNSS Real-time Kinematics (RTK) is a suitable technology to obtain centimetre-accurate positions using satellite positioning [17]. The prerequisite for centimetre-accurate GNSS-positioning is a reliable correction signal from a near GNSS reference base. This signal is transmitted either via mobile radio (3G/4G) from an Internet-based correction service (e.g. APOS service in Austria) or as in Koppl from a local reference station (e.g. via UHF frequencies) due to missing reliability of the mobile radio connection on parts of the route.

With regard to positioning, the test drives confirmed that LIDAR positioning is reliable in built environments as long as there are fixed objects such as buildings along the route for serving as references. As soon as the built area is left, other positioning approaches have to be used. Camera-based positioning based on road markings is often not possible in rural areas, since these are only poorly recognisable or entirely absent. Additionally, at the time of testing, the Navya Arma DL4 shuttle did not use cameras for positioning or driving tasks. With respect to Multi-GNSS RTK positioning, positioning quality heavily depends on satellite visibility as well as a reliable correction signals. In Koppl the reception of the correction signal was fairly stable however, at some occasions it was lost which resulted in immediate stops of the vehicle. The Arma DL4 version of the shuttle needs at least 14 visible satellites. Reaching such a GNSS coverage is rather challenging for all route sections at any daytime. Especially in situations with bad weather conditions, we faced situations with less than 14 visible satellites. Another challenge is the stable provision of GNSS correction data. The Internet-based service (e.g. APOS in Austria) was not reliable enough for Koppl primarily due to the varying availability and transmission quality of the 3G/4G data connection. The problem could be solved with a local GNSS reference base deployed by Navya Tech. However, it was very challenging to find a suitable location for the GNSS base with line-of-sight visibility along the whole route. In Koppl this could be solved by placing the GNSS base on the opposite hill, which is probably not a feasible solution for other routes. From our experiences, especially for rural areas, we recommend further development and testing of Multi-GNSS RTK positioning since this approach appears to be most promising for reaching high reliability and robustness. Technologies such as Galileo or 5G networks may contribute to more reliable Multi-GNSS RTK positioning.

3.2.3 Automated driving capabilities

The technical design of the Navya Arma DL4 is based on three functional layers: the action layer (1), which manages driving actuators such as steering, power control or braking; the perception layer (2), which controls the sensors for receiving data from the driving environment (e.g. recognising obstacles) and the decision layer (3), which plans and executes driving manoeuvers [13]. The vehicle recognizes the driving environment with eight LIDAR sensors (two Velodyne VLP-16360° multi-layer LIDARs and six 180° mono-layer LIDARs). All driving manoeuvers along the virtual path have to be pre-defined during the deployment phase. This means that the driving speed on different route sections, the priority rules at intersections, and the behaviour before and while turning, etc. must be manually defined. During an automatic drive the shuttle interprets this data and with the data from the LIDAR sensors predicts whether the next manoeuvre on the path can be safely executed or not. If there an obstacle is blocking the path, the shuttle reduces its speed and/or stops. If the obstacle is a static obstacle (e.g. a parking vehicle), the shuttle automatically stops and has to be manually driven around the obstacle in order to continue its automated drive on the path. If the obstacle is a moving obstacle, then either the shuttle waits until the obstacle has left the path or in case the obstacle moves slowly in front of the shuttle (e.g. another vehicle, a cyclist or pedestrian), the shuttle adapts its speed and follows the obstacle at reasonable distance. If the pre-defined path may not be followed due to road anomalies (e.g. construction works), the shuttle has to be either operated in manual mode for passing the anomaly or if the anomaly occurs for a longer time period, the pre-defined path has to be manually edited so that the road anomaly can be automatically circumvented.

With respect to environmental detection, it has been found that the detection of static obstacles generally works quite well and the self-driving shuttle stops reliably in front of the obstacle. Problems arise from dead angles, which prevent a reliable 360° detection of obstacles. This is mainly a problem of the unlucky position of the 360° LIDARs on the roof top (edges of the roof shadow the sensors to the side) and can be easily solved in the future. Other problems come from a too low spatial resolution of the used Velodyne VLP-16 LIDAR sensors. The 16 layers of these sensors are focused on the direct surroundings before and behind the vehicle and miss to reliably detect more distant obstacles, especially when these obstacles are approaching at higher speeds (> 30 km/h). This problem can be solved with higher resolution LIDAR sensors or with additional sensors such as RADAR sensors or cameras. Another problem occurred from LIDAR reflections caused, for example, by water lacquers on the road, heavy rain or snowfall, being misinterpreted by the shuttle as an obstacle on the road. At present, the shuttle is not able to classify objects which limits scene interpretation.

With respect to automated driving capabilities, the test drives revealed that the actual performance lags significantly behind the expectations (Table 1). Although the manufacturer claims that the Navya Arma DL4 shuttle is the first self-driving vehicle satisfying SAE J3016 level 5 (“full automation”) [15], based on the experiences gained in Koppl with respect to public roads with mixed traffic we classify the shuttle at maximum level 3 (“conditional automation”), in some specific situations only level 2 (“partial automation”). The shuttle currently can autonomously handle only very simple manoeuvres at low speed. The shuttle is able to stop reliably in front of obstacles that appear in front of the vehicle. The vehicle is also able to react upon moving obstacles such as other vehicles, but only if they are moving at slow speed (< 30 km/h). Higher speeds cannot be handled adequately and need manual intervention. In addition, the shuttle is not able to automatically circumvent obstacles (e.g. parking vehicles) or overtake other road users, such as cyclists.

Table 1 Summary of observed vehicle behaviours, possible reasons, applied solutions and occurrences

With the test drives, it has been shown that the self-driving shuttle is by far not yet capable of autonomously executing all the required driving manoeuvers along the route in mixed traffic. In Koppl, the following driving manoeuvers were tested: turn right into a priority road, turn left on a priority road, compliance with priority and stop signs, entering and exiting bus stops and interaction with other road users (trucks, cars, buses, bicycles and pedestrians). For the majority of the driving manoeuvers the human operator has to supervise the vehicle’s behaviour in order to be prepared for immediate intervention. The shuttle is only capable of handling simple, pre-defined manoeuvers. For example, the vehicle in most cases stops reliably in front of obstacles but cannot pass them, being expectable at least from a level 4 vehicle. Instead, a human operator is responsible to take over manual control for passing the obstacle. Another example are left-turns on priority roads: Since the shuttle cannot automatically handle these situations, planned safety stops were used for security reasons. In case of a planned safety stop, the operator has to manually trigger the left turn when it is safe to turn. The same interaction is necessary in case of exiting a bus stop or entering a priority road from a side road. From the test drives, it has also become very clear, that the SAE classification heavily depends on the driving environment. A classification level being reached in restricted areas is not possible on public roads in mixed traffic. The same is true for urban vs. rural areas or motorways vs. local roads. Overall, it is challenging to do a meaningful classification for the new vehicle category of self-driving shuttles. An appropriate classification has to be developed in the future.

3.2.4 Interaction with other road users

One of the major challenges of testing self-driving vehicles in mixed traffic arises from the interaction with other road users. In some situations, it is not clear what the vehicle will do next and how other road users should behave. For example, the shuttle signals a stop via a display on the back windshield. However, does this mean for the other road users that it is safe to overtake the shuttle or should they also stop behind the shuttle? Another example is that other road users abandon their priority that confuses the shuttle and thus leads to pat-situations where neither the shuttle nor the other vehicle move on. In some situations, other road users were not aware whether the shuttle had recognised them as an obstacle and if they can continue their drive or if they should stop. Right now, these kinds of questions are completely open and standards for the interaction with other road users are missing.

3.3 Passenger feedback

Passenger feedback was gathered via an online survey on a smartphone, directly after each test drive. In total 294 passengers participated in the survey.³ Results of the passenger survey have not been separately analysed for different test groups.

Regarding to the prior knowledge of automated driving, 13% of passengers said they did not have any knowledge about automated vehicles. Nearly 43% had already heard about automated driving, just over 44% said that they have already dealt with the topic. This high value arises from the fact that numerous company delegations with relevant knowledge in this field participated in the test drives. When asked about previous experiences with automated shuttles, 85% of the passengers answered that they had never used an automated shuttle before. 9% of the passengers already experienced a ride with another automated vehicle and just over 5% of the passengers repeatedly took part in a ride with the Digibus©. Figure 4 shows the share of mentioned reasons for the test rides as well as the expected purpose of use of a self-driving shuttle.

Fig. 4

Reasons for the test ride and expected purpose of use

Figure 4 shows the share of passengers liking or disliking the ride as well as feeling safe or unsafe. Just over 92% of the passengers enjoyed the ride with the Digibus© very well or well. According to given statements (selection of statements) from passengers, they particularly liked “the good detection of obstacles”, “the smooth and quiet driving behaviour”, “the advanced development of the technology” or “the design of the shuttle”. Just over 6% of the passengers said that they liked the ride not so much and only 1% of the passengers disliked the ride. The reasons were, for example, “lack of driving comfort”, “the high braking intensity” or “feeling of unsafety”. When we asked the passengers what they found surprising (in a positive and negative sense), we got the following answers: Passengers were positively surprised by the “comfortable and safe driving experience”, “the simple way of user interaction”, and “the good state of development of the technology”. Passengers reported negative surprises with respect to the “abrupt braking behaviour of the self-driving shuttle”, “the multiple manual interventions and restarting of the shuttles” and that “there is no fully automatic detection of the route, but it must be programmed manually”.

The passenger survey also revealed high positive values for a good feeling of safety on-board (Fig. 5). However, it has to be mentioned, that some passengers only felt safe because an operator was on board. The assumption is that the passengers’ sense of safety decreases if the shuttle drives completely driverless. When the passengers were asked for the reasons why they did not feel safe on board, the answers were “abrupt or jerky braking”, “not enough confidence in this new technology”, “lack of experience”, “poor sensor technology” or “the shuttle is not able to differentiate between people and vehicles”.

Fig. 5

How many passengers liked or disliked the ride and felt safe or unsafe

Figure 6 shows the passenger demographics, in particular age distribution and employment. 56% of the passengers were male and 41% female. Children under 3 years of age were excluded from transport due to safety reasons because appropriate safety systems for children are missing. 78% of the respondents stated that they possess a privately owned car, while just under 21% did not own a car. When the passengers were asked if they could imagine a self-driving shuttle replacing their privately owned car or their second car, almost 59% of the respondents negated this question. However, 40% could imagine that a self-driving shuttle replaces their first or second privately owned car. Wishes and suggestions of the passengers refer to “an extension of the route”, “faster speed” and “more comfort in the shuttle”.

Fig. 6

Passenger demographics with respect to age distribution and employment status

4 Conclusions

Although new demonstration and experimental projects with self-driving shuttles are announced almost every month, the test drives on public roads in mixed traffic showed that the tested self-driving shuttle currently does not fulfil the expectations of highly or fully automated vehicles. The vehicle may still be considered as kind of prototype at research and development stage. According to the ERTRAC Automated Driving Roadmap 2017, the introduction of highly automated cars is planned for the period 2018–2024 on dedicated routes and for the period 2024–2030 on public roads in mixed traffic. Until then, an enormous amount of research and development is necessary. Among others, the following fields of research have to be prioritised for the next years:

• Standardised and (partly) automated processes for digitising the driving environment: The current practice of proprietary and mostly manual setup of the digital driving environment has to be improved. Standardised tool chains for the digital modelling of the environment have to be established very soon.

• Systematic testing of driving scenarios: A systematic testing of driving scenarios especially with the respect to varying driving environments for autonomous shuttles is missing so far. Most of the testing consists of trial and error. A standardised and integrated process from simulation to system tests to closed test environments and open road testing including feedback loops on each stage is necessary.

• Testing on public roads: Since first regulations for testing of autonomous vehicles have passed legislations in European countries, these should be used to include public road testing in the research and development process (integrated with the previously mentioned process). However, public road testing should be accomplished following well-defined and transparent test procedures and results fostering improved learning for all involved stakeholders (e.g. vehicle manufacturers, technology providers, public transport companies and associations, public authorities). This transparent process should also include publication of test results.

• Role of digital infrastructure: Until now, we have only poor experiences which role digital infrastructure (3G/4G/5G networks, ITS-enabled traffic lights, vehicle-to-infrastructure communication,) can or should play in the context of automated vehicles. In future trials we should put more attention on how the vehicle should/could interact with such an infrastructure for improved reliability.

• Realistic environmental conditions: Most of the current tests are conducted in city centres at sunny weather conditions (also the majority of our test drives was at sunny weather). However, regular operation of automated shuttles also means operation during the winter season or on rainy days. Other aspects are rural areas or steepness of the route. Further testing has to pay more attention to such aspects.

• Interaction with other road users: Interaction with other road users has turned out to be one of the most challenging issues during public road testing. While research of the last years has predominately focused on the role of human drivers in the context of automated vehicles, the aspect of how other road users interact with automated vehicles has widely been neglected. In mixed traffic, a lot of traffic situations occur which cannot be adequately solved without human interaction. From our experiences, this topic is one of the key issues for safe and reliable operation and has to be immediately addressed.

• Passenger safety: Until now, there is little knowledge how passengers feel during driverless operation. Results from our trial as well as from others indicate that passengers feel safe with an operator on board [5]. However, how does this feeling change in driverless operation? Which kind of passenger interaction do we need for reaching the same or higher levels of trust in comparison to driver-operated buses?

• Design of automated mobility systems: In the future, we should pay attention to how automated vehicles can fit seamlessly into an existing (public) transport system or how we have to change these systems in order to cope with the new requirements. The test-drives in Koppl revealed, that at the current stage of development it is really hard to integrate an autonomous shuttle in existing public transport systems since the technology is by far not major enough. The main feedback from a lead user workshop with transport operators, village representatives and potential passengers was that everybody confirms the huge potential of the technology, but to be of use for daily operation, the technology has to work stable under all possible conditions. Passengers do not mind whether the shuttle is operated in automatic mode or not, they expect a reliable public transport link to their destination. However, currently, there are too many situations, which the shuttle cannot handle automatically and any manual intervention typically leads to service interruptions contradicting the idea of operating the shuttle as a reliable feeder to a major public transport line. Moreover, handling bad weather or winter conditions adequately is also a pre-requisite before going into daily operation. Especially for first/last mile scenarios, several open questions have to be addressed properly. Among others, these questions are how to guarantee smooth interlinking with other public transport lines, how to handle peak times where the demand exceeds the capacity of the shuttle, how to reach similar passenger safety and trust in comparison to driver-operated public transport or how to validate safe and reliable operations of an automated shuttle. As far as technical, legal and social questions are not fully answered, any automated shuttle operation has to be declared as experimental. It is essential that passengers are aware of the experimental stage for having realistic expectations and being aware of further research and development steps.

The results, which were gained during the test drives in Koppl laid the ground for further research and development activities such as the Austrian 3-years automated driving flagship project “Digibus® Austria”.⁴ Together with 12 research and development partners, the project coordinator Salzburg Research pursues the goal to research and test methods, technologies and models for a reliable and traffic-safe operation of automated shuttles on open roads in mixed regional traffic environments. Along with two other sites, Koppl was again selected test site.