Evaluation of innovative ideas for Public Transport proposed by citizens using Multi-Criteria Decision Analysis (MCDA)

Abstract

Introduction

The use of participatory techniques in the field of transport is coming to the forefront recently. In this frame, eight co-creation workshops and five online crowdsourcing campaigns took place in Thessaloniki, Southern Tuscany, Rotterdam/The Hague, and Frankfurt, from which many innovative ideas to enhance Public Transport were generated by citizens.

Purpose

A simple list of innovations would not be very useful for Public Transport Operators, as they cannot implement all of them at once. There was an obvious need for their ranking and this is the purpose of this paper.

Methods

The ranking was realized with the most used Multi-Criteria Decision Analysis method in transportation research, i.e. the Analytic Hierarchy Process, using three criteria: Feasibility, Utility, and Innovativeness. An online questionnaire was distributed to experts, using a modified snowball sampling technique, which yielded 97 completed questionnaires.

Results

Utility (42.90%) was found to be the most important criterion, followed by Feasibility (40.10%), and Innovativeness (17.00%). Four lists of innovations were derived, ranked with respect to a) all three examined criteria, b) Feasibility, c) Utility, and d) Innovativeness. The highest ranked innovation for a) and c) was found to be Mobility as a Service and platform with real-time travel, comfort, and multi-modal information; for b) City marketing from a Public Transport perspective; and for d) Advanced e-ticketing system.

Conclusion

The results revealed which of the innovations are the most promising and provide valuable insight into how to integrate innovation with Public Transport to make it more attractive. Public Transport Operators may use the results according to the peculiarities of their city and the importance they give to Feasibility, Utility, and Innovativeness.

1 Introduction

The growing urban population worldwide and the traffic congestion resulted from the increase of car ownership during the last decades are urging for a shift to more sustainable and collective transport modes. In order Public Transport (PT) to increase its modal share, it has to attract new passengers and satisfy adequately the changing needs of the current ones, by introducing innovations and new services. Huge investments are required for this purpose and Public Transport Operators (PTO) are consequently dealing with the need of hierarchizing their priorities, focusing more intensely on the needs of their customers.

The use of participatory techniques in the field of transport is coming to the forefront recently, following a general trend to shift towards a more client-centered approach and put emphasis on services, business models, and innovation. Among the most widespread approaches to ensure public participation in decision-making and products and services design are: co-creation and crowdsourcing. “Collective Innovation for Public Transport in European Cities” (CIPTEC) is a European Union (EU) Horizon 2020 research project, addressing the challenge for “Smart, Green and Integrated Transport” and a CIVITAS knowledge generating project in the area of “Tackling urban congestion” [1]. The project started in May 2015 and finished in April 2018. In the context of CIPTEC project both approaches were implemented, under the objective of identifying new innovative ideas and concepts to enhance Public Transport use in European cities by making it more attractive.

CIPTEC uses an integrated approach which attempts to combine ideas from marketing (customer orientation, marketing research, consumer intelligence, etc.), consumer behavior (advanced motivational research, behavioral experimentation, etc.), innovation (crowdsourcing, collective intelligence, co-creation and co-design of new ideas, fusion of business concepts with social innovation, etc.), evaluation (socioeconomic, technological, and ethical assessment), and co-exploitation within a wide stakeholder platform attacking the challenges that do not favor the growth of Public Transport modal share.

Since the spring 2016, CIPTEC has been exploring collective intelligence potential through the co-creation of new emerging ideas that might spark new solutions in the field of urban transport by a bottom-up approach. In this context, new innovative solutions were suggested and evaluated with the aid of eight (8) co-creation workshops, which were organized between May and December 2016 in four (4) urban areas (viz. Thessaloniki, Greece; Southern Tuscany, Italy; Rotterdam/The Hague, the Netherlands; and Frankfurt, Germany). The overall scope of CIPTEC co-creation workshops was to generate innovative concepts, both for services and products that could be integrated into Public Transport and contribute to its attractiveness and market share.

Moreover, five (5) different crowdsourcing campaigns were designed and ran online. Four (4) of them focused on specific European cities/regions (viz. Thessaloniki, Greece; Southern Tuscany, Italy; Rotterdam/The Hague, the Netherlands; and Frankfurt, Germany) in the respective national languages, while the fifth one was addressed to all European citizens, regardless the city they live in, and it was launched in English. Citizens from all European countries were given the opportunity to submit their innovative ideas in order to make Public Transport more attractive. The submitted ideas were disseminated, discussed, improved, and evaluated by other users of each platform, and by experts, and the best ideas were rewarded.

In the frame of CIPTEC, with a focus on Public Transport, the meaning of the term “innovation” was extended to include “every idea coming from other fields, and it is currently not applied in Public Transport or even any idea that, while it has been conceived and may already been implemented in some areas of the world, it has not been yet spread or adopted in a certain local or/and national context” [1].

In an attempt to find compromising solutions to meet the different, and often conflicting, objectives of the multiple actors involved in any decision-making process, flexible methods and tools are needed. Under the objective of evaluating the output of the aforementioned collective intelligence processes that were applied in the context of CIPTEC, Multi-Criteria Decision Analysis (MCDA) methods were employed. MCDA methods have gained wide acceptance due to the fact that they embody many quantitative and qualitative variables and the characteristics of several scenarios can be assessed simultaneously [2]. The use of MCDA in the field of transportation has shown a steadily increasing trend during the last years. There is no single way for conducting a MCDA; there is a large number of methods and techniques with the aid of which it could take place. The suitability of the method strongly depends on the research objectives and the scope of the analysis [3].

The MCDA method used in the frame of CIPTEC project is the Analytic Hierarchy Process (AHP), one of the most commonly MCDA methods used in the field transportation research. The main reason for that is that through pairwise comparisons of the selection criteria and the alternatives, AHP leads to an, as far as possible, unbiased hierarchy of the alternatives (i.e. of the suggested innovations in the frame of CIPTEC). A hierarchy of the alternatives is always useful for the decision-maker especially in cases such as Public Transport where PTOs cannot implement all the suggested innovations at once but, instead, they should focus on the most important (i.e. on those with the highest impact).

The remainder of the paper is structured as follows: the literature review is presented in Section 2, followed by the description of the methodological framework in Section 3 and the survey design in Section 4. After that, the main findings of the analysis of the survey are displayed and discussed in Section 5. The paper concludes in Section 6, with a critical overall analysis of the results and directions for future research.

2 Literature review

Two main participatory techniques were used to collect innovative ideas in the field of Public Transport: co-creation workshops and crowdsourcing campaigns.

Co-creation may “refer to any act of collective creativity, i.e. creativity that is shared by two or more people” [4]. More specifically, co-creation is a revolutionary user-centered, collaborative approach, where a multitude of stakeholders (e.g. users, professionals, firms, etc.) are involved in the process of designing a product or service aiming to jointly create value [5]. Co-creation is an active, continually changing process, as it involves interactions between the relevant stakeholders [6]. The most important aspect of co-creation methods is the fact that they are user-centered. When the users are involved in the design of a product or service, the end value is usually increased due to the adaptation of the product/service to the users’ needs. Moreover, among the most important benefits of the application of co-creation methods are the following: better and more effective decision making, reduced costs by being in line with the users’ requirements, increased product quality, competitive advantage, customized products/services, and better customer needs satisfaction [7]. Considering the fact that the most users of Public Transport are loyal and captive, the inclusion of non-users in the co-creation process is desired, whenever insights on how they would shift to Public Transport are required. Despite the fact that co-creation processes were implemented in the field of urban planning during the past decades, their expansion in the Public Transport sector was not as widespread as it could be. Actually, it seems that the most cited relevant paper is not even about urban Public Transport but about the Swiss Federal Railway [8]. However, although PTOs have been using the traditional marketing approaches since years ago, new user-centered approaches should be investigated. As a consequence, new marketing approaches, like co-creation workshops can contribute to the increase of Public Transport attractiveness enabling the identification of innovative concepts/ideas that could be further developed by the experts [9]. Perhaps the best way to apply co-creation methods are co-creation workshops in which users, and sometimes non-users like in the case of Public Transport, participate. Eight (8) such co-creation workshops took place in the frame of CIPTEC project, two (2) in each one of the following cities: Thessaloniki, Greece; Southern Tuscany, Italy; Rotterdam/The Hague, the Netherlands; and Frankfurt, Germany. The co-creation methods applied in these workshops and their results have been extensively presented in a relevant paper [10]. From these workshops came some of the innovative ideas that are evaluated in the frame of this paper.

Crowdsourcing campaigns were also used by CIPTEC to gather innovative ideas for Public Transport [11]. The term “crowdsourcing” was coined by Howe in 2006 who claimed that companies were using Internet outsourcing, thus the work to the crowd [12]. Crowdsourcing can be better understood as a field that combines three key elements: the “crowd”, outsourcing, and advanced Internet technologies [13]. In 2008, Howe suggested the following definition: “Crowdsourcing is the act of taking a job traditionally performed by a designated agent (usually an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call” [14]. Although the concept of crowdsourcing for innovation is relatively simple, it has been enabled during the last few years through the vast penetration of Internet technologies and smartphones in the market, but still its relevant literature is somehow limited. Most of the crowdsourcing for innovation examples are related to private sector product design and innovation process (e.g. Ford New Innovative Ideas [15] and IdeaConnection [16]). In many industry sectors (e.g. manufacturing, transport/warehousing, accommodation and food services, public administration, etc.) crowdsourcing was applied by big brands [17], however the examples of using crowdsourcing in Public Transport are not as many as in other domains. An indicative example is the “Bombardier Transportation”, the rail equipment division of the Canadian firm Bombardier Inc., which has held three crowdsourcing initiatives: Innovation Express (internally for employees), YouRail (external contest), and YouCity (innovation contest related to the future of urban mobility) [17]. Another interesting crowdsourcing contest initiative, for developing innovative ideas in order to improve Public Transport in the South Holland region, is the Slim OV-idee [18]. Nevertheless, crowdsourcing for innovative ideas regarding Public Transport sometimes is incorporated in smart cities crowdsourcing events (e.g. hackathons) or platforms. Through crowdsourcing, innovative ideas can be gathered faster and cheaper from a bottom-up and more democratic process than internally within an organization that creates certain thinking patterns. After comparing different ideas generated both by the crowd and professionals, the ideas of the professionals proved to be less innovative and had less benefit for the end-user; on the other hand, they were more feasible (i.e. implementable) [19, 20]. In addition, ideas generated by the crowd can be more commercially successful compared to ideas generated internally in an organization [21]. Although Public Transport affects a large and loyal community of people in any city and its services gain the interest of public, crowdsourcing for innovative ideas has not been widely used so far [11], apart from some exceptions [22, 23]. Most research on Public Transport and crowdsourcing is about information provision crowdsourcing for use in real-time mobile applications and not about gathering innovative ideas to be integrated into the Public Transport system. Five (5) different online crowdsourcing campaigns were designed and ran in the frame of CIPTEC project, four (4) in the aforementioned cities in the respective national languages while the fifth one was addressed to all European citizens regardless the city they live in and it was launched in English [11]. The co-creation methods applied in these crowdsourcing campaigns and their results have been extensively presented in a relevant paper [11].

Both the co-creation workshops [10] and the crowdsourcing campaigns [11] were fruitful and many innovative ideas for Public Transport were gathered using these collective intelligence participatory techniques. Obviously, there was a need to evaluate and hierarchize these innovative ideas and this is the purpose of this paper. Since the innovative ideas we gathered are abstract ideas and not projects to be realized, conventional evaluation tools such as Cost Benefit Analysis (CBA) or Cost-Effectiveness Analysis (CEA) were not applicable, as they “are constrained by the difficulty in quantifying non-market impacts and monetising total costs and benefits” [24]. Moreover, MCDA methods are increasingly used due to the complexity of issues and the inadequacy of conventional tools, such as CBA and CEA, and it “allows for participatory analysis and qualitative assessment” [24]. Macharis and Bernardini (2015), in one of the most comprehensive reviews of the use of MCDA methods for the evaluation of transport projects, retrieved 276 publication titles in the period 1985–2012 that cover a broad scope of MCDA applications on transport projects [25]. Their analysis clearly shows that the use of MCDA methods on transport projects increases, especially after 1990. During that period, the transport projects examined were “for 22% passenger transport oriented, for 15% freight transport focused, for 11% transport technology and for 52% of ‘general’ type” [25]. A closer to per transport mode classification showed that “mobility management has the highest application (22%) after the general group (25%) followed by infrastructure (11%), Public Transport (10%) and technology (10%). Minor applications can be seen for the sub-categories: Bike (1%), Rail (3%), Air (5%) and as well 5% for Water (Inland Waterways and Maritime transport)” [25]. Since our focus is on Public Transport, there were publications focusing on: automatic metro map layout using multi-criteria optimization [26], on AHP-based decision making process for construction of public transportation city model: case study of Jeju, Korea [27], on spatial multiple criteria decision analysis in integrated planning for Public Transport and land use development in Klang Valley, Malaysia [28], on a multi-criteria evaluation model of Public Transport networks [29], and on a multi-stakeholder multi-criteria assessment framework of mobile payments illustrating the Swiss Public Transportation Industry [30]. Since 2012, and even 2015 when the review [25] was published, more papers on MCDA methods applied on transport projects have been published, e.g. [31, 32], but as far as we know there was no paper applying a MCDA method on innovative ideas for Public Transport.

The final question is about which MCDA method should we use? There are many MCDA methods applied in transportation projects. Broadly, the MCDA methods are classified as follows: a) priority or utility methods, such as the Multi-Attribute Utility Theory / Multi-Attribute Value Theory (MAUT/MAVT) and the Simple Multi-Attribute Rating Technique (SMART); b) outranking methods such as the Elimination and Choice Expressing Reality, or “ELimination Et Choix Traduisant la REalité” (ELECTRE) in French, and the Preference Ranking Organization Method for Enrichment of Evaluations (PROMETHEE); c) distance based methods such as the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) and the Multi-criteria Optimization and Compromise Solution, or Višekriterijumsko Kompromisno Rangiranje (VIKOR) in Serbian; d) mixed and other methods such as the linear additive method, the Analytic Hierarchy Process (AHP), the Likert scale, the fuzzy-TOPSIS, the fuzzy-VIKOR, etc. [31]. Actually, there are more than a hundred MCDA methods [25]! Different researchers have used different multi-criteria analysis methods for combining the value of performance parameters [31]. Mardani et al. (2015) showed that among the different MCDA methods (100%), AHP (32.57%), hybrid Multiple-Criteria Decision-Making (MCDM) (16.28%), aggregation Decision-Making (DM) methods (11.70%), TOPSIS (11.4%), ELECTRE (8.65%), PROMETHEE (6.62%), and VIKOR (3.56%) were applied in different kinds of MCDA problems between 2000 and 2014 [33]. Macharis and Bernardini (2015) found that the MCDA method mostly used in transport projects is the AHP method, as from the total amount of 276 publications they examined during the period 1985–2012, “33% revolves around the application of AHP or develops a variant to this approach applicable in the transport project field” [25]. Mardani, Zavadskas, Khalifah, Jusoh, and Nor (2016) reviewed a total of 89 papers, published from 1993 to 2015, from 39 high-ranking journals and found similar results: “AHP and Fuzzy-AHP techniques (25.84%) and its applications have been used more than other individual approaches” in transportation systems [34]. Obviously, AHP is the most commonly used MCDA method in the field of transport.

3 Methodological framework

From the large spectrum of the existing MCDA methods, the AHP method was selected for the evaluation framework. AHP is one of the most commonly MCDA methods used in the field of transportation research. According to the aforementioned recent comprehensive literature review by Macharis and Bernardini [25] regarding the use of MCDA methods on transportation related projects between 1985 and 2012, AHP was applied in more than 1/3 of the examined case studies. AHP is a non-linear framework for addressing complex semi-structured decision-making problems, introduced by Thomas Saaty in the 1970s [35]. It is a scaling method for deriving priorities (weights) for a set of options/scenarios based on their importance. The method uses a hierarchical or network structure to represent the problem in question and then the relations within this structure are built using pairwise comparisons [36]. AHP relies on three main principles: a) the decomposition of the decision space into its fundamental elements, b) the comparative judgments, and c) the synthesis of priorities [37].

The hierarchies of objectives and sub-objectives that are created attempt to reflect and incorporate the knowledge, experience, and aspirations of the decision-makers regarding the problem in discussion [38]. Comparative judgments principle refers to the development of a sound and comprehensive base for deriving priorities among the parameters involved in the process. Local priorities are obtained by comparing each node against each one of its peers with respect to its parent node.

In order to assist the decision-makers in carrying out the pairwise comparisons, Saaty created a nine-point intensity scale of importance, known as the fundamental scale of preferences, ranging from equal importance (1) to extreme importance (9). Technically, this is achieved by forming pairwise comparison matrices, where the ratio a_ij, assigned by the decision-maker, expresses the dominance relation of the element in row i over the element in column j. These preference scores are then subject to a synthesis process; relative priorities (weights) are attained as the right principal eigenvector λ_max of the pairwise comparison matrix. In cases when the transitive property holds, the aforementioned matrix is consistent; however, in real-life problems this is a very rare and unlikely thing to happen. Therefore, AHP suggests a specialized index, the Consistency Ratio (CR), to investigate the existence and degree of inconsistency of the judgments. In case CR is equal or lower of 10% the inconsistency is considered “acceptable”, in the sense that its influence on the results is negligible or at least not of significant importance [36, 39].

A mathematic approach of how the AHP method works follows, based on a comprehensive AHP note [40], which is based on Saaty’s introductory book to the AHP method [35]. More specifically, the AHP method is implemented in three steps: 1) Computing the vector of criteria weights, 2) Computing the matrix of alternatives’ scores, and 3) Ranking the alternatives. Let m be the number of the evaluation criteria and n the number of the alternatives to be evaluated [40].

In the first step, in order to compute the weights for the criteria, the AHP starts creating a pairwise comparison matrix A, which is an m × m real matrix, where m is the number of the evaluation criteria. Each entry a_jk of the matrix A represents the importance of the jth criterion relative to the kth criterion. Obviously, if a_jk > 1 the jth criterion is more important than the kth criterion, while if a_jk < 1 the jth criterion is less important than the kth criterion. In case two criteria have the same importance, then the entry a_jk = 1. The entries a_jk and a_kj satisfy the following constraint [40]:

$$ {a}_{jk}\cdotp {a}_{kj}=1 $$

(1)

and, obviously a_jj = 1 for all j. The relative importance between two criteria is measured by the aforementioned fundamental scale of preferences, as shown in Table 1, where it is assumed that the jth criterion is equally or more important than the kth criterion. The values in the matrix A are by construction pairwise consistent, see (1), but the ratings may show slight inconsistencies which do not cause serious difficulties [40].

Table 1 Saaty’s fundamental scale (value of a_jk) [40]

From the matrix A, once it is built, it is possible to derive the normalized pairwise comparison matrix A_norm by making equal to 1 the sum of the entries on each column. Each entry $\overline{a_{jk}}$ of the matrix A_norm is calculated as [40]:

$$ \overline{a_{jk}}=\frac{a_{jk}}{\sum \limits_{l=1}^m{a}_{lk}} $$

(2)

Finally, the criteria weight vector w (which is an m-dimensional column vector) is built by averaging the entries on each row of the matrix A_norm, i.e. [40]:

$$ {w}_j=\frac{\sum \limits_{l=1}^m\overline{a_{jl}}}{m} $$

(3)

In the second step, the computing of the alternatives’ scores, which is an n × m real matrix S, takes place. Each entry s_ij of the matrix S represents the score of the ith alternative with respect to the jth criterion. In order to derive the scores, a pairwise comparison matrix B^(j) is built for each of the m criteria, j = 1, … , m. The matrix B^(j) is an n × n real matrix, where n is the number of the alternatives evaluated. Each entry ${b}_{ih}^{(j)}$ of the matrix B^(j) represents the evaluation of the ith alternative compared to the hth alternative with respect to the jth criterion. If ${b}_{ih}^{(j)}>1$, then the ith alternative is better than the hth alternative, while if ${b}_{ih}^{(j)}<1$, then the ith alternative is worse than the hth alternative. If two alternatives are evaluated as equivalent with respect to the jth criterion, then the entry ${b}_{ih}^{(j)}=1$. The entries ${b}_{ih}^{(j)}$ and ${b}_{hi}^{(j)}$ satisfy the following constraint [40]:

$$ {b}_{ih}^{(j)}\cdotp {b}_{hi}^{(j)}=1 $$

(4)

and ${b}_{ii}^{(j)}=1$ for all i. An evaluation scale similar to the one introduced in Table 1 may be used to translate the pairwise evaluations into numbers. Following, the AHP applies to each matrix B^(j) the same two-step procedure described for the pairwise comparison matrix A, i.e. it divides each entry by the sum of the entries in the same column, and then it averages the entries on each row, obtaining thus the score vectors s^(j), j = 1, … , m. The vector s^(j) contains the scores of the evaluated alternatives with respect to the jth criterion. Finally, the score matrix S is obtained as:

$$ S=\left[\ {s}^{(1)}\dots {s}^{(m)}\right] $$

(5)

i.e. the jth column of S corresponds to s^(j) [40].

In the third and final step, once the weight vector w and the score matrix S have been computed, the AHP obtains a vector v of global scores by multiplying S and w [40]:

$$ v=S\cdotp w $$

(6)

The ith entry v_i of v represents the global score assigned by the AHP to the ith alternative. As the final step, the alternatives ranking is accomplished by ordering the global scores in decreasing order [40].

Regarding the aforementioned consistency, the AHP method incorporates an effective technique for checking the consistency of the evaluations made by the decision maker when building each of the pairwise comparison matrices involved in the process, i.e. the matrix A and the matrices B^(j). This technique relies on the computation of a suitable Consistency Index (CI), and will be described only for the matrix A. It is easy to adapt it to the case of the matrices B^(j) by replacing A with B^(j), w with s^(j), and m with n. The CI is obtained by computing the scalar x as the average of the elements of the vector whose jth element is the ratio of the jth element of the vector A · w to the corresponding element of the vector w and then the CI is calculated by the following equation [40]:

$$ CI=\frac{x-m}{m-1} $$

(7)

A perfectly consistent decision maker should always obtain CI = 0, but small values of inconsistency may be tolerated. More specifically, if:

$$ CR=\frac{CI}{RI}<0.1 $$

(8)

the inconsistencies are tolerable, and a reliable result may be expected from the AHP. In (8), RI is the Random Index (RI), i.e. the CI when the entries of A are completely random [40]. The values of RI are related to the number of m and may be found in the relevant literature.

AHP is a well-established procedure that can be repeated; it allows performing consistency checks, both quantitative and qualitative data can be used as input and it is appropriate for group decision making. Moreover, as mentioned previously, it is applied widely in the academic literature. An additional advantage of AHP is that it tends to highlight slight differences among the examined alternatives which decision-makers are not always able to comfortably perceive [41, 42]. For all these reasons, AHP is considered a smart choice when multi-dimensional decision problems are examined.

4 Survey design

In order to evaluate the collective intelligence processes results, derived, as discussed above, from the CIPTEC co-creation workshops and the crowdsourcing campaigns, an online questionnaire survey was designed and conducted. Twenty (20) innovations, comprising a combination of the innovations which were shortlisted as the most promising in each of the aforementioned cases, were included in this survey to be further assessed against a number of additional criteria. The 20 innovations (abbreviations, full names, and descriptions) are presented in Table 2.

Table 2 The innovations that were included in the evaluation questionnaire (abbreviations, full names, and descriptions)

The selection of a proper set of criteria is a very important step towards designing a comprehensive and trustworthy MCDA evaluation framework. The European Institute of Innovation & Technology (EIT) had launched a call for Knowledge and Innovation Communities 2018 and evaluation criteria such as: strategic approach, business model and financial plan, synergies, impact on the societal challenge, human capital, job creation, and economic growth were used [43]. Balm et al. [44] evaluated innovative urban and interurban freight transport solutions based on: costs and benefits to society, integration of stakeholders’ opinions, and financial viability. According to the so-called “FAN” method, from the Synectics problem-solving tool, a recommended set of criteria to evaluate any innovative idea should include feasibility, attractiveness, and novelty [45]. Similarly, Terwilliger [46] suggested the Strategy, Need, Impact, Feasibility, Feel (SNIFF) test for innovation decision-making. Schöllhammer [47] believes that new, creative ideas should be evaluated with respect to how feasible, effective, efficient, and profitable they are.

It was decided to keep the number of criteria low and to include criteria with a rather generic nature than more specific ones, in order to keep the questionnaire as simple as possible, ensuring, thus, that it would appear more “attractive” to the survey participants. This way, we avoided the creation of a complex survey which could involve the risk of discouraging people to complete it, considering it time-consuming. After considering possible combinations of evaluation criteria based on the relevant literature and the limitations of the AHP method, three (3) main criteria were selected as the most suitable for this case: utility, feasibility, and innovativeness.

The AHP questionnaire was distributed online by the twelve (12) CIPTEC project partners from seven (7) different European countries to experts, through e-mails to their contact lists and through thematic social-media groups (i.e. using a modified snowball sampling technique), in order to collect their views about the innovative concepts that had emerged both from the co-creation workshops and the crowdsourcing campaigns. The online survey took place in June 2017 and yielded 97 fully completed questionnaires, while the respondents were experts of several European countries. The original questionnaire (Fig. 1) was prepared in the English language, however translation in four (4) other languages was provided (viz. Greek, Italian, Dutch, and German, corresponding to the national languages of the countries where the co-creation workshops took place), in an attempt to make people more eager to participate.

Fig. 1

Indicative cut screenshot of the online questionnaire: pairwise comparisons of the AHP criteria

The sample consisted of 20 women (20.62%) and 77 men (79.38%). The majority of the participants (i.e. 49 persons or 50.52%) were between 45 and 64 years old, while the age group 25–44 followed with 34 persons (35.05%). Only three participants were between 18 and 24 years old (3.09%) and the rest of the sample (i.e. 11 persons or 11.34%) were 65+ years old. There was also a question regarding the sector in which they work and the results were the following: Public Transport: 32 persons (32.99%), Business industry (freelancer, private company, chamber): 18 persons (18.56%), Academia (educational/research institution): 15 persons (15.46%), Government (national, regional or local authority): 11 persons (11.34%), Other: 7 persons (7.22%), and Civil Society (not-for-profit trust, charity, non-governmental organization): 5 persons (5.15%). There were also 9 persons (9.28%) who did not answer this question.

5 Results and discussion

After the collection of the 97 questionnaires, the next step was to aggregate the different answers by calculating the rounded up geometric means. The aggregated pairwise comparison matrix, which was used for the weight elicitation, is presented in Table 3.

Table 3 Weight elicitation process: pairwise comparison of the criteria

Table 3 reveals that, according to the survey participants, there are no intense differentiations among the three criteria, as the maximum value that appears in Table 3 is the number 3. It is worth mentioning that, although the topic of the survey is innovation in Public Transport, no precedence was given to the criterion of Innovativeness.

The weights (%) that were derived from this elicitation are displayed in Fig. 2. The most important criterion appears to be Utility (42.90%). Feasibility follows with a very small difference scoring (40.10%), a value close to that of Utility, leaving Innovativeness (17.00%) notably behind, which reflects a rather “sensible” and “rational” approach to Innovation.

Fig. 2

Weight (%) per criterion after the AHP application

As mentioned earlier, although the topic of the survey is innovation in Public Transport, no precedence was given to the criterion of Innovativeness. It appears like the radicalness of an innovation is a little sidelined in favor of the two more realistic criteria. This could be justified due to the fact that these ideas and concepts have resulted through a participatory process with the aim of encouraging innovation and triggering creativity. In this sense, innovativeness already comprises an inherent attribute of the selected ideas and, thus, experts may have placed impulsively more emphasis on the practical aspects of the concepts, drawing upon their expertise and knowledge of the real world. In any case, the motivation behind this result is worth further investigation.

Moreover, CR was calculated, and it was found to be approximately 2%, way under the 10% threshold suggested by Saaty [36]. The aggregated answers were then used as input to a specialized decision support software that implements AHP (Super Decisions by Creative Decisions Foundation [CDF]) [48].

Initially, the evaluation took place taking into account all three criteria (Utility, Feasibility, and Innovativeness), leading to the total ranking of the 20 innovations. Table 4 includes the innovations ranked in descending order, with the one which has obtained the highest evaluation score being on the top. Each innovation corresponds to two numbers, forming two different columns. The column named “Normals” comprises the evaluation score of each innovation that sum up to 1.00 (i.e. the percentage score of each innovation in total) while the one named “Ideals” is the same score scaled to the innovation with the highest score considered to have been given an evaluation score of 1.00.

Table 4 Total ranking of innovations

Figure 3 presents an illustration of the aforementioned total ranking. The 20 different innovations (with their respective code) are displayed in the vertical axis, while the horizontal axis corresponds to the ideal score of each innovation (%).

Fig. 3

Total ranking of innovations with respect to all three examined criteria (Ideals %)

The Mobility as a Service and platform with real-time travel, comfort, and multi-modal information [MAAS] is the innovation with the highest overall score (100%), with respect to which all others are compared. The second one, which gets a remarkably high overall score, is the Fully Equipped Public Transport making travel time exploitable time [FULLEQUIPPT] (99%). There are three more innovations that are considered to have an indisputable advantage compared to the rest (having obtained an ideal score of over 90%) and these are: City marketing from a Public Transport perspective [CITMARK], Integrating Public Transport use in entrance tickets of events [PTEVENTICK], and Real-time journey planner on app including management of customer requests [RETIJOURNAPP].

Another sub group of three innovations follow, the overall score of which range between 73%–75%: Innovative solutions for on-board and at bus stops passengers’ counting [PASSCOUNT], Use of data mining tools for enhancing service operation and performance [DATAMIN], and Advanced e-ticketing system [ADVETICKET].

At the same time, others such as the Public Transport funding by the beneficiaries of the system [PTFUND], the Use of E-ink technology for the screens of info-panels [EINK], and the Umbrella brand concept [UMBRRAND] innovations score particularly low (below 40%) not having succeeded in attracting effectively the participants’ attention. The rest of the innovations correspond to an evaluation score placed from approximately 40% to 60%.

From these observations, it is hence derived that the respondents are more positive towards an innovation which corresponds to a generic concept which has already been launched in some cases and has gained increasingly wide popularity recently: the Mobility as a Service (MaaS) concept. In addition to that, they appear eager to adapt innovations that aim to make their travel experience more convenient, constructive, and/or amusing. They seem to focus more on innovations that they believe would have a direct impact on their commute, rather than on those which would possibly enhance Public Transport system in the long-term but would require a potential money contribution by them in the short term (e.g. the Public Transport funding by the beneficiaries of the system [PTFUND] innovation ends up in the last place). Another interesting outcome from the overall ranking is that the City marketing from a Public Transport perspective [CITMARK] innovation gathered noteworthy attention while the rather similar Umbrella brand concept [UMBRRAND] innovation ended up third from the end; the marketing idea seems additionally attractive when it is accompanied by a more holistic, urban approach.

Despite the fact that the overall ranking allows a clear comparison among the preferences of the survey participants, in order to gain insight in the strong and weak aspects of the innovations with respect to each one of the examined criteria, the AHP was implemented three more times to rank the innovations based on Feasibility, Utility, and Innovation, respectively. This way, a sensitivity-like analysis was performed; the stability of the overall ranking was investigated through examining the impact of each criterion to the final outcome.

The ranking of innovations based exclusively on the criterion of Feasibility is illustrated in Table 5 and Fig. 4.

Table 5 Ranking of innovations with respect to Feasibility

Fig. 4

Ranking of innovations with respect to Feasibility (Ideals %)

An interesting outcome is that Mobility as a Service and platform with real-time travel, comfort, and multi-modal information [MAAS] is no longer in the first place; in this case, the innovation which outperforms the rest (and is used to scale the others, considered as having “1.00” as a score) is the City marketing from a Public Transport perspective [CITMARK] innovation, which was previously ranked third in the overall ranking.

Fully equipped Public Transport making travel time exploitable time [FULLEQUIPPT] remains in the second place. Another difference between the two diagrams (Figs. 3 and 4) is that the scores obtained in this latter case are considerably lower and, moreover, the scores of the last ones are also even lower than before. Public Transport funding by the beneficiaries of the system [PTFUND] does not manage to move from the last place in this ranking either; nevertheless, the 19th and 18th place are occupied by different innovations: Public Transport Seats [PTSEATAPP] and Bus lane with intermittent priority [BLIP].

Table 6 and Fig. 5 present the ranking of innovations taking into account only the criterion of Utility.

Table 6 Ranking of innovations with respect to Utility

Fig. 5

Ranking of innovations with respect to Utility (Ideals %)

This time Mobility as a Service and platform with real-time travel, comfort, and multi-modal information [MAAS] returns to the top of the ranking; Fully equipped Public Transport making travel time exploitable time [FULLEQUIPPT] is stable in the second place while a new entry completes the top three: Real-time journey planner on app including management of customer requests [RETIJOURNAPP]. The other edge of the ranking appears with no surprises concerning the last innovation, but meanwhile the Bus lane with intermittent priority [BLIP] that has scored low above, now almost doubled its score reaching 60%. Another noteworthy differentiation is that significantly high scores over 90% are met in this ranking also, likewise with the first (overall) one.

The ranking of innovations with respect to the last criterion (Innovativeness) is displayed in Table 7 and Fig. 6.

Table 7 Ranking of innovations with respect to Innovativeness

Fig. 6

Ranking of innovations with respect to Innovativeness (Ideals %)

The first innovation here is different from the other three cases; it is the Advanced e-ticketing system [ADVETICKET], leaving the Mobility as a Service and platform with real-time travel, comfort, and multi-modal information [MAAS] concept in the second place. The Funding the Public Transport system through its beneficiaries [PTFUND] innovation is once again in the final place. It is worth mentioning that the City marketing from a Public Transport perspective [CITMARK] innovation, which was placed higher in the other cases is somewhere in the middle this time (9th). The Innovative solutions for on-board and at bus stops passengers counting [PASSCOUNT] innovation climbs up to the top three.

The evaluation with respect to each specific criterion has proved that the AHP analysis results can be considered stable and consistent as no drastic differentiations among the innovations ranking were identified.

Figure 7 presents an accumulative diagram with the evaluation scores (i.e. Normals in %) of all the criteria per innovation that sum up to 100%. The trend-line has no actual meaning (i.e. it represents no physical trend but just the totals from the 1st to the 20th innovation).

Fig. 7

Accumulative diagram with the evaluation scores of all the criteria per innovation (Normals %)

Figure 8 presents the previous information in the form of a radar chart. The radar/spider chart is a graphical method of illustrating data in two dimensions using axes with a common starting point. All axes are radially allocated with the same distance among them. The use of different colors is helpful in order to easily compare the alternatives with respect to the different criteria. In this case, it could be said that Utility and Feasibility have a greater buffer zone than Innovativeness; hence, Innovativeness is an important criterion for a lower number of innovations compared to the other two criteria.

Fig. 8

Radar diagram with the evaluation scores of all the criteria per innovation (Normals %)

Finally, for easy referencing, Table 8 presents the previous information in numbers.

Table 8 Normals (%) per criterion and innovation

6 Conclusions and perspectives

The analysis and synthesis of the obtained results revealed very interesting observations regarding which of the innovations can be considered as the most promising among the ones that resulted from the collective intelligence processes applied in the context of CIPTEC project. Noteworthy similarities, but also contradictions, emerged among the overall ranking of innovations and the ranking with respect to each one of the examined criteria. The approach presented herein provides valuable insight into mobilizing innovation, in the form of ideas “provided by citizens, for citizens”, to make Public Transport in cities more attractive and hence increase its use. In order to contribute in achieving the challenging task of planning successfully innovative urban Public Transport systems for the years to come, the output of the survey described in the present paper will feed the development of the CIPTEC Toolbox for Public Transport Innovation, a “living-tool” to assist stakeholders and decision-makers in integrating innovation to urban policies, to co-shape with European citizens a bright, novel future for Public Transport [49]. Finally, the results of the present paper can give a first clue to PTOs on which innovations should be applied to each special case, depending on the peculiarities of each European city and the importance they give to Feasibility, Utility, and Innovativeness. Other techniques applied in the frame of CIPTEC project, such as the Conjoint Analysis, give even further insights, such as possible shift in Public Transport modal share and hidden groups amongst the Public Transport users and citizens of European cities [50].