The traffic signal control problem for intersections: a review

Abstract

Background

The intersection traffic signal control problem (ITSCP) has become even more important as traffic congestion has been more intractable. The ITSCP seeks an efficient schedule for traffic signal settings at intersections with the goal of maximizing traffic flow while considering various factors such as real-time strategies, signal timing constraints, rapid developments in traffic systems, and practical implementation. Since the factors constituting the ITSCP exhibit stochastically complicated interactions, it is essential to identify these factors to propose solution methods that can address this complexity and still be practically implemented.

Objective

The objective of this review is to provide a survey of problems, methods, and practices in the evaluation of the ITSCP. In this paper, a unified terminology for the ITSCP and a citation network of the current body of relevant research are accordingly presented, and various assumptions, constraints, and solution approaches are summarized. A review across the entire body of knowledge throughout the history of the ITSCP is therefore provided. This review also highlights open issues and challenges that remain to be addressed by future research.

Introduction

In this paper, we review research on the intersection traffic signal control problem (ITSCP), which has been consistently studied for over 60 years since Webster [1] first published guidelines for traffic signal settings. A decade later, Robertson [2] developed a software tool named TRANSYT that determined optimal fixed-time traffic signal settings and calculated the performance index of the signal network. Following their research, theoretical analyses have been conducted to derive equations for the traffic capacity and average delay per vehicle at a signalized intersection [3, 4], and various microscopic traffic simulators have been developed to optimize traffic signal settings (e.g., VISSIM) [5]. Based on these theoretical foundations, various algorithms have since been proposed considering the rapid development of traffic infrastructure such as vehicular actuated systems and induction loop detectors [6]; this review focuses on these algorithms.

Research on the ITSCP has grown much more important with increasing traffic congestion. Traffic congestion has always been a crucial aspect of urban planning but has become a serious issue that must be addressed due to the rapid increase in the number of vehicles and transportation demand [7]. Traffic signal control is an important tool in traffic flow management as it is considered as one of the most effective ways to reduce traffic congestion at intersections [8]. As automated and connected vehicle technologies have recently become more popular, further research on the ITSCP continues to be necessary.

As McKenney and White [9] stated, there is no single dominant approach for the study of the ITSCP. Because of vehicle flow interactions within the network, human behavioral considerations, stochastic traffic demand, and traffic accidents, the ITSCP is a complex problem [10]. The ITSCP is further complicated by not only the randomness of vehicular arrivals at an intersection, but also by the various configurations and numbers of intersections, types of vehicles in the network, and the different priorities of real-time traffic management strategies. A review of the problem characteristics and relevant solution approaches can therefore provide researchers a guideline for understanding the ITSCP to help them gain the intuition required to solve it.

The remainder of this paper is organized as follows. Section 2 presents the ITSCP background and defines the terms to be used throughout the paper. Section 3 describes the details and issues of the ITSCP. The next two sections classify the ITSCP according to several perspectives: Section 4 describes the classification schemes and Section 5 classifies the traffic flow models and methods used to solve the ITSCP. Finally, Section 6 summarizes the current state of research and suggests directions for future research.

Background and terminology

Because the terminology used in ITSCP research papers has been constantly changing for decades, it is essential to first clearly and consistently define traffic signal terminology. In this review, the definitions of terms are taken from the traffic signal timing manual of the Federal Highway Administration of the United States Department of Transportation [11].

At an intersection, the movements of various users such as vehicles and pedestrians follow the rules indicated by traffic signals. In traditional traffic signal settings, there exists a sequence of indications that periodically repeats. There are three main concepts describing traffic signal sequence settings: cycle, phase, and duration. The cycle is the total time required to complete one signalization sequence for all movements at an intersection, the phase is the controller timing unit associated with one or more movements, and the duration is the amount of time the signal spends in each phase, during which the signal indications do not change [11]. Furthermore, a traffic flow group is defined as one or more compatible movements of road users, and each phase has a set of timings for each traffic flow group. Figure 1 illustrates eight phases of typical vehicular and pedestrian movements at a four-legged intersection, in which the solid and dashed lines represent vehicle and pedestrian movements, respectively. Each number in Fig. 1 corresponds to a phase; for example, the straight westbound and right-turning vehicles, and westbound and eastbound pedestrians crossing the northern leg of the intersection are assigned to Phase 4.

Fig. 1

Typical vehicular and pedestrian movements at a four-legged intersection [12]

By combining the concepts of cycle, phase, and duration, a signal phase sequence can be defined that represents a sequence of vehicle movements regulated by the signal controller. Figure 2 illustrates a typical example of a signal phase sequence at a four-legged intersection in the United States, called a National Electrical Manufacturers Association (NEMA) standard ring-and-barrier diagram. In this diagram, a ring is a sequence of phases that are incompatible and thus must be served in a particular order, and a barrier is a reference point in the cycle at which a phase in each ring has reached its point of termination. In Fig. 2, the phases in both rings must be simultaneously turned to red at the barrier. Various phase combinations and orders can be used to define a signal phase sequence as long as any conflicting movements are avoided.

Fig. 2

Typical example of a signal phase sequence (NEMA ring-and-barrier diagram) in the United States [11]

As shown in Fig. 3, ITSCP network types can be classified as isolated intersections, arterial networks, and general networks. An isolated intersection is a single intersection and an arterial network is a sequence of consecutive intersections in one direction. If the network is not an isolated intersection or an arterial network, it is considered to be a general network. Thus, a general network consists of multiple intersections that are not all sequentially connected and includes grid intersections, which are also called A × B intersections.

Fig. 3

Types of ITSCP networks: (a) isolated intersection, (b) arterial network, and (c) general network. Dashed lines represent the network extent for each problem. All intersections are four-legged intersections

It is important to state the general assumptions when discussing any ITSCP. In this review, unless otherwise specified, drivers are on the right-hand side of the road and right-turn movements are not explicitly controlled by traffic signals. All intersections are four-legged intersections unless otherwise noted. Road users in the network are passenger cars, pedestrians, transit vehicles, passengers of transit vehicles, emergency vehicles, motorcycles, heavy goods vehicles (HGVs), large goods vehicles (LGVs), and bicycles. Passenger cars are considered normal vehicles that are not given any priority. Transit vehicles (e.g., buses), and emergency vehicles may be given priority. Motorcycles, HGVs, and LGVs do not have priority, but are specially considered when the ITSCP accommodates emissions and vehicle speed concerns. Researchers have taken into consideration some or all of the aforementioned road users in previous studies.

Problem description

The challenge of the ITSCP is to find an optimal traffic signal configuration schedule that maximizes the traffic flow in a network. In other words, the goal of solving the signal timing control problem is to determine optimal phase sequences and durations for each phase. To solve this problem, the geometric information describing the intersections in the target network, the traffic information including traffic demand and turning movements of vehicles, and the limits regarding traffic signal components are considered. This information is processed in accordance with the model formulation. For example, Lin and Wang [13] expressed traffic demand as the number of variables in each cell using a cell transmission model. The ITSCP can be solved by optimizing various performance criteria, such as minimizing the average vehicle delay or maximizing the throughput of the network. More details of such objective criteria are described in Section 4.4. In most studies, the constraints considered in the ITSCP are related rules regarding traffic flow and traffic signal laws such as total cycle length, green signal length, and phase sequence. The details of signal timing constraints will be explained in Section 4.5.

Table 1 shows a list of the literature included when preparing this review. The number of citations was retrieved from Google Scholar on July 1, 2020. Figure 4 shows the citation network for the selected literature, displaying the citation relationships among the reviewed papers as arrows pointing from the citing paper to the cited paper, in which each node number corresponds to the paper ID in Table 1. To provide an effective visualization of the relative importance relationships among the reviewed papers, only papers that were cited at least once by the other papers in Table 1 are shown. Node size is proportional to the number of citations from the other reviewed papers, and the node position along the y-axis indicates the publication year of that paper. Note that the number of arrows pointing into a node may not match the number of cited papers because references that were not cited by other reviewed papers were omitted from the figure. Using Table 1 and Fig. 4, the most important papers in the ITSCP field can readily be identified, and node information such as author(s) and publication year can be extracted. For example, nodes 2 and 23 are larger than the other nodes shown in Fig. 4. Then, using Table 1, it can be determined that the paper written by Gazis in 1964 (Node 2) and the paper written by Mirchandani and Head in 2001 (Node 23) are the most important among those evaluated in this review.

Table 1 List of reviewed papers

Fig. 4

Citation network for the literature reviewed in this paper and defined in Table 1

ITSCP classification based on problem characteristics

In this section, we classify the ITSCP according to its various characteristics. Due to the highly stochastic nature of the ITSCP, problem complexity is a crucial consideration. The complexity of the ITSCP depends on various factors such as the number and shapes of the intersections and the types of vehicles in the network, as well as the real-time strategies used (if any). Analyzing the sources of computational complexities in the ITSCP is relevant to the practical application of optimized traffic signals at intersections. Accordingly, to provide researchers with the insight required to solve the ITSCP and address open problems, we have chosen to mainly focus on the factors affecting problem complexity. The optimization objectives of the ITSCP are also discussed from a practical perspective. Table 2 shows our proposed classification criteria based on the ITSCP characteristics, and the details of each criterion are described in the following subsections. Table 3 summarizes the ITSCP literature with respect to these classification criteria, in which the ‘Network type’ column defines the network structure, where ‘I’ indicates a single intersection, ‘A’ indicates an arterial, and ‘G’ indicates a general network, while ‘(A × B)’ indicates that the target network has a grid structure with A rows and B columns, ‘(C TL)’ indicates that the target network contains C intersections, and ‘(N-leg)’ indicates that the intersection has N legs. The ‘Lane’ column defines the maximum number of lanes in a single direction of each road in the studied network. Finally, ‘-‘ means that no information for the given column is provided in the subject paper.

Table 2 ITSCP classification based on problem characteristics

Table 3 Literature classification based on problem characteristics

Network type: isolated intersection, arterial network, or general network

As discussed in Section 2, we classified the network types evaluated in ITSCP research into isolated intersections, arterial networks, and general networks. Computational complexity increases dramatically as the number of lanes and intersections increase, or as the intersections are connected in more complex structures. Earlier research therefore covered only ITSCPs at an isolated intersection. For example, Dunne and Potts [14] solved the ITSCP for an isolated intersection with a maximum of two lanes on each leg. Afterwards, the network scope expanded to include isolated intersections with multiple lanes in each direction and various shapes such as T-junctions [7, 26, 39]. Similarly, arterial networks with multiple lanes were studied in detail after Gazis [16] first discussed a 1 × 2 arterial network consisting of two sequential intersections. Finally, Wong [35] explored a general network containing 15 intersections with one or two lanes on each leg.

As computer hardware and software simulation tools have developed, the computationally affordable network size has increased. Recently, some papers have succeeded in applying algorithms to real-world networks such as a 9 × 7 grid of intersections in Ottawa, Canada and a general network containing 50 intersections in Tehran city [9, 75]. Nonetheless, the ITSCP is still being actively researched for isolated intersections or small arterial networks. Jin and Ma [73] and Li et al. [77] solved the ITSCP for an isolated intersection and 1 × 3 arterial network model, respectively. The networks evaluated in both papers considered contained intersections with only one or two lanes on each leg. Such small networks are still being actively researched because of the development of connected vehicles and new solution methods. For example, Christofa et al. [65] proposed a person-based optimization approach on arterial network by considering passenger occupancy of vehicles explicitly in a connected vehicle environment. When the passenger occupancies of vehicles are considered as decision variables, the number of constraints and variables increases with the number of vehicles in the system, necessitating a small network. Additionally, as new solution methods are developed, they are typically first validated using a small network.

Type of road users and priority consideration

In this review, we assumed that the traffic on the roads consists of passenger cars, pedestrians, transit vehicles and their passengers, emergency vehicles, motorcycles, HGVs, LGVs, and bicycles for the ITSCP. Because it is difficult to take all traffic types into consideration, most researchers have limited the type of traffic modes to specific categories. A large number of papers have considered only one type of passenger car without pedestrians. Improta and Cantarella [28] first expanded the type of road users considered to include pedestrians in addition to a single type of passenger car. Pedestrians are accounted for in the ITSCP in terms of the minimum green light time required for them to cross the road. Some papers dealing with physical queue lengths or the occupancy of the network have accounted for various types of passenger cars [39], and Chandan et al. [71] considered various types of passenger cars as well as HGVs to more precisely estimate emissions. Recently, studies considering bicycles have been conducted as the number of intersections with dedicated bicycle lanes increases to accommodate the growing number of cyclists [69, 79]. Portilla et al. [69] proposed separate vehicle and bicycle models for the ITSCP to reflect the ability of bicycles to be accommodated in smaller spaces as well as the simpler description of the dynamic behavior of bicycles.

Transit vehicle have been considered important road users in the ITSCP since Salter and Shahi [80] demonstrated that giving priority to buses reduced bus delay at the cost of increasing passenger car delay. Subsequent research efforts have been dedicated to finding more advanced transit signal priority logic considering the performance indices of the vehicles in the network. Ekeila et al. [29] proposed an algorithm to minimize the delay of transit vehicles while preventing negative impacts on street traffic. Christofa et al. [46] approached the problem from the perspective of the individual, especially the drivers of passenger cars and passengers of transit vehicles. He et al. [41] gave priority not only to transit vehicles, but also to emergency vehicles. With the advent of connected vehicles, it is now possible to obtain additional information about the network state and vehicle operations [54]. Using vehicle-to-infrastructure communication systems, the traffic signal control system can receive requests from appropriately equipped vehicles and pedestrians to generate an optimized signal timing plan that accommodates all of the active requests. As communication technology continues to rapidly develop, more research into solving the ITSCP with priority consideration is expected.

Real-time strategies: fixed-time, actuated, or adaptive

Three major traffic control strategies can be used when solving an ITSCP: fixed-time, actuated, and adaptive [57]. The fixed-time strategy establishes optimal signal plans for fixed signal phase sequences with a fixed time duration for each phase. Adopting the fixed-timed strategy assumes that traffic demand remains similar at all times to calculate the optimal signal plans based on historical traffic information. Gazis [16] and Smith [26] used the fixed-time strategy for a 1 × 2 arterial network and an isolated intersection, respectively.

The actuated strategy collects real-time data from infrastructure-based sensors and applies a simple logic criterion such as green light extension, gap out, or max out. Green light extension prolongs the green phase based on traffic flow rate. Gap out terminates a phase when the time interval between consecutive activations of a vehicle detector exceeds an established threshold. Max out terminates the green phase when it exceeds the established maximum green phase duration. Since Dunne and Potts [14] first adopted the actuated strategy of green light extension assuming a constant arrival rate per experiment, actuated strategies have been consistently applied in research [40, 54, 60, 75].

The adaptive strategy is similar to the actuated strategy, but utilizes predicted traffic conditions in the near future. Dell’Olmo and Mirchandani [33] identified vehicle platoons and predicted their movements in the network using the Approximate Prediction in Response to a Signal Network (APRES-NET) model. The adaptive strategy has been implemented using various other prediction algorithms, and several adaptive signal control systems have been developed accordingly. These systems include ACS-Lite [81], SCATS [82], SCOOT [83], OPAC [84], MOTION [85], UTOPIA [86], and RHODES [6]. Recently, Lee et al. [8] predicted information including lane-to-lane turning proportions, adjustment factors, queue lengths, and arrival and discharge rates using a rolling-horizon process and then calculated an optimized signal plan using a proactive global optimization method. Because adaptive strategies require highly accurate prediction algorithms as well as good signal plan optimization, developing algorithms based on an adaptive strategy could be more difficult than when doing so based on an actuated strategy.

Objectives

Lee and Park [87] discussed two measures for evaluating the performance of traffic signal control algorithms: mobility and sustainability. Mobility measures consist of the average total delay, average total throughput, average total travel time, average total number of vehicle stops, and average queue length. Sustainability measures consist of emissions and fuel consumption. Most research into the ITSCP has primarily used mobility measures.

Mobility measures

A fundamental performance measure when solving an ITSCP is the delay per vehicle, the minimization of which serves to minimize the average waiting time of vehicles at an intersection due to a red signal. This performance measure is the most commonly used in ITSCP design as indicated by the fact that 61 of the 72 papers in this review treated delay as the fundamental performance index. Some papers considered a weighted delay as a performance measure. Prashanth and Bhatnagar [36] gave a higher weight to main road traffic delay, and Murat and Gedizlioglu [76] proposed a weighted average delay considering traffic volumes in each direction as an objective value. For situations considering different traffic mode priorities, some researchers minimized the delay of transit vehicles [29, 41], and some considered weighted personal delays for both passenger cars and transit vehicles according to their respective passenger occupancies [46, 65].

Another important concept when evaluating traffic signal systems is the throughput of the network. In the ITSCP, throughput is the capacity of the network, defined as the number of vehicles passing through the network. Smith [26] attempted to maximize the throughput. Later, some researchers combined capacity maximization in terms of throughput with other measures [34, 36, 39, 52, 67, 70, 73].

The total travel time of a vehicle is the duration of time it moves in the network. Wong and Yang [45] considered the total travel time of vehicles as a performance index when solving both a signal setting optimization problem and traffic assignment problem. They attempted to take into account the fact that the equilibrium pattern flow of a network is strongly related to signal settings. Some studies conducted within an assumed connected vehicle environment have also used the total travel time of vehicles as a performance index [67].

Minimizing the total number of vehicle stops in a network has also been used as a mobility measure. Vehicle stops, which occur due to a red light or accumulated queue, are directly related to driver satisfaction. Some studies developed flexible models that minimize either the average delay or total number of stops [13, 37, 49, 51], and some studies combined the two performance measures using weighted combinations [35, 60, 64, 72].

To balance each traffic signal phase and each direction in an intersection, the concept of queue length, defined as the total number of vehicles waiting on the roads at each intersection, has been used. Queue length is correlated to the delay or number of stops and as such is typically applied together with these performance measures. Spall and Chin [10], De Schutter and De Moor [42], and Feng et al. [57] proposed the minimization of the average queue length as an additional objective of an ITSCP. Specifically, Feng et al. [57] verified that the minimization of queue length can lower the variance of vehicle delay in each phase. To balance queue length for all roads in the subject network, Sen and Head [37] and De Schutter [62] minimized the maximum queue length.

Sustainability measures

As awareness of the importance of environmental protection has grown, researchers have begun to investigate the environmental impacts of traffic signalization. Aslani et al. [75] employed a microscopic emissions/fuel consumption model to minimize both exhaust products such as carbon dioxide and fuel consumption to improve sustainability. Models evaluating emissions/fuel consumption require the assumption of some constraints on vehicular speed, deceleration, and acceleration. As stated by Han et al. [63], emissions-related objectives make traffic signal optimization problems more difficult due to their nonlinearity and non-convexity.

Other measures

Additionally, various performance indices have been used in accordance with different assumptions and problem-solving methods. For arterial networks, the bandwidth, or portion of a signal cycle during which a vehicle can progress through the signals without stops, has been maximized [31, 33]. Improta and Cantarella [28] and Wong and Wong [70] considered cycle-time minimization as a secondary objective, claiming that if two signal time plans output similar levels of delay and capacity, the plan with the shorter cycle time is better. Arel et al. [30] minimized the likelihood of intersection cross-blocking. Some authors compared the aggregated average speed of vehicles [9] and the number of vehicles in bottleneck links [59]. When accounting for cyclists in the ITSCP, Wang et al. [79] attempted to maximize safety by proposing a traffic conflict index estimated based on the probability of vehicle crossing and the potential traffic conflict severity.

Signal timing constraints

In this sub-section, we summarize ITSCP constraints regarding cycle length, green phase duration, and phase sequence signal timing constraints.

Cycle length

Constraints on cycle length can be classified into four types: fixed, limited minimum length, limited maximum length, and unrestricted. Dell’Olmo and Mirchandani [33] optimized traffic signals while maintaining the total time of one complete phase sequence; in other words, they assumed a fixed cycle length. Some papers limited the minimum and maximum cycle length [31], whereas others limited only the maximum cycle length, such as Gallivan and Heydecker [7], who limited it to 120 s. Spall and Chin [10], however, calculated the total cycle length, red–green splits, and offset times without any restrictions.

Green phase duration

In a similar fashion, the green phase duration can either be constrained by a minimum/maximum limit or allowed to be any value. The selection of the minimum and maximum green phase durations is dependent on the traffic characteristics of the study area and the space available for vehicles to queue [76].

A minimum limit on the green phase duration is normally required for safety and to guarantee that no phase is skipped [65], and is also relevant to pedestrians. Because signals from different road directions are entangled in a single traffic signal system at an intersection, a minimum limit on the green phase duration in one direction can accordingly be interpreted as a minimum limit on the red phase duration in the perpendicular direction. To ensure that pedestrians can cross the road comfortably, this red phase should not be too short [39]. When setting the minimum green time to accommodate a pedestrian crossing, the minimum duration depends on the width of the crossing and the assumed walking speed of the pedestrians [35].

A maximum limit on the green phase duration is usually defined to limit the green extension for signal groups [73]. Most papers started from a minimum green time value and extended the green phase duration until reaching the maximum limit. Though some papers defined only a minimum limit on the green phase duration and allowed a long green phase [35], it is normal practice to constrain the maximum green phase or the total cycle length.

Phase sequence

The signal phase sequence represents a kind of rule between vehicle drivers and traffic signals. Some researchers have claimed that the control system should use a fixed signal phase sequence so as not to confuse drivers, while others have argued against a fixed phase sequence for the sake of performance improvement.

In the early years of ITSCP study, most researchers treated the phase sequence as a given parameter and formulated the problem using fixed phases [14, 16, 18, 20, 22, 24, 26, 28]. Ross et al. [18] addressed the problem using only two phases whereas Wong [35] generalized it to multiple phases. The assumption that the sequence of phases is fixed enforces safety and fairness constraints [6, 88].

Some studies predefined signal phase groups in which compatibility was assured and selected a proper signal phase sequence from among these groups at each rolling horizon. For example, Lee et al. [8] proposed a multi-resolution strategy for updating the elements of the signal plans that included a cycle-by-cycle signal phase sequence and adjusted the current second-by-second green signal timing. Some studies formulated the existing group-based signal as an agent and applied a multi-agent system strategy [55, 73]. The advantages of using predefined signal groups include a high degree of flexibility when specifying signal plans and the ability to deal with a wide range of traffic patterns in a systematic way [8]. Dynamic programming has been widely used to choose phase sequences because the ITSCP can be solved recursively without fixed phase constraints within affordable limits of computational complexity [6, 27, 37].

When not using fixed phase sequences, Dell’Olmo and Mirchandani [33] claimed that any sequence of phases and their associated phase durations could be considered for signal plans. In this case, the ITSCP involves a choice of phase sequences and timings to optimize a specified performance index. For example, Haddad et al. [32] simplified traffic flow as a set of vehicle movements at an isolated intersection and determined when to switch the green–red signal for each vehicle movement. By using flexible phase sequences, phase pictures were generated considering real-time traffic patterns so the travel delay caused by inefficient phase formulations could be reduced [55].

ITSCP classification based on solution methods

This section reviews the various ITSCP solution methods applied in the literature, as summarized in Table 4. Some ITSCPs have been formulated as mathematical models based on a framework reflecting traffic dynamics and solved using analytic methods. The Lighthill–Whitam–Richards model [89, 90], cell transmission model (CTM) [91], and other various models have been used to account for macroscopic traffic dynamics [32]. Analytic methods such as the branch and bound (B&B) algorithms have been used to solve iteratively small and simple ITSCPs. There are, however, various solution approaches that have been applied to ITSCPs regardless of problem size or conditions, including those employing a rule-based method, genetic algorithm (GA), simulation-based approach, dynamic programming (DP), multi-agent system (MAS), game theory approach, neural network (NN), or reinforcement learning (RL), which are reviewed in this section.

Table 4 Solution methods for ITSCPs

Most early researchers employed a rule-based method to solve the ITSCP [14, 18, 22, 24, 33]. The rule-based method defines key states and proper actions corresponding to each state to optimize the performance criteria. In a signal timing control problem, key states are usually represented in terms of whether or not the current queue length or the duration of the current green phase is within the specified range. Actions can then be taken to switch the phase sequence or extend the green phase duration. Recently, some researchers have also combined rule-based methods with other methods or assumed various problem scenarios for evaluation. For example, Murat and Gedizlioglu [76] combined fuzzy logic and a rule-based method by defining key states with approximated input. Considering an adaptive strategy, Ekeila et al. [29] used a dynamic rule-based system that changes rules depending on traffic conditions to handle real-time problems.

The GA is a remarkable heuristic method that has been widely used in signal timing design for decades since Foy et al. [92] first used a GA to optimize the phase sequence and green phase duration in a traffic network of four intersections, showing a significant improvement in system performance [21]. Lo et al. [58] formulated a mathematical model based on the CTM and developed a heuristic approach based on a GA to reduce the extensive computation time required to solve the mathematical model for large networks. Liu and Chang [38] explicitly modeled physical queue evolution according to lane group to account for shared-lane traffic intersections and used a GA to solve the model. Once it had been verified that GAs are adept at obtaining high-quality optimization solutions using microscopic simulation tools [93], many studies developed GA-based simulation programs [21, 48]. In particular, Stevanovic et al. [21] extended the capability of a GA formulation by optimizing transit-prioritized traffic settings on roads with both private and transit traffic.

In addition to GA-based simulation programs, general simulation-based approaches have been proposed by many researchers to account for traffic flow interactions [38]. In a simulation-based approach, decision variables are optimized based on performance indices generated from the underlying traffic flow model that are developed using mathematical models in the simulation tool. The TRANSYT [2] and TRANSYT-7F [94] programs are the most widely used off-line signal timing optimization simulators. For adaptive traffic control systems, the SCOOT [95], SCATS [96], OPAC [83], PRODYN [97], CRONOS [17], and RHODES [6] programs have been developed. Many researchers have solved optimization problems and evaluated their models using these simulation tools. Examples of simulation-based optimization include McKenney and White [9], who attempted to find the optimal traffic signal settings using a simulation within the SUMO traffic simulation environment, and Dell’Olmo and Mirchandani [33], who used APRES-NET to estimate the traffic flows in a network. A number of studies have evaluated models using simulations because such evaluations can reflect the stochastic inter-arrival times of vehicles as well as vehicle interactions. For example, VISSIM [8, 29, 41, 57, 71], NETSIM [31, 60, 70], AIMSUN [46, 65, 75], and several other traffic simulation tools have been used to evaluate the performance of traffic models.

Motivated by its generality, some researchers have used DP to develop a flexible control algorithm applicable to a variety of performance measures and traffic conditions [6, 27, 37, 44]. Sen and Head [37] first applied DP to the controlled optimization of phases, using phases as stages and green phase durations as control variables in the DP formulation. In a follow-up study, Mirchandani and Head [6] developed an algorithm incorporating DP to control a 1 × 9 arterial network. Since the computational demand of the recursive calculations in DP dramatically increases as the network size increases, Cai et al. [27] proposed a modified DP algorithm to approximate the state space using RL. Zheng and Recker [44] developed a recursive optimization procedure consisting of data processing, flow prediction, parameter optimization, and signal control.

With increasing attempts to control traffic networks in a distributed way, the MAS has been investigated as an important solution approach [78]. Multi-agent learning, the game theory approach, and RL are typical solution approaches related to MAS. Roozemond [56] proposed a system that can autonomously adapt to changing environments by defining each component of the traffic system as an agent, then applied artificial intelligence to the defined agents—intelligent traffic signals—to provide prediction and control strategies. The need to define the agents of a traffic system is associated with the challenge of having a large number of agents that act in a highly coupled environment, but McKenney and White [9] succeeded in developing an MAS for a 9 × 7 grid of intersections. As an example of a game theory-based approach, Villalobos et al. [23] represented an intersection as a non-cooperative game in which each player (signal) tries to minimize its queue and then found the Nash equilibrium as the solution of the ITSCP.

Reinforcement learning has been widely used as a practical computational tool to obtain an optimal control policy [30]. Choy et al. [68], Bazzan [78], and Balaji et al. [34] used RL to implement cooperative hierarchical MASs for real-time signal timing control of complex traffic networks. The largest traffic network evaluated with RL was a general network containing 29 traffic lights studied by Balaji et al. [34]. Some studies have combined RL with fuzzy relations or neural networks (NN). Choy et al. [68] simultaneously applied a fuzzy NN and RL to optimize traffic signal timings for a large complex traffic network by dividing the main ITSCP into sub-problems, adjusting the learning rate and weights related to the fuzzy relations using RL, further adjusting the fuzzy relations using an evolutionary algorithm, and then calculating the optimal signal timing parameters. Among the various RL algorithms, Q-learning, which is a popular model-free RL algorithm, has been applied to ITSCP in many studies. Abdulhai et al. [66] employed Q-learning for an isolated intersection under various traffic conditions. Because constructing a Q-learning model in ITSCP requires a considerable number of states and actions, Prashanth and Bhatnagar [36] modified the model to include function approximation and solved the problem for multiple networks including a 3 × 3 grid of intersections and a 1 × 8 arterial network. Advanced RL algorithms such as the actor–critic algorithm have also been applied to the ITSCP [75].

In addition to the approaches discussed above, the Petri net has also been used to model ITSCP states. List and Cetin [98] and Dotoli and Fanti [99] modeled the dynamics of traffic network systems using Petri nets while adopting signal timing control algorithms from other researchers. Di Febbraro et al. [74] used a hybrid Petri net (HPN) model to express the problem of coordinating traffic lights with the goal of improving the performance of transit and emergency vehicles, and then developed control algorithms based on the HPN model. In a traffic network modeled by HPN, traffic flows are modeled as fluids, and the event-driven dynamics of the traffic lights and their influence on the flow dynamics can be considered explicitly. In another approach to modeling the dynamics of traffic network systems, Portilla et al. [69] used a model-based predictive control method to represent the flows of vehicles and bicycles as well as their interactions.

Discussion and future research directions

Trends

In this paper, we reviewed the available literature to provide a comprehensive overview of the various methods that have been applied to solve the problem of traffic signal control at intersections. We described the ITSCP and classified the related literature in terms of the various aspects of the problem. As shown in Table 1, the signal timing control problem has been widely studied. In this section, we present the trends of several aspects of ITSCP research including problem features, solution approach, and infrastructure development. In the first two subsections, changes in the problem definitions and methodologies over time are discussed. The last three subsections describe how the development of infrastructure technology such as vehicle connectivity, autonomous vehicles, and vehicle detection has affected the ITSCP.

Problem features

With the expansion of the ITSCP research field, various performance measures including vehicle emissions, vehicle fuel consumption, and transit passenger delay have been considered in addition to vehicle delay. Critically, the size and shapes of networks as well as the categories of road users have diversified over time, and as communications technology and computational methods have been developed, strategies dealing with real-time traffic data have become popular in the ITSCP field.

There are several trends among the problem features reviewed in Section 4. Studies considering transit vehicles as road users are typically related to priority consideration, and studies accounting for pedestrians typically limit the minimum value of the green phase durations to ensure adequate crossing time. The signal timing constraints used (phase sequence and cycle length) are typically related to the type of intersection network considered as summarized in Table 5, in which the columns indicate the type of intersection network, and the rows represent the phase sequence and cycle length constraints applied. The numbers in each cell in Table 5 indicate the papers corresponding to the problem features defined by that cell. Note that these reference numbers are different from the node IDs presented in Table 1. According to Table 5, most studies constrained the phase sequence of the traffic signal to a fixed sequence or a set of predefined sequences. As the phase sequences become more flexible, the time complexity of the ITSCP increases considerably. Additionally, research using isolated intersections tends to rely upon flexible cycle length constraints whereas more than half of the research conducted using arterial networks relied upon fixed or limited cycle length constraints.

Table 5 Literature summary based on intersection networks and signal timing constraints

Solution approach

The approach used to solve an ITSCP depends on the size of the target network and the assumptions in the problem. Mathematical programming models have been widely used to analyze small-size intersections. In almost all cases except for several early papers, the proposed solutions were evaluated using simulations of various traffic scenarios. The recent application of RL and MAS to large intersections and complex traffic situations has enhanced the ability of researchers to study the ITSCP at larger scales. Focusing on these recent applications of RL and MAS, Bazzan [100] presented problems and methods related to MAS in traffic engineering and emphasized remaining challenges that should be addressed in future MAS research.

Table 6 summarizes the ITSCP literature based on the solution methods reviewed in Section 5 and the problem features, in which the columns indicate the major methods used to solve the ITSCP and the rows represent the target intersection network type and real-time strategy employed. The numbers in each cell indicate the papers corresponding to the solution method and problem feature indicated for the cell. As can be seen from Table 6, analytic methods have been predominantly applied to isolated intersections, whereas RL and MAS have been predominantly used to control signal timings in general networks. Most of the papers in Table 6 used actuated strategies for real-time control, and adaptive strategies that require prediction logic for traffic flows have rarely been developed.

Table 6 Literature summary based on solution methods and problem features

Vehicle connectivity

Advanced cars with vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) connectivity have also recently been developed. Vehicle connectivity can now be classified into these two different types according to the communications target: V2I gathers the data generated by vehicles at infrastructure points to share traffic information from infrastructure to vehicles; V2V involves communications among vehicles regarding their speed, position, and traffic situation. In a connected vehicle environment with both V2I and V2V connectivity, a great deal more data describing the traffic states near an intersection can be collected and utilized for signal control. As algorithms based on connected cars are expected to potentially improve the performance of urban signalized intersections, Guo et al. [101] reviewed methods to estimate traffic flow and optimize traffic signal timing in connected and automated vehicle environments.

Autonomous vehicles

The technological advancement of autonomous vehicles is a valuable tool for the investigation of ITSCP solutions. McKinsey & Company predicted that up to 15% of new cars sold in 2030 could be fully autonomous [102]. If all cars could be controlled autonomously, there would be no need to consider driver reaction times, allowing for improved switching priority between several roads and the setting of very short green phase durations. Additionally, a reservation-based system for isolated intersections that do not specifically act as conventional traffic signals can be provided to ensure orderly movements of autonomous vehicles [78]. In other words, intersections without traffic signals can be developed as long as all vehicles in the network are autonomous.

Vehicle detection

Finally, the development of vehicle detection technologies enables the practical implementation of traffic signal control algorithms that have been researched in the ITSCP field for decades. Even though various real-time algorithms have been proposed, only a few algorithms have been used in practice due to the cost and time issues arising from the need to install detection sensors. In order to gather traffic information to serve as the input of an algorithm, detection sensors would previously have needed to be installed under roads at considerable cost and effort, but in recent years, technological advances in vehicle detection have allowed for the use of cameras or Bluetooth sensors as detectors instead, which is likely to solve such practical issues. Current camera-based vehicle detection technology can clearly distinguish the license plate number and type of each vehicle by analyzing pictures or videos [103, 104]. The use of cameras to detect vehicles therefore offers a much more economical and simpler approach as they can be mounted on extant traffic signal poles. Another new type of vehicle detector, the Bluetooth sensor, can estimate the travel times and velocities of vehicles and detect the turning movement of each vehicle at low installation and maintenance costs [105]. Furthermore, new vehicle detection technologies can provide various data that were unattainable in the past to serve as input data for traffic signal control algorithms.

Future directions

Although various formulations and solution approaches have been applied to solve ITSCPs over the decades, there are still many opportunities for future work. The following subsections describe problems that require further study.

Generalization of intersection networks

The most important direction for future research in the ITSCP field is the generalization of the problem. In other words, it remains necessary to optimize traffic signals for generalized intersections. Despite the efforts and advances made through various ITSCP research, the networks that have been studied remain too simple and small to apply the research results in practice. From the perspective of network simplicity, most researchers assumed the shape of their subject networks to consist of four-legged or sometimes three-legged intersections. However, it remains necessary to develop an approach that can handle a network consisting of variously shaped intersections. To address issues associated with the size of the network, larger intersection grids must be accommodated. Additionally, determining the appropriate size of a network to enable practical utilization appears to be a crucial research topic.

Consideration of real-world problem features

In addition to intersection network types, real-world problem features including road user types, physical properties of the road, and traffic flows should be considered in order to generalize solutions to the ITSCP. Lan and Chang [106] presented results regarding the effect of heavy mixed scooter–vehicle flows on the propagation of intersection queues. Similarly, different road users—including pedestrians, transit vehicles, motorcycles, bicycles, light rail cars, and HGVs—have complicated interactions with the environment due to their unique characteristics. For example, HGVs tend to have a lower acceleration and deceleration rate than other vehicles. As such, optimizing traffic signals for networks comprised of various road users is necessary for real-world implementation. Bicycles in particular have recently emerged as an alternative transportation mode, and studies have accordingly been conducted to investigate the interactions between bicycles and passenger cars, minimize the delay of cyclists, or explicitly consider two-stage bicycle left turns [69, 107]. As studies assessing the delays of cyclists and identifying the states of bicycles are currently underway, research into the ITSCP considering bicycles is expected to accelerate.

From a practical perspective, the slope and speed limit of roads, which are related to acceleration and deceleration rates, and the road capacity, which is used to calculate the volume of traffic flow that an intersection network can handle, are crucial issues. In addition, some intersection networks contain specifically designed lanes that allow unusual vehicle movements, such as contra-flow bus lanes and U-turn lanes. Most traffic signal control algorithms proposed thus far consider only simplified networks without special lanes, but the existence of those lanes could considerably influence the overall performance of traffic signal timing control. Therefore, it is necessary to categorize lanes or roads that are designed for special purposes in order to assess the efficacy of signal control algorithms in networks with such lanes or roads.

Finally, scenarios other than unsaturated or oversaturated traffic conditions, such as an accident scenario, should also be considered to ensure robust signal timing control. The accident scenario represents a situation in which some portions of some lanes or of the intersections in a network are disabled due to traffic accidents. Further research regarding traffic signal control in such accident scenarios are mandatory to prepare for the coming age of autonomous vehicles.

Coming age of autonomous vehicles

Indeed, the coming age of autonomous vehicles is another key point that must be addressed. There exist many differences between a case in which humans drive cars and a case in which vehicles drive by themselves. In the current system, it is widely accepted that signal phase sequences should be fixed so as not to confuse human drivers, but this is no longer necessary in the era of self-driving vehicles. In other words, as autonomous vehicles increasingly become a reality, the signal phase sequence will no longer need to be fixed as machines do not get confused. Thus, autonomous vehicle technology relaxes a constraint regarding the phase sequence. Furthermore, reaction time can be ignored when selecting the cycle time for autonomous vehicles, though safety will continue to be a critical performance index to prevent vehicle collisions. Thus, more research considering the characteristics of autonomous vehicles as they apply to solutions of the ITSCP is needed.

Connected vehicle environment

Similarly, further study on ITSCPs in a connected vehicle environment is required. Since Lee and Park [87] solved the traffic signal timing control problem in a connected vehicle environment assuming 100% penetration rate of connected vehicles, Feng et al. [57] have proposed an algorithm for solving the ITSCP according to various connected vehicle market penetration rates, in which they estimated the states of unconnected vehicles based on connected vehicle data. Chandan et al. [71] also considered the connected vehicle environment. None of the other reviewed papers dealt with connected vehicles. Even though the infrastructure for connected vehicles is developing rapidly, insufficient traffic signal control research has been conducted in this environment. In a connected vehicle environment, vehicles can share information regarding their speed, acceleration, position, and turning movement. Under these conditions, it is possible to give priority to emergency vehicles, control vehicle movements to prevent collisions, and share the occurrence and location of accidents. Furthermore, the connected vehicle environment enables the collection of information describing individual vehicles, so that equity can also be considered to be a performance measure of the ITSCP. For example, the maximum delay that any individual vehicle may experience can be limited to ensure equity in a connected vehicle environment.

Vehicle movement control

The ongoing development of autonomous and connected vehicles influences not only the assumptions and constraints of the traffic flow problem but also the problem definition itself. As technology for communication between vehicles and infrastructure is developed, control of each individual vehicle will become feasible. Information sharing between vehicles and road infrastructure will enable detailed tracking of each vehicle and make it possible to prevent collisions with adjacent or crossing vehicles. In recent years, several studies regarding the scheduling of vehicle movements based on the arrival and departure of each vehicle at an intersection, rather than relying on signal timing control for traffic flow, have appeared [87, 108,109,110,111,112,113]. This problem employs a dynamic traffic model for vehicle movements without considering conventional traffic signal rules. Considering the current state of the technology and potential advances in the near future, additional research on the vehicle assignment problem under the assumption of a connected vehicle environment is necessary. With further research, heavy traffic may be managed without traffic signals in the future. Vehicle movement control is also related to a path controlling scheme that considers the dynamics between the routing decisions of autonomous and connected vehicles and signal timing decisions. In other words, the phenomenon that vehicles may change their routes if they experience long delays must be considered in future work.

Algorithm robustness

As new connected and autonomous vehicle technologies are still being developed, improvements in algorithm robustness with respect to system malfunctions including noisy and delayed state measurements, communication dropouts, or other incidents are necessary to handle the exceptional cases and address potential safety concerns [87]. Furthermore, traffic signal control should be investigated under a scenario in which both connected or autonomous vehicles share the road network with conventional vehicles as it will take time for all vehicles on the road to be fully updated with such new intelligent vehicle technologies.

Unified simulation framework

Finally, there is currently no standard for comparing the performances of different traffic signal control models. Because none of the current network designs, tested traffic scenarios, or defined objective functions can be discussed in a unified manner, it is difficult to quantitatively compare the performances of different models. For instance, most researchers have minimized the delay per vehicle in the timing and control of traffic signals, but their definitions of delay may differ. “Delay” may indicate standstill times at the intersection due to congestion, the length of time for which the vehicle speed is less than a threshold speed, or the time difference between arrival and departure [48, 55, 87]. This problem could be resolved by proposing a set of benchmark instances. Such standards could also contribute to the standardization of ITSCP terminology, aiding research, communication, and development of traffic control methods.