Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing

The rising popularity of self-driving cars has led to the emergence of a new research field in recent years: Autonomous racing. Researchers are developing software and hardware for high-performance race vehicles which aim to operate autonomously on the edge of the vehicle’s limits: High speeds, high accelerations, low reaction times, highly uncertain, dynamic, and adversarial environments. This paper represents the first holistic survey that covers the research in the field of autonomous racing. We focus on the field of autonomous racecars only and display the algorithms, methods, and approaches used in the areas of perception, planning, control, and end-to-end learning. Further, with an increasing number of autonomous racing competitions, researchers now have access to high-performance platforms to test and evaluate their autonomy algorithms. This survey presents a comprehensive overview of the current autonomous racing platforms, emphasizing the software-hardware co-evolution to the current stage. Finally, based on additional discussion with leading researchers in the field, we conclude with a summary of open research challenges that will guide future researchers in this field.

SECTION I. Introduction

What aerospace engineering is to aviation, motorsport is to automotive technology. For over a century now, racing series such as Formula 1, Indy Car, or the World Rally Championship have inspired research and product innovation to improve performance and safety in commercial road vehicles. These developments include well-known elements such as the disc brake, the turbocharger, or production measures for fiber composites (e.g., carbon). In more recent years, developments in the hybrid powertrain and the connectivity (real-time measurement and transfer of vehicle data) of the vehicles have emerged. With millions of dollars of investment and prestige at stake, these developments target a singular goal: The racecar must achieve the fastest lap time and thus win the race at the end. With a vehicle that moves at the dynamic limits of handling, that reaches high velocities, that is designed for aerodynamic efficiency, and that consists of pure lightweight construction, traditionally the target of the minimum lap time can only be achieved through extremely novel, sophisticated, and radical developments. But the technical development of the racecar is only half of the effort.

In motorsport racing, it all boils down to the ability of the driver to operate the racecar at its limits [1], [2]. Expert race drivers are extremely proficient in pushing the racecar to its dynamical limits of handling, while accounting for continuously changing parameters such as tire wear, changing brake bias, and engine maps from turn to turn, communicating strategy and status with the team in the pits, and trying to maintain track position or overtake fierce competitors all while driving at speeds exceeding 300 km/h.

All of these facets manifest themselves in the innumerable research and development challenges sought to be addressed in the burgeoning field of autonomous vehicle racing. The current state of the art of autonomous driving software – either from commercial companies or researchers – can operate autonomously but only to a limited velocity. Everything that we find in classic motorsport can also be found in autonomous motorsport - with one difference: The racecar-driver is based on software only. This means that a highly sophisticated autonomous driving software needs to replace the human pilot. It should be capable of detecting other vehicles, localizing the vehicle position relative to the opponents and the track while driving at high speeds, planning dynamic trajectories to allow overtaking in adversarial environments, and correcting at high frequency to the steering angle to stay on the racetrack. Furthermore, the vehicle needs to execute a performance assessment on its own by adjusting the aerodynamics, energy distribution, differential settings, or brake balance settings based on tire wear, temperature, and weather. On this premise, an autonomous racecar exceeds the requirements for the software to a vast extent in comparison to an average passenger car - which provides many learning outcomes, research questions, and new algorithmic developments [3], [4]. It is also worthy of note that the requirements for autonomous racecars versus passenger vehicles can be quite disparate - precluding a direct transplant of a complete autonomous driving stack. Therefore, autonomous racing has emerged as a field where advanced algorithmic approaches are tested and then individually transferred to the development of autonomous passenger vehicles - similar to classic motorsport.

This paper provides the first survey of state-of-the-art research in the field of autonomous vehicle racing. By summarizing, classifying, and evaluating the different software and hardware developments, we provide a holistic overview of the research in this field. Finally, we discuss future research directions by highlighting open questions and challenges in autonomous racing.

A. Contributions

In this survey, we present the efforts and research conducted in recent years in the field of autonomous racing. This work has four main contributions:

1. We provide the first survey to comprehensively cover the topic of autonomous vehicle racing for both software and hardware developments.

2. We extensively review all research papers that developed new autonomy software for autonomous racecars. By splitting this software review into subsections of perception, planning, and control, we detail which methods and approaches were used. We compare the different approaches and explain their algorithmic setup. Furthermore, we discuss recent efforts made with techniques from the field of deep neural networks (DNN) and reinforcement learning (RL) to achieve a partial or full end-to-end pipeline for autonomous racing.

3. We review the current autonomous racing competitions, which provide hardware, a racing environment, and an organization (e.g., sports and technical regulations). We compare the different racing series and hardware against each other and give a holistic overview for potentially interested researchers.

4. Finally, we list open research questions and challenges in the field of autonomous racing. We discuss that these open challenges can be applied to the area of passenger cars, too, and provide opportunities for future researchers to work on relevant research topics.

B. Preliminary Remarks

1) Definition: Autonomous Racing

Although the term autonomous racing can be referred to different applications (e.g., drone racing), we focus on research in the field of autonomous racing cars. These racecars need to have four wheels, can either have a combustion engine or electrical engine as the main power unit, can be real racecars (e.g., Formula 1 car) or small-scale vehicles (e.g., 1:10 scale). In addition, the software and hardware surveyed here must have a clear connection to the field of automobile racing. This means that the authors of these papers either used specific hardware that is acting in a racing environment (e.g., racetrack, adversarial setup) they used a particular simulation that displays a racecar within a racing environment (e.g., a racing game for PC) or their research demonstrates specific solutions for a racing problem (e.g., driving fast autonomously around the racetrack). Although some authors present results and algorithms for high-speed autonomous driving on the freeway, this work is not covered in this survey because neither the aspect of handling the vehicles at the limits nor the adversarial context is given here sufficiently.

2) Research Categorization

The field of autonomous racing provides plenty of development and research categories. For this survey, we want to display both software and hardware efforts in the area of autonomous racing and therefore, we use the following perception - planning - control pipeline [239] depicted in Figure 1 to categorize the research papers.

FIGURE 1.

Autonomous driving pipeline including both hardware and software that provides the categorization for the survey in the field of autonomous racing.

Research and developments in the field of autonomous racing hardware (sensors and vehicle hardware) are discussed in Section III. Unfortunately, we have not seen any specific developments of sensors for the purpose of autonomous racing and therefore, we heavily focus on the used racing vehicle platforms. By presenting vehicle setups (sensors, computation hardware, racing environment), we provide a clear overview of which hardware is available for researchers. The biggest part of this survey paper presents research and software in the field of perception, planning, and control in Section II. In Section II-A perception, we cover all algorithms that provide either a solution for mapping, localization or object detection. In Section II-B planning we display global and local trajectory and behavior planners. The final Section II-C is used to present algorithms in the field of control and displays the solutions for path and velocity tracking at the handling limits. Unfortunately, many papers out there act on the intersection of planning and control. Those papers have no clear distinction in which field they belong and therefore, we decided to categorize them in either field based on their focus. The method of DNNs has become popular in recent years and different authors proposed so-called end-to-end approaches that solve the autonomous driving task. These techniques are described in Section II-D. In addition, some authors proposed evaluations with racecars, complete software pipelines, modeling efforts, and simulation environments for autonomous racecars that do not fit in the proposed categories. Those papers are listed in Section II-E. In summary, this survey covers all research papers in the field published until the beginning of 2022.

SECTION II. Autonomous Racing Software

A. Perception

Perception is the general term for all algorithms that perceive the environment and derive knowledge about it. In particular, perception includes detecting objects, detecting the free space, mapping the environment as well as localizing the autonomous vehicle. In an autonomous racing environment we deal with high speeds and therefore the question arises: How fast is too fast? Falanga et al. [56] tried to answer this question for autonomous robots with an additional case study on autonomous quadrocopters. The authors conclude that the maximum latency an autonomous system can tolerate to guarantee safety (not crashing in an object) is related to the desired speed, the agility of the system (e.g., the maximum acceleration it can produce) and the perception parameter of the sensors (e.g., the sensing range). For autonomous racecars the same parameters can be taken into account but no particular evaluation regarding high speed perception for autonomous racecars has been done yet. The current state of the art in the field of autonomous racing perception is summarized, categorized and displayed in Table 1.

TABLE 1 Overview of Research in the Field of Autonomous Racing Perception

A racetrack normally provides some specifics that can not be found on normal streets. As depicted in Figure 2, a racetrack consists of a single lane that is the main driveable space. This lane is defined by an inner and outer bound that can consists of additional curbs in the turns. On both left and right side of the track there are zones consisting of grass, gravel or tarmac (run-off area) where racecars can drive in an evasive maneuver or if they miss the race line. The racetrack is finally surrounded by walls. Depending on the racing series these types of the racetrack features can vary (e.g., Formula E: no gravel or grass). We define the fundamental problems for autonomous racing perception as the following:

• High speed object detection.

• High speed localization and state estimation.

• Localization on wide areas without specific landmarks.

• Precise localization information necessary to achieve high dynamic trajectory planning and control.

FIGURE 2.

Racetrack with environmental specifics: Inner-and outer bounds, racecar objects, walls and run-off area.

Although the racetrack provides a very simple structure with a single driveable lane, the long distance to the walls and non-existing landmarks make this environment quite difficult to perceive. None of the cited papers use predefined High-Definition Maps (HD-Maps) that are known from passenger autonomous driving development. An open-source library [79] of racetracks provides a simple 2D-birds-eye-view with inner and outer bounds ($x$ -and $y$ -Position) of about 30 racetracks around the world that can be used for planning but not for localization. In Nobis et al. [136] an adaption and enhancement of well-known simultaneous localization and mapping (SLAM) algorithms (Google Cartographer [240], GMapping) is described to create maps of large-scale outdoor environments. Palafox et al. [144] use a vision-based method to detect the free space when no lane lines are present by only using camera images and depth information as input for a DNN.

Most perception research for autonomous racing is listed in the field of localization techniques. Although many research vehicles are equipped with differential GPS (dGPS) that delivers a high localization accuracy, the goal of many autonomous racing researchers is to deliver software based localization solutions only. Both [28] and [70] use a 1:10 scale vehicle for their localization techniques. While Brunnbauer and Bader [28] use the camera to detect cones that create features to enhance the odometry localization, Gotlib et al. [70] map the track with an onboard 2D-LiDAR and run a Robot Operating System (ROS) based Adaptive Monte Carlo Localization (AMCL). The same AMCL approach is also used and adapted by Stahl et al. [177] to run on the Roborace research vehicle. By using pre-generated maps based on LiDAR data, the car achieves a mean absolute lateral error of 0.086m at a velocity of 150 km/h. A comparison to this work is done in [213] were odometry, GPS and LiDAR data is fused in a Kalman Filter (KF) based on a purely kinematic vehicle dynamics model to achieve localization at high speeds. The Roborace vehicle is also used for the localization research of Renzler et al. [158], Zubaca et al. [224] and Schratter et al. [229].

To increase the localization performance at high speeds the distortion of the LiDAR measurement is analyzed and a compensation is proposed in [158]. It is shown that this correction can be straightforward and has a high performance with objects moving at faster speeds. The work from Zubaca et al. [224] presents an extended $\text{H}\infty $ Filter (EHF) based on a kinematic motion model assuming constant turn-rate and acceleration to fuse LiDAR, IMU (inertial measurement unit), and vehicle dynamic sensors’ measurements. The proposed EHF shows slightly better estimation performance in high dynamic driving scenarios in comparison to an extended Kalman Filter (EKF). In [229] a complete process for both mapping an localization on racetracks with the Normal Distributions Transform (NDT) method is displayed. Based on this approach the Roborace vehicle reaches up to 122 km/h and an average localization error of 0.2 m. Massa et al. [129] are using two multi-rate EKFs and an extend AMCL that exploits some a priori knowledge of the environment on the Roborace vehicle. The authors showed that the pose error heavily depends on the car’s velocity, and varies in average from 0.1 m (at 60 km/h) to 1.48 m (at 200 km/h) laterally and from 1.9 m (at 100 km/h) to 4.92 m (at 200 km/h) longitudinally.

For Formula Student Driverless (FSD) vehicles Le Large et al. [118] show a comparison between GraphSLAM and an EKF-SLAM. Based on their experimental analysis with the FSD vehicle and the maps generated by the algorithms they showed that GraphSLAM outperforms EKF-SLAM in terms of accuracy. In [13], [69], [194] the localization and mapping approaches for the FSD vehicle of the AMZ team is presented. While in [69] only a LiDAR based SLAM was used, the student team extended the work with an additional LiDAR and camera based object detection for cones on the track [13]. With this setup the team achieved a velocity of 10 m/s while doing mapping, localization and planning at the same time with an RSME error of 0.29 m. On the same vehicle a recurrent neural network (RNN) was applied [175] to derive an accurate velocity estimation. By taking different vehicle sensors (e.g., IMU, wheel encoder) into account this learning based approach reached $15\times$ better performance than an EKF approach with an RSME of $v_{x}$ = 0.141 m/s and $v_{y}$ = 0.059 m/s. Finally, for the IAC vehicle Sauerbeck et al. [168] provide a multi-sensor localization approach by focusing on decoupling the lateral and longitudinal position of the vehicle. With this approach the vehicle reached 300 km/h with an average deviation of 1.471 m in simulation. The current state of the art in autonomous racing is heavily based on single vehicle races. Therefore the subcategory of object detection algorithms for high speed applications was not given much attention. Nevertheless, in the FSD competition teams need to detect both color and form of cones to let the vehicle drive autonomously as depicted in Figure 3. In [47] a case study with different convolutional neural network (CNN) methods (Tiny-YOLO, Proteins) are done in comparison to a YOLO v2 setup [48] to display the best approach for cone detection in the FSD scenario. Strobel et al. [182] present a combination of a YOLO v3 based object detection, pose estimation, and time synchronization that uses data from both stereo and monocular cameras. Furthermore, besides these software focused developments the authors of [153] evaluated the performance and energy consumption of popular, off-the-shelf commercial devices for DNN inference in the formula student context. Finally, to help and support other FSD teams in their development, Vödisch et al. [49] presented a collaborative dataset for vision-based cone detection systems that is open-source available.

FIGURE 3.

Exemplary cone detection in the Formula Student Driverless competition. The racetrack is defined by a left boundary (blue cones) and a right boundary (yellow cones) which need to be detected by the teams. This image is part of the FSOCO dataset [49] the supports FSD teams with the development of their perception pipelines.

B. Planning

In the following subsection we cover algorithms that plan trajectories for the autonomous race vehicle to drive around the racetrack. Strategies that drive the vehicle end-to-end directly from perception to actuation is excluded from this part and is discussed with further details in Section II-D. We split the discussion into the three following parts.

1. Global planning provides an optimal path, better known as raceline (depicted in Figure 4), around the racetrack. In the context of racing, global planning often optimize for the lowest lap time. Therefore, when following this raceline, the car drives an optimal path around the racetrack – under the constraints of the raceline generation – as fast as possible.

2. Local planning (or motion planning) plans on a finer granularity compared to global planning, usually under the assumption that an optimal global trajectory is provided. Local planners operate in a certain time horizon, and aim to avoid obstacles while still provide a fast and reliable path that does not deviate too much from the optimal global raceline.

3. Behavioral planning provides information about the high-level mission planning of the racecar. This can include the decision making about overtaking maneuvers (overtaking left/overtaking right/stay behind), the energy management strategy, interaction with other vehicles and the reaction to inputs from race control (e.g., flags, speed limits). As a summary, Table 2 provides an overview of research efforts in the field of planning for autonomous racing. We define the fundamental problems for autonomous racing planning as the following:

   • Minimum-time optimization for a global optimal raceline.

   • Long local planning horizon for recursive feasibility.

   • Obstacle avoidance and vehicle reaction at high speeds.

   • High re-planning frequency for real-time capability.

   • Decision making under high uncertainty.

   • Interaction planning with non-cooperative agents.

TABLE 2 Overview of Research in the Field of Autonomous Racing Planning

FIGURE 4.

Theory of raceline (blue) and slow line (orange) on the racetrack.The raceline provides the global fastest path around the complete racetrack.

(i) Global Planning: Research from the field of global planning can be roughly divided into different strategies using the objectives of the overall optimization: lap time, geometric properties of the race lines, or energy spent. Racing, as a context for optimization, provides a clear measure of quality in lap time $t_{lap}$ on participating agents. So naturally lowering $t_{lap}$ is a popular choice when it comes to global planning (Figure 5).

FIGURE 5.

Comparison of global optimal raceline algorithms based on shortest path, minimum-curvature [78] and minimum-time [43] optimization, which lead to different lap times $t_{lap}$ and trajectories.

A first category of global planning approaches is the usage of variations of Evolutionary Algorithms (EAs) to optimize for lap times. This approach is used in [25], [141], [154] with different parameterizations of the search spaces. In these category, an individual “gene” models a complete configuration of the racing environment, and sometimes vehicle hardware and software. The algorithms also require evaluation functions to gauge the performance of an individual; here, this is the simulated lap time given a configuration. Initially, a pool of genes (referred to as a population) is created by sampling the search space. Then, in each iteration of the algorithm, genes are evaluated, and mutations are performed following different strategies depending on the specific algorithm. Eventually, the individuals in the population should converge, and a best configuration for the global optimal raceline is found.

Another popular approach when optimizing for lap times as the objective function is to form an optimal control problem (OCP), usually non-linear. The OCP uses the race vehicle’s lap times as the objective function, and respects constraints of the geometry and friction limits $\mu _{max}$ of the race track as well as the dynamics and control limits of the racecar.Proposed approaches usually choose different vehicle dynamic models, and solvers to solve the optimization problem. Metz and Williams [130] formulate an OCP where the objective is the lap time, and solutions are found by quasi-linearization with integral penalty functions, and splicing of constrained and unconstrained arcs to form a two-point boundary value problem. Kelly and Sharp [106] use Sequential Quadratic Programming (SQP) to solve the non-linear programming problem where lap time is the objective. Rucco et al. [163] formulate an OCP where the objective is the lap time. By reformulating the objective with eliminating explicit time step terms, the problem becomes a fixed-horizon free-endpoint problem. Projection operator-based Newton’s method is ultimately used for the trajectory optimization. Theodosis and Gerdes [189] initialize the optimization problem with a path created by connecting center line on the straights and clothoids between the center lines. The sequential gradient based, non-linear optimization then uses the lap time as the cost function to find an improved global race line. Pagot et al. [143] present a non-linear model-predictive framework to formulate a optimal control problem where time is the objective. Vazquez et al. [196] formulate a minimum-time optimal control problem by using the centerline of the race track and discretize the continuous space dynamics. Two regularization terms with slip angle cost and a control input rate of change cost are also included in the objective function for reducing the lap time. Hermann et al. [84]–​[86] formulate an optimal control problem where the lap time, the path, the velocity profile and the energy consumption is included in the objective function. By formulating a multi-parametric SQP it is possible to find the optimal velocity plan along a race line while not violating dynamic and energy requirements of the drivetrain. Finally, the authors of [127] devise a apex-finding method and calculates the optimal time global race line by solving an OCP.

Some researchers show approaches of calculating the race line by satisfying certain geometric properties. Usually, the assumption that vehicles experience lateral acceleration that results in lateral tire forces is made when it comes to autonomous racing. It is then often desirable to minimize the lateral acceleration to minimize the possibility of side slip. In these cases, the aim is to find a race line with the minimum-curvature overall. However, it is widely known that the least curvature path is not the ideal path for a racecar which is asymmetric in braking and acceleration and is operated in a combined slip range. Braghin et al. [26] create design and solve a dynamic problem to find the best compromise between the shortest track and the least curvature track based on the vehicle’s dynamic behavior. Similarly, Cardamone et al. [38] also try to find the compromise between the same conflicting objectives, but use a Genetic Algorithm (GA) to find the best weighting parameter between the two objectives. Heilmeier et al. [78] extends the work of Braghin et al. [26] to solve a quadratic optimization problem where constraints on vehicle dynamics limits are set up to find the minimum-curvature path around the race track. This approach is compared to the minimum-time optimization of [43]. For the same racetrack [78] achieves a laptime of $t_{lap}$ = 86.13 s while [43] is even faster with $t_{lap}$ = 84.90 s. The advantage of the minimum-curvature planning is obviously the reduced set of parameters for the vehicle dynamics model and the faster calculation time.

Lastly, some approaches also choose to mimic the geometric properties of a race line driven by human drivers. These approaches often decomposes a turn into different segments and require race lines to satisfy different properties in each sections. Kuhn [113] mimics the behavior of a racecar driver by defining the same important decision making points on the track. It calculates the race line by first fixing the locations of the following points: the braking points, turn-in points, apex points, turn-out points, and accelerations points. Then a piecewise rational spline function is used to interpolate all the points and create a race line. Theodosis and Gerdes [188] mimic the three phase cornering technique used by professional drivers. It first finds all straights along the track, and linking the straights with curve structures. Then, each connecting curve is found by combining clothoids and a circular arc. The parameters are adjusted to minimize the overall curvature and ensure the path is tangent to the following straight.

(ii) Local Planning: In local planning, the main objective is to plan the cars motion for a fixed horizon by avoiding collisions with either the environment or adversaries (Figure 6). There are three main strategies:

1. Modifying the global plan via optimization.

2. Sample multiple dynamically feasible trajectories and select the best one around obstacles.

3. Sample in the free space around obstacles to find a feasible trajectory.

FIGURE 6.

Local trajectory planning on the racetrack: The vehicle needs to prediction the opponents motion, plans a feasible trajectory around the opponent and stick closely to the global raceline.

From the state of the art we could deduce that many authors use Model Predictive Control (MPC) methods for local trajectory planning. Although this method is control technique, it is also suitable for planning a local trajectory. In this section, we present work that is on the very thin and fluid boundary between planning and control, but mainly addresses the planning problem. Pure path tracking with MPC is described later in Section II-C.

In the first category, the global plan is modified to allow for obstacle avoidance. In these types of formulations, model predictive controllers are usually utilized to optimize the global plan. Upon encountering an obstacle or opponent vehicle, the constraints or cost functions of the optimization problem is modified, and a new motion plan is formulated. Anderson et al. [12] switch between two MPC modes to optimize for minimum-time objective or maximum velocity objective to mimic a professional driver. Kapania et al. [102] first find the optimal velocity profile given a reference path, and then updates the given path with the fixed velocity profile to find the minimum-curvature path by solving a convex optimization problem. Williams et al. [206] propose a sampling based MPC that relies on path integral control for entropy minimization. In [63] a planner is presented that is capable of mediating between conflicting objectives when performing collision avoidance, vehicle stabilization, and path tracking. Subosits and Gerdes [183] present a real-time trajectory planning algorithm by approximating a re-planning problem as a convex quadratically constrained quadratic problem (QCQP) with a simplified point-mass model. Alcalá et al. [9] reformulate the non-linear vehicle dynamics in a Linear Parameter Varying (LPV) form. This can then be used to create a convex optimization problem which is easier to solve for search for a local path. Kalaria et al. [233] use a nonlinear-MPC (NMPC) for local planning where the objective consists of progress along the race line, avoiding collision, and use drafting (reduce drag) to make progress. Brüdigam et al. [236] use a Gaussian Process (GP) to predict the opponent’s maneuver. These stochastic information is then used in a Stochastic MPC to plan efficient overtaking maneuvers.

In the second category, multiple motion primitives, or prototype motion plans, are generated by forward simulating the vehicle dynamics given the current state of the vehicle using multiple different actuation input sequences. This usually results in multiple splines or arcs to select from. In addition, cost functions are used to give each primitive an attached value. With a search for the best (lowest/highest) cost in these primitives a final trajectory can then be chosen. Liniger et al. [119], [120] generate a library of trajectories by forward simulating the vehicle using a grid of vehicle velocities and steering angles up to a certain horizon. Stahl et al. [178] propagate the race track with a graph that covers the entire space. The nodes are first placed equidistantly along cross sections of the race track, then edges connecting nodes are created by optimizing for cubic clothoids. This planner was tested on the Roborace vehicle and achieved $v_{max}$ = 223 km/h with an update rate of 16.8 Hz on and NVIDIA Arm electrical control unit (ECU). O’Kelly et al. [141] use a uniform grid of points along the global race line as local goal points. Afterwards cubic clothoids are optimized to connect the vehicle’s current state and the grid points which leads ultimately to planning a local trajectory. Finally in [171] a set of local goal points in front of the vehicle is sampled with the help of a normalizing flow method. Again, cubic clothoids are optimized here to connect the vehicle’s current state and the grid points to derive a driveable trajectory.

In the third category, sampling-based methods are used. These approaches randomly sample the free space around the current state of the vehicle for goal states. Once an available goal state is found, a motion plan is then generated connecting the current state of the vehicle with the selected goal state. By introducing randomness in the sampling process, these algorithms are usually efficient, but do not provide guarantees on their optimality. Jeon et al. [94] combine the rapidly-exploring random tree (RRT*) method with a local steering algorithm utilizing the dynamic model of the vehicle. Arslan et al. [14] combine RRT with closed-loop prediction based on the vehicle model, and incorporate relaxation methods for efficient construction of a tree that guarantees asymptotic optimality. Feraco et al. [57] combine RRT with Dubins curve to generate dynamically feasible local plans. Finally, Bulsara et al. [31] use RRT to find collision free reference paths in the free space.

(iii) Behavioral Planning: In behavior planning, the focus is usually on high-level decision making on tasks such as selecting an appropriate weighting of different objectives, or selecting plans that impedes the progress of opponents. The research in this area mainly focuses on two different strategies:

1. Assigning multiple cost functions with weighting and selecting the plan with the lowest overall cost.

2. Combine the local planner with game theoretic methods.

In the first category, cost functions are used that represent specific racing values like progress along the track, proximity to the obstacles, effort for control inputs and the deviation from optimal global plan. An overall cost is then found by combining all cost functions for each candidate local trajectory. Cost functions could also incorporate hard constraints by eliminating unqualified plans. Finally, the trajectory with the minimum overall cost is chosen to be the local trajectory. Liniger et al. [119] use the prototype trajectory without collision and makes the largest progress along the track. This approach is extended in [120] by applying viability kernels on the track which only generates viable trajectories. O’Kelly et al. [141] assign cost functions representing proximity to the global plan, collision with the environment to prototype trajectories and select the best one. Finally, Sinha et al. [171] assign cost functions for progress along the track, overall curvature, maximum velocity, and collision with predicted opponent motion to all prototype trajectories.

In the second category, game theoretic approaches are used to usually find the best action in a two or multiple player game. The continuous motion planning problem is usually transformed into a step-by-step game where each player is allowed to make a “move” one by one. These approaches usually incorporate the concept of regret to try to find the best response for winning the racing game either immediately or in the long run. Williams et al. [207] combine a best response model of the opponent behavior with variation of MPC by including predicted opponent trajectories into the cost of other vehicles. Notomista et al. [137] propose sensitivity-enhanced Nash equilibrium seeking, which uses iterated best response algorithm to optimize for a trajectory in a two car racing game. In [202] a iterated best response with Nash equilibrium approximation is used to plan receding horizon trajectories. This technique helps to maximally advance the racecar along the track while taking into account opponent’s intentions and responses. Sinha et al. [171] build a library of opponent prototypes offline and uses the EXP3 algorithm to solve for a multi-armed bandit problem to approximate the current opponent’s driving policy by using the library. Liniger and Lygeros [124] repeat the multi-player game in a receding horizon fashion, which results in a sequence of coupled games. With this non-cooperative game approach the authors could show that the vehicles create blocking maneuvers although the risk of a collision gets higher. Wang et al. [203] use sensitivity enhanced iterated best response to seek convergence to the Nash equilibrium in the joint trajectory space for all agents. Finally, Schwarting et al. [170] use local iterative Dynamic Programming (DP) in belief space to solve a continuous Partially Observable Markov Decision Process (POMDP).

C. Control

In the previous subsection, we discussed how to compute either a global or local trajectory on the racetrack. The trajectory includes both a path ($x(t)$ , $y(t)$ - position) and velocity profile $v(t)$ which provides the reference information for the lateral and longitudinal control. In what follows, we provide control methodologies that leverage such a reference trajectory to compute control actions to navigate the car along the waypoints. The goal is to reduce the lateral and heading error to stay as close to the reference line and to reduce the velocity error to be as fast as possible (Figure 7).

FIGURE 7.

The goal of the control task in the autonomous racecar is to reduce lateral and heading error while following a global optimal raceline as a reference trajectory.

FIGURE 8.

Qualitative comparison of an LQR, MPC and Tube MPC controller in an autonomous racing setup based on [216]. While the MPC is outperforming the LQR controller, it can be seen the Tube-MPC is not violating any dynamical constraints, e.g., the maximum lateral and longitudinal acceleration.

At this level of abstraction, the control actions are usually the steering angle $\delta $ and a throttle or brake request (acceleration commands $a_{long}$ ) that are sent to low-level controllers for actuating the motor and the brakes. We denote $x^{\mathrm {ref}}(t)$ as the state associated with the reference trajectory at time $t$ , and our goal is to design a controller policy $\pi $ that, given the current state of the vehicle at time $t$ , denoted as $x(t)$ , and the reference state $x^{\mathrm {ref}}(t)$ , computes the control input $u(t)$ .

Table 3 gives an overview of the papers that address the control problem for autonomous racing. The control papers try to address the issue of “handling at the limits” and follow the raceline/reference trajectory as accurate as possible. We define the fundamental problems for autonomous racing control as the following:

• Highly accurate path tracking for low lateral errors.

• Highly accurate path tracking for low heading errors.

• Highly accurate velocity tracking for fast lap times.

• Stable vehicle behavior at high accelerations.

• Exact modeling of the nonlinear vehicle behavior.

• High control frequency for real-time high speed driving.

TABLE 3 Overview and Categorization of Research Paper in the Field of Control for Autonomous Racing

Since the accurate control of the vehicle is relies heavily on precise vehicle dynamics modeling, we include the vehicle dynamics models used in each case. Authors use either a point mass model, a bicycle (single-track) model, or a four-wheel (double-track) model. Furthermore, to capture the lateral behavior, the authors use the bicycle model either with only the kinematic behavior (no lateral forces), linear behavior (linear tire forces based on cornering stiffness), or a nonlinear behavior (nonlinear tire forces based on the Pacejka Magic Formula or Fiala Tire). For a better overview and understanding of the different types of control research we categorized the papers into six subsections.

1. Classic Control: First, we survey papers that cover classical and well-known principles in the field of path and velocity tracking. Ni and Hu [133] present a path following controller for their FSD vehicle. Their overall controller architecture consists of longitudinal, lateral and yaw controllers that operate the vehicle on a predefined G-G diagramm (maximum lateral and longitudinal acceleration.) In both [185] and [186] we see the usage of a simple lookahead controller that provides path tracking at high speeds event at the limits of tire adhesion. In [111] an autonomous racing controller is presented that uses the vehicle’s centre of percussion (COP) to design a feedforward and feedback steering. This showed how to simplify the equations of motion and highlights the challenge of controlling a vehicle with highly saturated tires. Furthermore, a special focus on path tracking at the tire/vehicle limits is presented in the work of [59], [100], [112], [114]. While the research of [59] and [112] displays the usage of a G-G diagram to display controllers that operate the vehicle at the limits, Laurense and Gerdes [115] presents a slip angle-based control strategy to maintain the front tires at a certain slip angle to create the maximum tire forces. A special focus on longitudinal control (speed control) is shown in [110], [114], [148]. Laurense et al. [114] presents a control framework for full tire-force utilization with slip-angle based steering control, combined with explicit control of the path-tracking dynamics through longitudinal speed feedback to achieve a better path tracking. In [148] a model-free nonlinear controller for longitudinal speed control is presented. In this approach, a dynamic reconstruction of information on the vehicle’s motion concerning the inputs acting on the system with sensor data is displayed. With this its possible to reconstruct the maximum longitudinal tire forces for current states which can be used for accurate speed tracking. Finally, in both [198] and [199] additional sensitivity analysis of path controlling at the limits and high sideslip maneuvers are displayed.

2. Model Predictive Control: The second and most-popular strategy used to realize an autonomous racing controller is Model Predictive Control. In MPC a sequence of control actions is computed by forecasting the future trajectory of the vehicle over a short time window. In particular, given the state of the system $x_{t}$ , an MPC solves a Finite Time Optimal Control Problem (FTOCP) to compute an optimal sequence of states $\{x_{t}^{\ast}, \ldots, x_{t+N}^{\ast}\}$ and inputs $\{u_{t}^{\ast}, \ldots, u_{t+N-1}^{\ast}\}$ over a fixed horizon $N$ . In autonomous racing the objective of the optimal control problem is to either track a global reference trajectory or to minimize the lap time. Upon computing such sequence of optimal states and actions, the first control action $u_{t}^{\ast}$ is applied to the system and the process is repeated at the next time step based on the updated state $x_{t+1}$ . MPC-based methodologies are the main method behind several autonomous racing controller which have been implemented on real vehicles. The advantages of MPC are that 1) forecast is used to act proactively and to 2) feedback is naturally incorporated in the controller that repeatedly updates the optimal trajectory. Notice that when the planning horizon $N$ is short, the planned trajectory may not account for the future behavior of the system and as a result the controller may take shortsighted control actions. However, computing such quantities that exactly approximate the cost and constraint beyond the prediction is challenging. In practical applications it is preferred either to use a long prediction horizon [121] or to approximate these quantities based on historical data [162].

In [206] a sampling based MPC algorithm is derived. This so called Model Predictive Path Integral Control (MPPI) algorithm is using the methodology of path integral control that derives an optimal control based on stochastic sampling of trajectories. It is demonstrated that this approach explicitly provides a formula for the controls over the entire time horizon and that it relaxes the usual condition between control authority and noise required in path integral control. The authors use this fundamental control approach and enhance it with additional decision maker [64], game theory [209], DNNs [51] and reinforcement learning [208] methods to derive further improvements.

Carrau et al. [39] present at sparse Randomized MPC (SRMPC) approach that is based on a Stochastic MPC. This approach is used to deal with model uncertainty at high speeds and high accelerations. While driving with the vehicle it collects data along the track which is then used to identify the model uncertainty probabilistically. This tightens the constraints for the MPC automatically while still having the size and structure of a standard MPC problem. The optimization problem is solved in 20 ms for a 1:43 and the results show that for a desired violation the controller achieves faster lap times and fewer constraints violations than a standard MPC algorithm. Reference [121] is building on top of this approach and enhancing it with disturbance feedback policies to optimize over the state feedback matrices.

In order to capture the nonlinear dynamics of the vehicle and tires a nonlinear MPC (NMPC) can be designed and modeled [27]. Although this is computationally expensive, with the help of nonlinear optimization solver like FORCES PRO [244] these type of optimizations can be solved in real-time on the vehicle. In [138] a hierarchical NMPC is presented that consists of two controllers: Firstly, a high-level NMPC with point-mass model that simplifies the vehicle dynamics and is constraint by the tire G–G diagram. Secondly, a low-level NMPC with a high-fidelity model uses the output (velocity profile) of the first NMPC as a terminal constraint. This method helps to reduce the prediction horizon and therefore calculate the vehicle dynamics in real-time. Furthermore, this approach was improved in [196] with a simpler vehicle model to run on a FSD vehicle.

Li et al. [228] use a NMPC with a minimum-time objective and a collision-avoidance constraint. By applying a Mixed Integer Quadratic Programming (MIQP) method a control strategy is created that is optimized regarding the safety and the laptime. The authors conclude that for such an approach the prediction horizon needs to be large enough for creating feasible results although this leads to a higher computation time and is therefore non real-time capable. The authors of [123] combine a low-level MPC with a viability kernel that efficiently generates finite look-ahead trajectories to maximize the progress of the car. At the same time the viability kernel creates trajectories that remaining recursively feasible with respect to static obstacles. The authors apply this algorithm to both a simulation where the effects of various design choices and parameters are identified and showed that a hierarchical controller can be improved by incorporating the viability kernel in the trajectory planning phase.

Beal and Gerdes [19] use estimations of the friction coefficient and vehicle sideslip to define state constraint and unstable vehicle behaviors. This information is utilized in an model predictive envelope controller to create a region of stable vehicle motions. With this approach it is possible to operate the vehicle on the handling and stability limits. Similarly, Wischnewski et al. [216] and Williams et al. [211] present a Tube-MPC (TMPC) approach where nonlinear effects and external disturbances are taken into account of the MPC design. By approximating a tube of reachable sets over the prediction horizon the vehicle guarantees a space of constraint satisfaction. Finally, because tuning the parameters of a controller is time-consuming and needs experience from experts some researches try to automatically tune the parameters with optimization techniques. In [141] the superoptimization toolchain TUNERCAR is presented which is using a Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to optimize both vehicle hardware parameters (center of gravity, mass) and control parameters (P-, I-, D-parameters) of the car. The evaluation is based on the laptime of the car and the algorithm shows the capability of reducing the laptime driven by the car by optimizing these parameters.

(iii) Learning Based Control: An additional control strategy to improve the tracking of a reference trajectory is to leverage Iterative Learning Control (ILC) based methodologies [6]. ILC methods are useful for applying them to an autonomous race vehicle, since the vehicle is running on the race track repeatedly for multiple laps. In this case the tracking error and vehicle data from previous laps can be used to compute a feed-forward correction term that improves the path and velocity tracking performance significantly (Figure 9).

FIGURE 9.

Qualitative demonstration of a learning-based control approach based on [101]. For each new lap the car is decreasing the tracking error by applying a higher correction steering angle for each vehicle movement.

ILC-based strategies for autonomous racing have been successfully implemented on full-size vehicles [101], [103]. In [30], [159]–​[162] a Learning MPC (LMPC) is proposed which is an optimization and data-driven framework to make the car faster every lap and therefore reduce the lap time. The optimal control problem of the MPC is enhanced in a way so it tries to compute a solution by solving at time $t$ of each lap the finite time constrained optimal control problem. This creates a convex optimization problem which can be solved with respective solvers. The authors show that the LMPC finds a faster trajectory for each new lap while maintaining the set constraints.

Hewing et al. [88] present an learning-based cautious NMPC which aims to learn from vehicle sensor data with Gaussian Processes to improve the vehicle dynamics model. The GP model is used for regression to identify uncertainties and a mismatch in the vehicle dynamics model parameters based on measurement data. The NMPC is extended with this learning model and reformulated in a stochastic setting which improves the performance and safety of the vehicle at the same time. Furthermore, the authors implement this approach on a FSD vehicle in [98] and demonstrate the implementation and experimental validation of this kind of learning-based control approach. Finally, Jain et al. [92] use a similar approach but only with an extended kinematic vehicle model for the MPC to prove that this type of learning-based control can also leverage the usage of simplified vehicle models.

Because of the control system quality and unmodeled effects, it is well known that there is a gap between the planned and the driven trajectory. This gap is unknown and depended on the environment the vehicle is driving in. To mitigate this gap, [214] presents a learning control approach on the method of Gaussian Process for a nonlinear regression. This GP learns online, while driving, how big this gap is and then tries close it over the time by using a so called scale-factor. This scale-factor serves as an optimization variable that tries to maximize longitudinal and lateral accelerations each lap.

Finally, the authors of [95] proposed a control scheme that consists of a robust steering controller and a DNN. While the path tracking is done via backstepping variable structure control (BVSC) the DNN is integrated to estimate nonlinear functions, e.g., the uncertainty of tire cornering stiffness.

(iv) Drifting Control: Although it is not following an optimal raceline, racing head-to-head or striving for the fastest laptime, the field of autonomous drifting is a special subcategory for autonomous racing. Here, the researchers show algorithms that are able to maneuver the car autonomously beyond the stable handling of limits and stabilize the car in a point of high slip angle. In [89] a successive loop structure is presented as a controller that tracks only the vehicles sideslip based on the yaw rate as a control input. Goh et al. [65]–​[67] present a controller framework that is able to drift autonomously with the vehicle while tracking a predefined reference path. This enables drifting maneuvers at special references path, e.g., a circle or figure of 8. The authors using a single track vehicle model [65] and experiment with different variations of control values (sideslip [65], rotation of the vehicle’s velocity vector to track the lateral error [66], [67]) to reach a sideslip angle of up to −40 degree with speeds up to 45 km/h on a real test vehicle. Finally, Joa et al. [96] present a 3-level structure for a drift controller. First, they designed a supervisor which determines rate and rear longitudinal slip ratio for the drift maneuver. Second, a upper-level controller calculates lateral force (front) and longitudinal force (rear) for tracking the planned vehicle motion. Finally, a low-level controller converts the commands (forces) defined by the upper-level controller into control inputs for the vehicles (throttle, steering angle). With this setup the authors derived a steady state drifting on a real test vehicle.

D. End-to-End Driving for Autonomous Racing

The previous subsections described in detail which research efforts in the fields of perception, planning and control have been achieved for autonomous racing. Besides that, researchers have tried to set up partial or full end-to-end approaches to master the autonomous racing task. As displayed in Figure 10, in the context of autonomous driving end-to-end means that either partial modules or all software modules are completely replaced with data-driven approaches like a DNN.

FIGURE 10.

Classic autonomous driving software pipeline in comparison to partial and full end-to-end software pipeline.

The partial end-to-end approach aims towards replacing or combining modules with a DNN. This has the advantage that the DNN provides a low-dimensional intermediate representation of the racetrack (e.g., a trajectory) that can be then used in a classic control systems (e.g., PID-Controller). In contrast to a full end-to-end system the final actuator output (steering angle, throttle position) is not predicted directly. On the one hand the field of autonomous racing provides a perfect proving ground for end-to-end approaches: clear driveable area, no signage (e.g., traffic lights), one class of objects, clear objective for training (fastest laptime). On the other hand, when using end-to-end systems large data sets are needed to train the DNNs. This data must contain a wide variety of situations so that the DNN achieves a good level of performance. Similar to other DNN applications the generalization and performance of these systems are the biggest issues. In addition to these common known issues, we define the following main problems for end-to-end autonomous racing:

• Open to question system architecture design: partial vs. full end-to-end.

• Difficulty to learn the vehicle dynamics parameter - especially the nonlinear vehicle and tire dynamics.

• Training purely in simulation environments lead to simulation-to-reality gap.

• High amount of various data necessary for training the artificial networks.

• Out of distribution events can cause drastic failure cases: Driving at high speeds is rare and thus learning how to correctly react is difficult

In the following we present research efforts that use end-to-end approaches for autonomous racing which are summarized in categories in Table 4.

TABLE 4 Overview and Categorization of Additional Software in the Field of End-to-End Approaches for Autonomous Racing

Perez et al. [150] derives the control commands based on a rule-based evolutionary strategy. Although the car is driving successfully around the racetrack and follows a raceline the controller is only able to handle low speeds. In both [164], [166] the authors propose two fuzzy controllers for calculating the steering angle and computing the target speed of the car based on sensor information in the TORCS simulator [241]. In Oliveira et al. [142] Bayesian optimization (BO) is used to find a control policy that minimizes the time per lap while keeping the vehicle on the racetrack. The BO helps to search more efficiently over high-dimensional policy-parameter spaces an outperforms other evolutionary algorithms.

A solution for a partial end-to-end approach is presented in [204], [205], and [230]. The DeepRacing Framework is an end-to-end simulation environment and virtual testbed for training and evaluating algorithms especially for autonomous racing. In [205] three versions to control the racecar are presented: Pixel to control, pixel to waypoints, pixel to curves. It was shown that a partial end-to-end approach that provides parameterized trajectories based on a DNN outperforms a full end-to-end approach in terms of laptime and failures. Reference [116] shows the combination of MPC and CNNs to create a perceptual attention-based predictive control algorithm. With this, the MPC learns how to place attention on relevant areas of a visual input, which allows the vehicle to detect unsafe conditions faster. Drews et al. [52] are providing a framework that combines DNN based road detection as well as MPC to drive aggressively using only the sensor data from a monocular camera, IMU, and wheel speed sensors on the AutoRally vehicle. By combining CNNs and a Long Short Term Memory (LSTM) network the car is able to learn a local cost map representation of the track based on the camera input. This enables a global position estimation with a particle filter against a schematic map at high speeds. The local trajectory planning is afterwards done with the MPC. The authors of [128] provide an evaluation about the image sizes for a full end-to-end approach on a 1:10 scale vehicle. Based on their experiments the authors show that by decreasing the image size as an input for the end-to-end pipeline the car can drive faster and has a higher response time. Another evaluation for the usage of end-to-end algorithms is done by Wadekar et al. [235] in an simulation environment. The authors explored different data collection strategies with the goal of reaching high speeds and stable driving with the racecar. They conclude that even in the racetrack setup a high diversity and high amount of training data is necessary to achieve decent results.

Beside these plain usage of DNNs in the software pipeline, additional research efforts were done in the field of reinforcement learning. The autonomous racecar is seen as an agent that interacts with its environment in a continuous form. At each timestep $t$ the agent fulfills an action $a_{t}$ that leads to a reward $r_{t}$ as well as an observations of all environment states $s_{t}$ . Based on the reward $r_{t}$ the agent tries to maximize the sum of the rewards over time and therefore can learn a specific behavior in this environment. Autonomous racing researchers that develop RL algorithms are using either the F1TENTH Gym [139], the Roborace Simulator [80], the SVL Simulator [75] or TORCS [241]. All these simulation environments have an openAI Gym [243] interface that was created for the setup of RL developments. Both [93] and [151] address the problem of autonomous racing by applying the method of Advantage Actor-Critic (A3C) to a simulation rally racing game. A complete framework for the training of the autonomous agents in a distributed system with different tracks and road conditions is presented as well as the RL method for achieving the end-to-end driving. They generate reasonable results with a fast and reliable vehicle maneuver, especially on different road conditions, but the approach fails to generalize well. To create a better generalization for different racetracks de Bruin et al. [46] provide a combination of Q-Learning and state representation learning to display that this combination learns policies quicker and generalize better to new racetracks then single RL. Both [135] and [156] use the method of Deep Deterministic Policy Gradient (DDPG) to explore the usage of RL in autonomous racing in the TORCS simulation. In both experimental setups DDPG is specially enhanced for the usage on the racetrack and shows good learning and execution results. The application of Soft Actor Critic (SAC) with enhancement and variations is displayed in [42], [72], [172]. While [72] is only using a simulation, Chisari et al. [42] are applying this approach to 1:43 small-scale vehicles and compare the SAC method to a MPC path planner – the MPC outperforms the RL method. The work from Fuchs et al. [60], [172] is using the racing game Gran Turismo as both a training and evaluation environment. While in [60] the framework for training the RL agents is presented Song et al. [172] uses and enhances this approach not only to drive with a single vehicle but also with multiple agents. They show that their RL agent is capable of driving fast, following the raceline and overtaking other agents without crashing. Finally, Schwarting et al. [169] and Brunnbauer et al. [29] present model-based reinforcement learning approaches which can learn competitive visual control policies through self-play in imagination (World Models idea [245]). Especially in [29] it is shown that model-based RL approaches outperform non-model based methods. In addition, the authors display the sim-to-real transfer by testing the trained agent on a F1TENTH vehicle. The vehicle shows good generalization on unknown tracks but no high performance (e.g., low laptime) because of oscillating steering.

E. Applied Autonomous Racing Studies

In the final subsection all applied autonomous racing studies are displayed that clearly do not belong in the previous Sections II-A–​II-D. These research papers provide Evaluations that run either simulator studies or overall analysis in the field of autonomous racing. In addition, the efforts of Complete Software Stack developments are displayed here. In order to achieve the vehicle’s driving dynamics limits, there must be in-depth knowledge of the driving dynamics behavior and thus sufficiently good vehicle dynamic modeling. In the category of modeling the research that shows all vehicle dynamic modeling efforts for later usage in either trajectory planner or control algorithms is displayed. Finally, we present Simulation efforts and environments for autonomous racecars. A summary and overview of the research in those categories can be found in Table 5.

1. Evaluations: To gain more knowledge in the field of racing different researchers conducted studies with race drivers or racecars. Kegelman et al. [105] did a study with real (vintage) racing cars and collected vehicle and position data from their runs on the racetracks. By examining the statistical dispersion of the vehicles race lines, the author displayed a quantification of the repeatability of professional racecar driver performance. In addition, they concluded that different driving styles (combination of path and velocity) can lead to similar lap times. In [167] the raceline trajectory information (dGPS data) from a test vehicle is collected to derive a path fitting algorithm that is describing a raceline. Based on this setup the race lines can be analyzed in-depth and results for autonomous raceline planning can be derived. A direct comparison between autonomous racecar against a human race driver is both done in [157] and [82]. Remonda et al. [157] conduct this study in a simulator environment and compared the lap times, telemetry data and the performance level of a human race driver against a pure autonomous racing software based on RL. By doing this the researchers were able to analyze which features have the most impact on the drivers performance. Those features where used afterwards to enhance the RL approach. In contrast, Hermansdorfer et al. [82] conduct a real world study by comparing an autonomous racing stack on the Roborace vehicle against a professional Formula 2 driver to find indications where the autonomous car fails to meet the performance level of the human race driver. The main reasons are that the human driver is detecting the vehicle limits (tire limits) more accurate, bringing the vehicle more often beyond the limit (higher slip angle) and is applying both brakes later and throttle earlier. Finally in [24] an evaluation about a crash of an autonomous racecar is displayed.

2. Complete Software Stack: Although many researchers are just deploying a single algorithm for testing and evaluation, to fully run an autonomous vehicle a complete software stack consisting of perception, planning and control algorithms is necessary. Therefore, many publications aim towards designing a holistic autonomous software stack that describes the individual software components, the methods, the transfer of messages from one module to the other and a final evaluation on real hardware or simulation. In [7], [41], [44], [61], [98], [132], [191], [221] the efforts of developing an autonomous racing software stack for FSD vehicles are presented. Based on the tasks in the FSD competitions (Section III) these cars need to map the environment, localize themselves, plan the path on the fly and follow the path fast and reliable. The teams provide different concepts to solve those individual tasks and display at the same time the underlying computation hardware of the their autonomous race vehicles. In addition, the teams provide insights in the middleware (e.g., ROS) as well as computations times of their algorithms. In contrast to the FSD efforts the publications [22], [23], [37] show their autonomous racing software stacks for running the Roborace vehicle. In [23] the research is aiming towards a software that can operate in a multi-vehicle scenario and therefore displays a dynamic local trajectory planner as a main component. In addition, to achieve high dynamic trajectory planning maneuvers the team displays a Performance Assessment Module that is observing the controller and the tires while adjusting parameters accordingly. Caporale et al. [37] display a holistic planning and control stack that has a real-time NMPC as main backbone to track a pre-planned racing line as well as a mapping and localization approach for high speed driving.

3. Modeling: The modeling of the vehicle dynamics behavior of the racecar is an essential part in the field of autonomous racing. Either these models are used in the simulation environments or model-based trajectory planning/control design approaches. The current state of the art provides many variations of vehicle dynamics modeling such as single track model, double track model or full vehicle model. The more complicated the vehicle dynamics model, the more parameters are needed. Unfortunately not all of those parameters are available in detail for a vehicle and so different methods for estimating these parameters are proposed - especially for nonlinear vehicle parameters like the tires. In [218] the design of the standard joint-state Unscented Kalman Filter (UKF) is presented to estimated vehicle dynamics parameters of a model car both in simulation and with experimental data. The experimental results show satisfactory estimates of the model parameters. Unfortunately the tuning process of this algorithm is time consuming and can only be implemented offline. Park and Gerdes [147] describe the region of feasible tire forces mathematically with constraints on the limits of actuation. They conclude in their work that with reasonable assumptions, the border of feasible tire forces can be displayed by an ellipse and circle for a wheel with steering and braking actuators. The papers [83], [90], [173], [212] are using learning based approaches by applying DNNs to identify the model parameters. All of these works show that DNNs can learn and identify the vehicle parameters more accurately than a purely parametric model. In addition, the researchers showed that the DNNs can generalize better than a purely non-parametric model especially when it comes to capturing the unknown dynamics. This makes the usage of DNNs ideal for real-world applications where collecting data from the full state space for a vehicle is not feasible and when different environment dynamics (e.g., icy road) need to be captured in the model.

4. Simulation: The final paragraph in this subsection displays all simulation efforts that have been done in the field of autonomous racing. Espié et al. [241] are the authors of TORCS - The Open Racing Car Simulator. This lightweight 3D simulator provides different race tracks, different cars, NPC opponents as well as a sophisticated vehicle physics model. This simulator is used for research in the field of control, trajectory planning, game theory and RL and is therefore providing a solution for autonomous race engineers. Roborace released its own simulation environment that is enhanced with an OpenAI Gym interface especially for classical control or RL tasks [80] which they called Learn-to-Race (L2R). The simulation environment provides a racetrack, sophisticated vehicle physics simulation as well as a wide variety of sensors. The SVL Simulator [75] is a 3D end-to-end autonomous vehicle simulation platform that provides different maps, vehicles, sensor modeling, weather simulation, APIs to well-known open source software stacks (e.g., Autoware.Auto, Baidu Apollo) and the possibility of a distributed simulation. The SVL Simulator is offering both a 3D-vehicle model of the F1TENTH and the Indy Autonomous Challenge vehicle with different racetracks (Figure 11).

TABLE 5 Overview and Categorization of Applied Research and Development Studies in the Field of Autonomous Racing

TABLE 6 Autonomous Racing Hardware: Overview Over Different Available Hardware and Racing Competitions Available for Researchers

FIGURE 11.

F1TENTH vehicle in the SVL simulator [75] on a 1:10 scale version of the Indianapolis Motor Speedway.

For the F1TENTH vehicle different simulation environment exist. Babu and Behl [15] present a ROS and Gazebo based autonomous racing simulator that is providing different maps, visualizations and model physics. The advantage here is access to the ROS community that enables the integration of robotics packages. Another F1TENTH simulator is the F1TENTH Gym [140] that provides a lightweight, 2D-simulation with an openAI Gym interface. Based upon the Carla Simulator [242] the authors of [238] present an Autonomous System Operations (AutOps) and continuous integration (CI) and testing framework to evaluate the software in the context of autonomous racing. Especially for the evaluation of trajectory planning maneuvers in a multi vehicle environment Stahl and Betz [180] display an open-source graphical user interface that allows the fast generation of multi vehicle race scenarios. These dedicated scenarios (e.g., overtaking maneuvers) can be used to evaluate the trajectory planner or safety assessment algorithms in an autonomous racing stack.

SECTION III. Autonomous Racing Hardware: Vehicles and Competitions

The previous section gave an overview on the efforts in the field of algorithm and software development for autonomous racing vehicles. Almost all of the papers provided an evaluation of their proposed methods in an specific simulation environment. About half of the papers did additional evaluations on real vehicle hardware. This hardware is ranging from (powerful) passenger sports cars, specific research vehicle prototypes, small-scale race vehicles or real racing cars. In the following section we provide an overview of currently available hardware and racing competitions (Table 7) that are available for researchers.

TABLE 7 List of Abbreviations

A. Small-Scale Autonomous Racing Vehicles

The first type of autonomous racing vehicles are so called small-scale or reduced-scale vehicles. These racecars were mainly developed for the purpose of testing the new developed autonomous racing software. Those racecars are normally derived from remote controlled (RC) cars and therefore provide an electrical engine and a battery as a main powertrain unit. Those vehicles are then modified with additional hardware (sensors, ECUs), are constructed and maintained by a team of students and researchers and usually costs a few hundred to a few thousands of dollars. Although these are small-scale vehicles, they reach high speeds and accelerations for their size and therefore can be compared to real racecars.

1. 1:43 vehicles: In the ORCA (Optimal RC Racing)¹ project researchers from the ETH Zurich developed a test bed consisting of a race track, an infrared camera based tracking system and modified 1:43 cars, in order to apply research in the field of MPC algorithms at high speeds and in real-time. A vision system captures the cars on the track and estimates both positions and velocity of each car. This information is then sent to a specific control platform where the MPC controller calculates the control inputs for the cars. This information is then sent via Bluetooth to the embedded cars where the control input is actuated. The research published with this 1:43 cars is heavily in the field of planning and control. As a result the researchers displayed new developments in the field of MPC [39], [88], [119]–​[122], game theory [124] and reinforcement learning [42].

2. 1:10 vehicles: In the next bigger size researchers use modified 1:10 scale RC cars for their autonomous racing research. In the last years different institutions released their documentation for both hardware and setup of these 1:10 vehicles and so currently versions like the Berkeley Autonomous Racecar,² the MIT Racecar [104], the MuSHR racecar [174], the RoSCAR [76] or the F1TENTH [139], [140] vehicle from the University of Pennsylvania exists. The sensor setup on these cars is interchangeable and so it is possible to apply monocular cameras (e.g., Raspberry Pi, OpenCV OAK-1), stereo cameras (e.g., ZED, ZED2, Intel Realsense d435i, OpenCV OAK-D), 2D LiDARs (Hokuyo 10LX, Hokoyu 20LX), IMU, indoor GPS or wheelspeed sensors. As a main computation platform these vehicles use embedded GPU systems like the Nvidia Jetson (Models: TX1, TX2, NX, AGX Xavier, Nano). This gives the possibility to speed up the inference of DNNs. With the F1TENTH vehicle an additional, annual autonomous racing competition was launched where students, researchers and hobbyists can race against each other. The competitions consists of a single vehicle time trial and a head-to-head two vehicle race with knockout phase. In addition to these in-person competitions virtual competitions are organized to test the software of the developers. Similar competitions, where those kind of racecars or variations of it are used, are the DiYRobocar events or the Amazon DeepRacer [17], [18] competitions. The research published with these 1:10 cars is spread completely over all topics in perception [28], [70], planning [97], [232] and control [30], [31], [91], [92], [143], [162], [187], [237]. In recent years these type of vehicles got more important for optimization pipelines [141], [171], the application of RL techniques [29], [54], [55] and the evaluation of game theory methods [201], [203]. In addition, the 1:10 vehicles are used for education purposes [7], [15], [53], [104] to teach hands-on fundamentals of autonomous driving.

3. 1:5 vehicles: A special version of an autonomous small-scale vehicle is the so called AutoRally [68] vehicle, a 1:5 scale autonomous racecar developed by a team of researchers from Giorgia Tech. The AutoRally autonomous vehicle platform is based on a RC trophy truck (length: 1 m, width: 0.6 m, mass: 22 kg) with a top speed of ~ 90 km/h. The AutoRally vehicle uses two monocular cameras (Point Grey Flea3 FL3-U3-13E4C-C color) as a main sensors to perceive the environment, has an IMU for acceleration measurements and hall-effect sensors to measure the wheel speeds. In addition, this vehicle has a GPS receiver (Hemisphere P307) integrated which provides an absolute position at 20 Hz with an accuracy of approximately 2 cm under ideal conditions with Real-Time Kinematic (RTK) corrections that are derived from a GPS base station. The main computation unit consists of standard consumer computer components (Intel i7-6700 -3.4 GHz quad-core, 32 GB DDR4, Nvidia GTX-750ti SC) which are modular and reconfigurable and are all connected on a Mini-ITX motherboard.

   This brings the AutoRally setup closer to real passenger vehicles and allows a high computation power. The research published with the AutoRally car is in the field of planning [14], [117], [207], [219], [220] and control [52], [64], [116], [200], [206], [208]–​[212], state estimation [58], [218] and the application of deep neural networks for perception and planning [52], [145] with an overall special focus on low friction surfaces.

4. eV Grand Prix Autonomous: In 2021 a new racing competition called eV Grand Prix Autonomous started in the USA. Student teams need to acquire a standardized electric go-kart chassis and are then allowed to modify the vehicle. The teams can change the complete electric drivetrain and integrate new components, e.g., battery, electrical engine, inverter. In addition, the teams can choose their own sensor setup (camera, radar, LiDAR, (RTK) GPS, IMU) to create an autonomous vehicle. Furthermore, the teams develop the software that drives the autonomous go-kart around the racetrack. The current race setup consists of single vehicle time trial laps. Because the eV Grand Prix autonomous is still in its early stage there was no research published with these kind of vehicles so far.

5. Formula Student Driverless: Since 2017 student teams can develop a driverless vehicle for the Formula Student Driverless competition. The students can choose on their own how to design and equip the vehicle with both powertrain or autonomous driving hardware. Therefore different setups with different computation platforms (e.g., consumer hardware, Nvidia Drive PX hardware, Nvidia Jetson hardware) and sensors setups (monocular cameras, 3D LiDAR) exist. The cars compete in different single vehicle competitions: Acceleration (driving 75m straight with standing start), skid pad (two congruent circles with a diameter of 18.25m), autocross (racing on closed loop track with unknown layout), track drive and efficiency (racing 10 laps on a track with additional efficiency scoring based on the consumed energy). In addition to these pure driving competitions the cars are then judged in an additional business plan (business idea of the vehicle), design (judgement of hardware and software) and cost (financial planning an manufacturing) competition.

The research published with the Formula Student cars is spread completely over all topics in object detection [47]–​[49], [153], [182], localization [13], [69], [118], [175], planning [57], [176], [196] and control [59], [95], [98], [125], [133], [134] with a focus on holistic software pipelines [41], [50], [61], [99], [132], [190], [191], [194], [221], [222] with adjustments for the specialities of the FSD competition.

B. Full-Scale Autonomous Race Vehicles

The small-scale vehicles offer a low-cost and easy to set up platform for researchers. In addition, only a small space is needed to run the vehicles and therefore these kind of small scale vehicles are very attractive for research in the field of autonomous racing. Unfortunately because of the scaling there is still a mismatch between those vehicles and real racecars. This has not only to do with the performance ($v_{max}$ , $a_{long,max}$ , $a_{lat,max}$ ) but also with the kind of sensors or computation units these vehicles equipped with. Furthermore, a real racecar has a different dynamic behavior because of the stiffness of the chassis. Based on this some companies/institutions decided do develop real autonomous race vehicles which are explored in further detail. There was an additional development of an autonomous dragster [20], [21], the application of which, apart from in these papers, has not taken place elsewhere and is therefore not be considered in the further discussion.

1. Roborace: Roborace is a U.K. based company that developed different autonomous racecars (Devbot 1.0, Devbot 2.0, Robocar). The Robocar was only used for internal Roborace events and both Devbot 1.0 and 2.0 where provided to interested university teams and companies for their research. The Devbot 2.0 is based on a Le Mans Prototype (LMP) chassis and is a rear wheel drive, fully electric racecar. The vehicle is equipped with camera, LiDAR and radar sensors and two main ECUs (Nvidia PX2, Speedgoat Mobile Target Machine) to run the autonomous software. The goal of this vehicle development efforts is to provide both a vehicle platform and an annual competition where teams can compete against each other. The teams only need to develop the software, the hardware setup is equal for all teams. In 2018 single vehicle time trials were executed, in 2019 the so called Season Alpha provided different race formats (single vehicle, multi vehicle, localization) on racetracks in Europe. In 2020/2021 Roborace Season Beta started with seven university teams competing against each other in single-vehicle races (time trials). A special software from Roborace called Metaverse provides virtual static and dynamic objects on the track that needed to be avoided by the teams - otherwise they get time penalties for hitting these objects. The university teams used the Roborace vehicles for their research and provided plenty of published papers in the field of localization and motion estimation [129], [158], [177], [213], [224], [229], mapping [136], [144], planning [36], [43], [78], [81], [85], [86], [177], [179], [181] and control [33], [40], [214]–​[216], [226] as well as energy management [84], [85] and holistic software stack development [3], [22]–​[24]. In addition, the Roborace vehicle was used to derive some new simulation [77], [79], [80] and scenario environments [180], vehicle dynamics modeling [83], [90] and autonomous racing benchmarks [82].

2. Indy Autonomous Challenge: In 2020 the Indy Autonomous Challenge (IAC) was launched as a successor of the DARPA Grand Challenge and DARPA Urban Challenge. The IAC racecar is based on an Indy Lights chassis and is a rear wheel drive racecar powered by a combustion engine with 6 gear sequential transmission. The IAC vehicle is equipped with camera, LiDAR and radar sensors for perception and has one main ECU to run the autonomous software. The IAC provides universities both the vehicle platform and several competition types. The teams only need to develop the software, the hardware setup is equal for all teams. As a main middleware ROS2 is used. The IAC challenge consists of a single vehicle race around the Indianapolis Motor Speedway in October 2021 and a two vehicle head-to-head race on the Las Vegas Motor Speedway in January 2022. The aim is to drive 290 km/h with those vehicles and therefore the teams need to develop a high performance autonomous software stack that executes perception, planning and control. Since the IAC competition just finished its competition only a few papers [168], [217], [227], [231], [235] have been published so far.

SECTION IV. Discussion and Conclusion

In this section, we present short discussions on each area of research we have presented above. We try to point out the most predominant methods in each field and explain why these methods are popular and preferable for autonomous racing applications from the authors’ perspective. Since not all the works surveyed here explicitly present their results numerically, and even when they do they are rarely on the same metrics. Furthermore, hardware used (i.e., small-scale v.s. full-scale vehicles) differ tremendously between each publication, and some approaches were only evaluated in simulation. Thus, the authors think that an explicit comparison of numerical values would lead to the reader’s misunderstanding of each algorithm’s overall performance. Instead of showcasing numerical analysis and direct comparison of results between each approach, we only compare approaches by displaying their advantages and disadvantages on a high level.

A. Software Performance

1. Perception: In Autonomous Racing, we saw authors adapting mostly well-known algorithms from SLAM, computer vision, and deep learning for perception. These algorithms are modified and enhanced for usage on the racetrack. A common thread we see in perception for Autonomous Racing is that LiDARs and odometry are still preferred for almost all SLAM approaches. Although there is plenty of research on camera-only perception, and it is widely used on Tesla vehicles worldwide, it’s effectiveness has yet to be demonstrated in autonomous racing. Similarly, for object detection, well-known methods such as YOLO and its variations are used. From the authors’ perspective there is no preferable method in this field since most approaches are adaptation from software used on road vehicles. On contrary, as pointed out in Section V, we see the trend to develop high-speed localization and object detection methods. And with the recent development of Event-based Vision [5], it would a great candidate as a perception sensor in racing scenarios.

2. Planning: For global planning, the predominant approach is to set up an optimal control problem (OCP). The OCP not only considers the objective for the global planner (minimum lap time), but also considers hard constraints on the vehicle’s dynamic behavior. The optimization produces competent racelines has a choice of several efficient solvers for fast computation. Since the global plan is generated offline, the higher computation time of these methods can be neglected.

   For local planning, we recommend decoupling local trajectory generation and local trajectory tracking. In trajectory generation, one can use a nominal dynamic model that describe the vehicle’s behavior under most conditions to create multiple dynamically viable local plans with different characteristics. Requirements and cost objectives can be baked into these trajectories thus path selection can be done efficiently. E.g., sampling-based trajectory generation. Then, when tracking the selected trajectory, an MPC could be used with up-to-date operating condition of the vehicle. The decoupling of trajectory generation, selection, and tracking also allows integration with RL and game-theoretic methods during trajectory selection.

3. Control: With the decoupling of planning and control, the predominant approach amongst surveyed work is to focus on high precision path and velocity tracking with a dedicated algorithm. Although classic controllers (e.g., PID controllers) can achieve a good level of control precision, MPC approaches captures and considers the vehicle’s dynamic behavior in the control loop, and thus usually achieve higher control precision and accuracy. With the development of high performance solvers, (e.g., [244]), we see that Non-linear MPC can be solved in real time on real vehicle hardware with satisfying efficiency. Furthermore, learning-based approaches can address the limitation of traditional MPCs where the underlying predictive model can be updated online. However, it is yet to be shown the effectiveness of these approaches in a multi-vehicle competitive scenario.

4. Deep and Reinforcement Learning: In all the work surveyed in this field, only a small portion of algorithms are tested on real vehicles in realistic high speed environments. Among these algorithms implemented on the real vehicle, none of them consider safety and robustness constraints, which are crucial in the adversarial racing scenario. Nevertheless, RL and Deep Learning approaches show promising results in encoding the highly non-linear interaction and dynamics of a racing scenario. Together with the development of high-performance perception pipelines, the viability of these algorithms for multi-agent interaction is expected to improve significantly.

B. Hardware Performance

This survey displays five different types of autonomous racing hardware. Additionally, researchers are using modified sports cars to apply their developed algorithms. Because the research in the field of autonomous racing is very much at the edge of the dynamic driving limits, we can state that the research work with actual vehicle applications better covers the potential of the respective algorithms. Furthermore, we conclude that the robustness of the algorithms demonstrated on real hardware reaches higher speeds and accelerations and displays higher robustness in contrast to the research that was shown purely in simulation.

Since all autonomous racing hardware displayed in this paper differs in their purpose, hardware setup, vehicle dynamics capabilities, and racing competition type, no conclusion about the most favorable hardware can be drawn. We are concluding that especially the small scale and reduced scale vehicle sizes are interesting for research labs since both their setup and operation are demanding less personal and financial resources.

C. From Passenger Cars to Racing Cars

In all the software sections displayed, we saw researchers are using and improving algorithms from the field of robotics and regular passenger cars. This concludes that the transfer from normal autonomous passenger cars to autonomous racing cars is manageable and straightforward. One will see impressive results in the car’s behavior by applying state-of-the-art autonomy algorithms. For example, the open-source autonomous driving software Autoware.Auto is used in different research papers as a baseline software stack. Unfortunately, none of the state-of-the-art algorithms is capable of driving fast near the vehicle’s dynamic limits or executing high-overtaking maneuvers. To achieve this, an extension of the software is necessary with the methods surveyed in this paper.

D. From Racing Cars to Passenger Cars

A direct transfer of software from autonomous race cars to autonomous passenger cars has not been done yet. This is because this field is relatively young and emerging and researchers have been focusing on driving on the racetrack only so far. This is mainly due to the fact that the context and objectives for an autonomous racing scenario are too different from a street scenario. For example, some areas like the global optimal trajectory planning are not entirely of interest for passenger cars with their current definition. Therefore, we recommend focusing on the transfer of software from the named subcategories. Currently, we see the most substantial transfer possibilities in control and planning. For example, the knowledge gained in the field of MPC is beneficial for regular passenger cars to create a highly accurate path and velocity tracking on our road networks. Additionally, the displayed work in the field of behavioral planning, e.g., with game-theoretic approaches, has a high potential for being integrated into passenger cars since modeling the interaction between different road users is of high research interest.

SECTION V. Open Research Questions and Challenges

In the previous sections we provided a detailed overview of all the efforts that have been made in the field of autonomous racing for both software and hardware. The goal of these research efforts is to contribute to the development of safer autonomous passenger vehicles and the possibility to derive knowledge for the development of new and advanced autonomous driving algorithms. Although the state of the art is quite extensive, there are still some open and unsolved research questions where the field of autonomous racing can support, help and leverage the development of future autonomous driving algorithms. Based on additional discussions with leading researchers in the field we present a list of challenges in the field that determine open research questions:

Challenge 1 - Autonomous high speed perception: None of the previous work addresses high speed object detection or provides detailed insights into different fusion techniques for high speed localization. The current state of the art presents standard SLAM or object detection methods that are then adapted to the field of autonomous racing. We are currently missing methods, techniques and algorithms that are especially made for high speed driving where increased motion blur occurs and sensor synchronization becomes more important. A reliable detection distance above 100m is required. This can be achieved by decreasing the computational delay, an enhancement in the sensor fusion performance (camera + Radar + LiDAR) and with an increase of the object detection quality. When it comes to vision-based localization we see successful research in the field of drone racing which can be adapted and applied to the field of autonomous racing. Besides that there is currently no public dataset for high speed driving. However, the availability of rich data is essential for the development of comprehensive perception algorithms for this speed range.

Challenge 2 - Multi-vehicle trajectory planning: Most of the papers surveyed focus on a single vehicle racing scenario and only a handful of researchers tried to address multi-vehicle scenarios (>3 vehicles). Dynamic local trajectory planning at high speeds with multiple vehicles (e.g., for overtaking) is difficult and not covered completely in the state of the art and displays therefore a grand challenge for future research. The trajectory planning method must be capable of finding a path in a non-convex environment that is collision free, recursive feasible, incorporating dynamic vehicle constraints and is executable in real-time. Both the path and the velocity must be planned while taking the vehicle dynamics into account to leverage the current tire performance of the vehicle. We see this either as a chance for creating new types of methods and algorithms for trajectory planning or as a test environment for new heuristics that decrease the computational heavy calculations.

Challenge 3 - Multi-vehicle interaction: The interaction with other vehicles is an essential part of racing especially when it comes to head-to-head racing (e.g., overtaking, blocking). This interaction is covered with game theory approaches in some of the work but is not explored extensively. This interaction provides the need for new prediction algorithms that can deal with the high uncertainty of the opponents movements/behavior in the less structured environment of the racetrack. Hence, the prediction of surrounding objects can not rely on lane information or traffic rules, but has to be based on a comprehensive understanding of interactive scenarios. Besides that, a fundamental aspect of future state prediction is that it is inherently stochastic, as agents cannot know each other’s motivations, so multiple modalities have to be considered. We seek a model of the future that can provide both (1) a weighted set of discrete trajectories that covers the space of likely outcomes and (2) a closed-form evaluation of the likelihood of any trajectory. In addition, there are no sophisticated behavior planners that can derive critical interaction based maneuvers for competing in a race environment. The goal is to enable a tight coupling between reason about the influence of the surrounding agents on the ego systems trajectory and maintaining full capabilities of the ego systems vehicle dynamics.

Challenge 4 - Adversarial driving: The racetrack enables the testing of the capabilities of an adversarial vehicle that is exploring and evaluating the risk of future actions by planning and performing high risk maneuvers. This kind of research enables knowledge for autonomous cars that need to operate in highly crowded multi vehicle and multi passenger scenarios while minimizing the possibility of a freezing robot problem. This research heavily includes the calculation of risk for a perceived environment, a risk evaluation as well as new high precision local behavioral and trajectory planning algorithms.

Challenge 5 - Real-time vehicle dynamics modeling: Autonomous vehicles that operate on the limits of handling need to have an exact knowledge about the current vehicle dynamics state. One crucial factor here is the tire which creates the road-vehicle contact (friction value) which is changing drastically with aerodynamics (downforce), road conditions (tarmac), weather conditions (rain, snow) and the current vehicle maneuver (load shift due to braking, acceleration). The high degree of model uncertainty due to external influences combined with strongly non-linear effects in tire and vehicle dynamics represents a challenge for both motion planning and control (e.g., MPC) algorithms. Available models approximate the vehicle dynamics to a certain degree but are computationally demanding, especially when it comes to tire models. Current research efforts try to calculate the dynamical behavior of the vehicle with the help of artificial neural networks to be computationally faster than classical physical models.

Challenge 6 - Balancing safety and performance: The current work is heavily exploring the limits of an autonomous vehicle from a software and hardware perspective with the goal to drive fast. When it comes to a racing scenario we have to make a trade off between safety (not crashing the car), high performance (staying close to the opponent), energy management and making decisions while not violating the handling limits (stay behind opponent, overtake in particular turn). This setup creates the need for software that explores the trade-off between safety and performance. This software is then coupled with motion and behavioral planners and decides which actions to take next. In addition, the current state of the art does not cover the safety aspects of the autonomous racecar in particular and therefore we have an open research area where algorithms need to be derived that evaluate and balance both safety and performance of the vehicle.

Challenge 7 - Autonomous racing regulations and rulebook: Although the community currently consists of many different racing series with different cars we have no agreement on the driving rules for autonomous racecars. Although this can be declared as less research and more a community effort, when it comes to autonomous driving a rulebook based definition for racecars could be helpful. This leads to general guidelines researchers can rely on when developing their algorithms. Ultimately, this leads to software that is compliant for specific racecar competitions.

Challenge 8 – Overall system software performance: Besides the development of particular algorithms for autonomous racing in each part of the software stack, the in-depth analysis of the overall software performance is a research field that is rarely covered. The synchronization of modules and the application of real-time conditions can reduce the overall latency significantly. Additionally, the delay of sensors and actuators influence both reaction time and vehicle performance. Unfortunately, these need to be considered in the development of a full software stack for autonomous racing. The optimal scheduling of software execution steps and an efficient management of the CPU and GPU usage by orchestration and hypervisor methods are additional research fields for the software development.

Challenge 9 - Autonomous high speed hardware: All platforms discussed in this survey rely on standard consumer hardware (e.g., ECUs, sensors). There is currently no specialized hardware that aims towards high speed driving or that was made particularly for autonomous racecars regarding computational demand as well as for vibration and shock resistance. Especially when we have a closer look to execution times displayed in the listed papers we see that some of the algorithms can be executed faster if particular hardware would exist.

SECTION VI. Summary

This survey paper presents a comprehensive overview of the current state of the art in the field of autonomous vehicle racing. By discussing the previous and ongoing research efforts in this field we were able to demonstrate what kind of algorithms were developed to derive autonomous high speed driving on the racetrack. By splitting this paper into different sections for perception, planning and control we showed the individual achievements by researchers to establish the autonomous driving task for a racecar. Furthermore, we displayed and categorized research in the field of end-to-end algorithms, vehicle dynamics modeling and simulation environments. This survey aims towards a holistic review in the field of autonomous racing so additionally all hardware developments and autonomous racing platforms that are available are explained in detail. Some of these vehicles are associated with annual competitions that provide opportunities for researchers to test and evaluate the performance of their software. In total this survey is covering 237 papers in the field of autonomous vehicle racing. Furthermore, in the last four years we saw an increase of papers in this field (Figure 12).

FIGURE 12.

Evaluation: Published paper in the field autonomous racing from 2009 until the end of 2021.

Undoubtedly, the field of autonomous racing is an emerging field in intelligent vehicles, robotics and transportation system that is attracting more interest from researchers. Although we see an emphasis of research in the fields of planning & control, emerging fields like reinforcement learning for physical systems are applicable to autonomous racing. Based on these results we listed open research challenges for autonomous racing. This list can be used as a guideline for future researchers who can participate in the autonomous racing competitions. Finally, the list of papers surveyed in here are uploaded to a Github repository and updated on a regular basis so other researchers have an easy, open-source and structured access to the papers in the field of autonomous racing.

CONTRIBUTIONS AND ACKNOWLEDGMENT

Johannes Betz initiated the idea of this paper, created the overall structure and contributed to all sections of this survey paper. Hongrui Zheng contributed to the path planning section. Alex Liniger and Ugo Rosolia contributed to the path planning and control section. Phillip Karle contributed to the path planning section and open research challenges. Madhur Behl and Venkat Krovi revised the paper critically. Rahul Mangharam contributed to the overall structure of the paper, the open research challenges and revised the paper critically.

The authors would like to thank Todd Murphy (Northwestern University), Davide Scaramuzza (University of Zurich), Chris Gerdes (Stanford University), Markus Lienkamp (Technical University of Munich), Panagiotis Tsiotras (Georgia Institute of Technology) and Sertac Karaman (Massachusetts Institute of Technology) for their talks at 2021 IEEE ICRA 1st Workshop “Opportunities and Challenges with Autonomous Racing.”⁸ which contributed to the creation of Section V in this survey paper.

Appendix

See Table 7.