Contents Introduction
In nature, to make up for the deficiencies of a single individual, many biological populations form coordinated and shocking cluster sports scenes through mutual communication and cooperation between individuals, such as predation of wolves, aggregation and migration of birds, and the gathering of bees, honey, ant colony movement [1], etc. By studying potential individual behaviors, the working mechanism of the biological colony system can be obtained through the method of mathematical modeling, that is, through the exchange of internal information in the system, to achieve the working mechanism of external rules and orderly cooperative behavior [2], [3]. The classic algorithms include particle swarm optimization algorithm [4], [5], ant colony algorithm [6], which are often used in cluster collaborative control scenarios such as path planning and task allocation. For some emerging algorithms, the wolf swarm algorithm [7], bee colony algorithm [8], [9], and firefly algorithm [10] are widely applied in distributed UAV swarm cooperative control.
Aiming at the application of the algorithm in actual scenarios, three robots inspired by termites can build pyramids and other shapes based on simple rule and local perception [11]. Reference [12] developed an e-puck robot for teaching, and realized the cooperative behavior of 20 robots such as gathering and foraging. Reference [13] designed the low-cost Kilobot robot, and designed the collaborative functions such as foraging and formation, and carried out thousands of cluster demonstrations, achieving the scale of the robot swarm breaking through a thousand orders. Reference [14] combined with swarm cooperative observe orient decide to act (OODA) loop, the formation flight of more than 10 UAVs was realized for the first time [15]. Through the swarm intelligence behavior mechanism, the autonomous cluster flight of 10 quadrotors in an outdoor environment was realized. Introduced how swarm intelligence is applied to the strategic deployment of a group of autonomous drones [15], [16]. By adopting an elf-organizing map method, a large number of autonomous drone arrays can be automatically and adaptively coordinated to adapt to different terrain and complex environment [17]. Introduced various swarm intelligence-related technologies comprehensively and gave corresponding mathematical proofs [18].
However, as the scale of UAV clusters increases, whether in theory or system implementation, the difficulty index of clusters increases, and the architecture design becomes more challenging. Research shows that hierarchical control can reduce the complexity of task assignment in UAV clusters and improve the efficiency of cluster tasks. Coo W J and others believe that the UAV swarm task planning problem belongs to the combination optimization of complex problems, and it is planned to use a layered control method to solve such problems from the perspective of operations research [19]. Reference [20] proposed a six-layer hierarchical structure off drones swarms, CoMPACT, which effectively combines task planning, dynamic registration, reactive motion planning, and sudden biologically inspired group behaviors.
Sanchez-Lopez et al. proposed a hybrid reaction/deliberative open-source architecture AeroStack for multi-UAV systems, including five layers of reaction, execution, deliberate, reflection, and society [21], [22]. Grabe et al. put forward an end-to-end control framework for heterogenous drone clusters, and its high-level tasks are concentrated on the ground [23]. Tsourdos A divides the task allocation of multi-UAV systems into three aspects: task planning layer, collaborative path planning layer, and control layer from the direction of multi-UAV coordination [24]. Table 1 gives the comparison of related work on swarm intelligence framework.
This article will draw on the idea of Boskovic JD [25] to decompose the research problem of UAV swarm intelligence into five layers. The relationship between the layers as shown in FIGURE 1. The decision-making layer is responsible for the evaluation, planning and assignment of tasks [21], [22], [26]–[55], and generate decision data to the path planning layer. The path planning layer manages the sub-tasks and generates corresponding sub-task planning paths based on the decision data [56]–[73]. The control Layer performs task coordination between clusters according to path information, and realizes automatic obstacle avoidance and formation control [74]–[95]. The communication layer conducts network communication according to the interactive information generated by the control layer to realize the sharing of information between individuals [97]–[123]. The application layer will feedback the corresponding environment information to the decision-making layer according to different application scenarios [124]–[142]. Through the hierarchical optimization, the unmanned platform can target complex task scenarios and different application fields, to achieve hierarchical coordination, and quickly complete tasks.
On the basis of the framework, the research trends and future insights of each layer are analyzed. The limitations of swarm intelligence are discussed, and look forward to the future development of UAV swarm intelligence.
The rest of this article is organized as follows. Part II, basen on hierarchical, describes the research status and trend analysis of UAV swarm intelligence. Part III discusses the limitations and the latest technology trends. Part IV gives the conclusion. FIGURE 2 provides a detailed structure of the survey.
Research Status and Trend Analysis of UAV Swarm Intelligence
A. Decision-Making Layer
The decision-making layer is responsible for the task planning in the UAV cluster task and is the core part of the swarm intelligent system. The current research directions are mainly swarm architecture [21], [22], [26]–[30], swarm combat effectiveness and mission assessment [31]–[35], scheduling and management technology for complex tasks [36]–[51] and intelligent decision-making and game technology [42], [52]–[55].
1) Swarm Architecture
What kind of structure is used to combine multiple unmanned platforms to exert greater efficiency is the first problem to be solved in cluster implementation. In the past few years, more architecture-related research has been conducted in the field of UAV clusters. As shown in FIGURE 3, Sanchez-Lopez et al. proposed an aerospace architecture for an autonomous multi-UAV system, which integrates advanced concepts such as intelligence, cognition, and social robots, including five aspects of the reaction, execution, thinking, reflection and socialization [21], [22]. Execution control is a key task in the robot architecture and has a profound impact on the quality of the final system.
Molina M et al. describe a general method of executive control, combining distributed behavior control methods (such as status checking and performance monitoring) and centralized coordination. This method ensures a consistent original design for concurrent execution and can effectively work on different types of aerial missions [26]. To effectively reduce the scale of the formation problem, as seen in FIGURE 4, the manned aerial vehicle (MAV)/UAV mission alliance is formed in three phases: task clustering phase, UAV allocation phase and MAV allocation phase, and the method can give a reasonable mission planning scheme according to the battlefield situation [27].
Affected by the rescue scene, the communication between drones is greatly restricted. Aiming at the rescue mission planning problem of UAV group under limited conditions, [28] built a mission planning model based on task sequence. This model takes mission priority execution order as input to generate mission planning strategy. For the real-time requirements of task allocation algorithm and the limitation of individual resources in the dynamic environment of the UAV groups, a dynamic task and resource allocation algorithm for UAV groups based on task sequences is proposed. Each request sequence strictly distinguishes the necessary task time and synchronization waiting time [29]. Considering the types of drones, resources are essential in the coordinated control of multiple drones. Under the resource constraints, a multi-type UAV collaborative task assignment method based on cross-entropy is introduced, and the method can efficiently and accurately assign tasks to different types of collective drones [30].
2) Swarm Combat Effectiveness and Mission Assessment
The evaluation of UAV combat effectiveness and the role is of great significance to its future development. However, most of the current research is still at the stage of qualitative analysis. Reference [31] established a combat effectiveness evaluation model of UAV cluster combat system based on system dynamics (SD). Corresponding nine tree models were established for the weapon system in the combat process, taking into account the characteristics of the UAV cluster, and using the survival rate of the UAV and mission completion as evaluation indicators, an SD model was established. Immediately after the earthquake disaster, a rapid assessment of the earthquake-stricken is crucial for subsequent rescue work. Because drones can quickly reach disaster areas and acquire images, they are widely used in post-earthquake rapid assessment. However, sensor noise and other inevitable errors will affect the quality of images acquired by UAV sensors, which in turn reduces the quality of evaluation. Using the Rapid Assessment Task Allocation Problem (RATAP), the rapid assessment task allocation scheme for multiple drones can be established in consideration of target weights, drone endurance and sensor errors [32]. The flying ad hoc network (FANET) of a mobile UAV has a dynamic topology. Due to the limitation of drone battery resources and maneuverability, the routing in FANET is unstable. FIGURE 5 shows a biologically-inspired FANETs clustering algorithm, through the use of gray wolf optimization and clustering algorithm based on ant colony optimization, the performance of the system is evaluated in a terms of cluster construction time, energy consumption, cluster life cycle and delivery success probability [33].
Advanced multi-UAV control technology requires a general task assignment algorithm under resource constraints. Through the use of a cross-entropy-based multi-type UAV synergic task assignment method, under limited resource conditions, efficient and accurate task assignment for different types of cooperative UAVs is achieved [34]. To be able to fully synchronize all the drones in the group during the entire flight, the mission-based UAV swarm coordination protocol (MUSCOP) can effectively maintain the group cohesion of different UAV formations, and under different experimental conditions, it has high flexibility for channel loss. It can be seamlessly extended to a large number of drones without significant performance loss [35].
3) Scheduling and Management Technology for Complex Tasks
a: Complex Task Planning Problem
UAV formation mission planning is the process of formulating a flight plan for a specific target, which usually needs to be completed within a period of time. For the complex task assignment problem with known target location, it can be seen as a combination of the backpack problem [36] and the traveling problem [37], the goal of the system is to maximize some scores to complete tasks constrained by time and other resources [38]. For solving such problems, Ramirez-Atencia C uses an improved multi-objective genetic algorithm to deal with the mission planning problem [39]. Amila Thibbotuwawa studied a declarative approach to cope with the weather uncertainty during task execution [40]. Based on combined optimization mode, the method based on the direction graph and a new meta-heuristic optimization algorithm, namely the modified two-part wolf pack search algorithm (MTWPS), can be used to solve the problem, which can reduce the large number of UAV targets Simulation time [41]. For multi-UAV search attack task planning based on an intelligent self-organized algorithm (ISOA), Ziyang Z et al. adopted the distributed ant colony optimization algorithm to maximize surveillance coverage and attack benefit [42].
b: Complex Task Planning in Dynamic Environment
For the real-time complex mission planning problem of the multi-heterogeneous system in a dynamic environment. Yaozhong Zhang et al. effectively adjusted the consensus-based beam algorithm under the limitation of task timing, controlled UAV resources, dynamic task addition, and instant requirements (CBBA), developed a new method [43]. Yimeng Lu has built a safety planning framework for multifarious tasks in an unstable environment. With the traceability problem of computing, [44] designed a new concept based on the safety transition probability, and decoupled the computation of the dynamic evolution of the environment from the trajectory of the robot to solve the planning problem. Zhen Z et al. used a combination of hybrid artificial potential field and ant colony optimization (HAPF-ACO) to propose a UAV swarm intelligent collaborative task planning scheme for searching and attacking time-sensitive moving targets in uncertain dynamic environments [45]. Wilhelm et al put forward the use of fixed wing and quadrotor drones, using genetic algorithm (GA) method to demonstrate adaptive mission planning, including simulation to evaluate the total flight time [46]. Ziyang ZHEN et al. presented a distributed intelligent self-organizing mission planning (DISOMP) algorithm and ACO algorithm and parallel method (PA) for searching and attacking, using the Dubin curve design threat avoidance module has good flexibility, scalability and adaptability in dynamic target search and attack problem [47]. From a known environment, employing a hybrid component approach to multi-rotor UAV immediate mission programming, specifying assignments using metric temporal logic (MTL) and utilizing a hybrid model to capture various modes of drones operation can be greatly reduces the computational complexity of the problem, making it possible to realize the problem in real time, while ensuring the safety and timeliness of the UAV [48].
c: Multi-Task Assignment Problem
Considering the needs of distributed computing, [49] advance a new distributed task allocation theory for multi-task assignment in search and rescue scenarios, the concept of local cost is used to measure each sub-task and ultimately achieve the overall goal optimization. The battlefield changes rapidly, and a two-level task planning model based on the simulated annealing algorithm and tabu search algorithm is built to solve the question of multi-objective and multi-machine mission assignment. At the same time, combined with the five-state Markov chain model, the optimal mission planning scheme is determined by judging the survival probability of the flight platform [50]. In military applications, it is essential to minimize the exposure of the UAV group to enemy threats or avoid implementation of the plan. Reference [51] studied the mission plan of the integrated drones system at the tactical level, employing specified sensors to identify the target. TABLE 2 summarizes the scheduling and management technology for complex tasks.
4) Ntelligent Decision-Making and Game Technology
When multiple UAVs perform tasks, the information obtained by the commander conceals some variability. How to choose strategies based on uncertainty information will directly affect the success or failure of the UAV task. Reference [52] introduced a situation matrix to simulate the indeterminacy of the war information, and established a multi-UAV confrontation model based on uncertain information. An intelligent self-organizing algorithm (ISOA) based on a distributed control structure decomposes the global optimization problem into multiple local optimization problems. Each UAV can solve its local optimization problem, and then through the UAV Information exchange, optimal decision-making for multiple UAV systems [42]. An interpretable intelligent model is deduced for the decision-making logic of the UAV when performing scheduled tasks and choosing to deviate from the specified path. The intelligent model is an if-then rule derived from a Sugeno-type fuzzy inference model on a visual platform [53]. For the autonomous routing problem of formation drones flying in areas where communication is refused, this area does not allow the exchange of information between drones. Modeling the problem under the framework of game theory, the drone can choose multiple strategies to take the next step [54]. Reference [55] proposes a muti-UAV Bio-Inspired Optmizaed Leader Election (BOLE) method, the selected leader acts as a decision maker and assigns taks to other drones.
5) Research Trends and Future Insights
From the above research, we can see that there are still some challenges in the current accumulated architecture research.
Firstly, gradual work has only been verified in small-scale scenarios. As the scale of the transfer increases, the compactness factor of the system will also increase. Besides, the current UAV architecture design is mainly aimed at the downgrade of rotor drones for execution. There is relatively little research on the task of overloading the large fixed-wing helicopter.
Secondly, in the scheduling and management scenarios for complex tasks, heuristic algorithms are usually used to solve the problem of the UAV task assignment. However, due to the high computational complexity, it may take a long time to find the optimal solution, so it is necessary to study the task planning algorithm with fast convergence ability. In addition, due to the complexity and variability of mission scenarios, research on real-time mission planning for heterogeneous UAV systems will also become a direction in the future.
Finally, due to the uncertainty of cluster tasks, it brings great challenges to planning decisions. At the same time, the cluster system is usually composed of a large number of individuals, and the intelligent decision algorithm is exponentially dependent on the problem dimension, resulting in a sharp increase in computational complexity. Then compared with ground robots, drones have a faster speed, and the designed intelligent decision algorithm needs to meet the strong real-time requirements. In general, to adapt to the ever-changing and complex dynamic environment, and at the same time realize the intelligent decision-making technology that takes into account system optimization and rapidity, is still a challenge in the future.
B. Path Planning Layer
The path planning layer is responsible for transforming the decision data into waypoints and using the UAV attitude information and environment perception information to generate the UAV flight path through the waypoint. UAV path planning is to study how to determine the most feasible path between the start point and the endpoint. This problem is an NP-hard problem, so it can be modeled as an optimization problem. To better solve the path planning problem, many scholars have proposed various deterministic and meta-heuristic algorithms. FIGURE 6 shows the mainstream path planning algorithms, it can be divided into two categories. One is classic algorithms that need to load environmental information in advance, such as artificial potential field (APF) method,
According to the different research directions, this article divides the research work of UAV cluster path planning technology into four aspects: three-dimensional (3D) path planning [56]–[60], dynamic path planning [61]–[64], optimal road strength planning [65]–[67] and area coverage path planning technology [68]–[73]. TABLE 3 lists the classification of the path planning layer.
1) 3D Path Planning Technology
3D multi-UAV path planning is very difficult, because the UAV needs to find a feasible and least complicated path between the start point and the endpoint. Dewangan R K et al. used the improved GWO method to deal with the 3D path planning problem, and realized the feasible flight trajectory while avoiding obstacles [56].
A multi-trajectory planning scheme for the UAV cluster based on a 3D probabilistic road map (PRM) is mentioned. The UAV swarm can reach different places (marked and unmarked) in different situations and support emergency conditions in the city environment [57]. For the UAV path planning in the threat and confrontation area is a non-linear optimization problem with multiple static and dynamic constraints, through a multi-drones collaborative path planning method on 3D rugged terrain multi-swarm fruit fly optimization algorithm (MSFOA), which solves the shortcomings of the original algorithm that the global convergence speed is too slow and the local optimization [58]. In the scenario where the UAV is used for 3D oilfield detection, the pigeon inspired optimization (PIO) algorithm is used to optimize the initial path, and the fruit fly optimization algorithm (FOA) is used to perform local optimization to avoid obstacles while finding the best path [59]. In the static rough terrain environment, the path length, height, and adjustment angle need to be considered. The path planning problem is described as a three-objective optimization problem, and an improved particle swarm optimization (PSO) algorithm is used to solve it [60].
2) Dynamic Planning Technology
When the UAV is performing tasks in a complex environment, it needs to continuously avoid static and moving obstacles, at the same time, it must respond to sudden threats in the surroundings. Therefore, it is necessary to design a corresponding dynamic path planning algorithm. To solve the above problems, under the condition of static sudden threats, a series of candidate paths are generated using the cubic spline second-order continuity principle, finally, a total cost function is established to select the optimal obstacle avoidance path. This method has the characteristics of short time consumption and strong real-time performance [61]. When face with a changing unknown condition, an adaptive route planning with complementary sensors is necessary. The wall-follow method (WFM) and the artificial potential field (APF) method provide an alternative solution to the acyclic problem of WFM and the local minimum problem of APF [62]. Track detection and automatic scene understanding based on abstract vision is the key to the work of drones in complex outdoor environments (such as isolated disaster scenes). By building a support vector machine-based tracking detection and tracker combination framework, it is possible to achieve tracking direction estimation and stalking with lower computation and input [63]. Aiming at the inconsistency in the communication status information of multiple drones in a dynamic environment, by calculating the collision probability of the UAVs, and then using the Kalman filter for state prediction, the path conflicts of the UAV formation during flight can be avoided [64].
3) Optimal Path Planning Technology
With the popularity of the internet of things (IoT), UAVs have been widely used due to their rapid deployment and controllable mobility. However, the battery capacity of drones has limited their endurance and performance, so it is necessary to plan the optimal path for the UAV to execute the mission. In the field of broadened mobile crowd perception (MCS), the fixed-wing UAV-assisted MCS system is taken as the research object, and the corresponding joint path planning and task allocation problems are studied from the perspective of energy efficiency, and the original NP-hard joint optimization problem is transformed for the bilateral two-stage matching problem, this method has achieved good results in energy consumption, overall profit and matching performance [65]. To achieve maximize the power consumption of drones and sensors and terminal compliance to ensure that all data is collected efficiently with minimum energy consumption [66]. When a group of heterogeneous fixed-wing UAVs are traversing multiple targets and performing continuous tasks, to determine the optimal flight trajectory, a coupled distributed planning method combining task assignment and trajectory generation is proposed. Under the condition and relaxed Dubins path, the cooperative task planning problem is reconstructed. This method significantly improves the operating rate of the system and has the potential to be applied to practical tasks [67].
4) Aera Coverage Path Planning Technology
The task of coverage path planning (CPP) is to design a trajectory so that the UAV can move at every point in the area of interest. Control the UAV carrying a camera to take off at a random location in the target area and land in the terminal area. Spatial information is fed back to the drone through the convolutional network layer by using the class diagram input channel, and then a dual-depth Q-network (DDQN) is trained to make control decisions for the drone to balance the limited power budget and coverage goals [68]. In the coverage search of the target region, the problem is usually modeled as a potential game, and an improved binary logarithmic linear learning (BLLL) algorithm and potential game theory are researched to coordinate the coverage of multiple drones and collaborative control issues [69]. Aiming to reduce the oscillation of the UAV’s trajectory, the reciprocal method between adjacent UAVs is considered. The interactive decision-making method is self-organizing, distributed and autonomous. Under the premise of ensuring optimal mission planning, it can effectively improve the survivability of drones. Simultaneously loading equipment such as network cameras can realize the positioning and orientation of moving targets in the area [70]. The UAV airborne camera has the problems of coverage, positioning and orientation. By proposing the method of clustering targets, it can solve the low complexity problem of flying robots tracking moving targets. At the same time, it uses part of the knowledge of target mobility to improve the efficiency of the algorithm [71]. To improve the detection capability of the UAV in the designated area, [72] combines the simplified gray wolf optimizer (SGWO) and improved symbiotic organism search, and proposes a new hybrid algorithm. The algorithm effectively combines the exploration and development capabilities, simplifies the stage of the GWO algorithm, accelerates the convergence of the algorithm, and retains the detection capabilities of the population. When a drone tracks a ground target in an obstacle environment, the target will be lost due to the sight and other factors.
By using an improved depth deterministic strategy gradient algorithm (GA), and then constructing a reward function based on line of sight and artificial potential field (APF) to guide the UAV to achieve target tracking, and finally use the penalty to smooth the trajectory. At the same time, to improve the detection capabilities, multiple drones are used at each level to perform tasks, and long and short memory networks are used to approximate the environmental state, which improves the accuracy of approximation and data utilization efficiency [73].
5) Research Trends and Future Insights
UAV cluster path planning should not only ensure the optimal global path and the shortest time to complete the task, but also ensure that the UAV cluster can avoid obstacles and avoid collisions between the single machines during the completion of the task. In the path planning process, most studies utilize the drone as a mass point without considering the size and load of the drone, which makes the modeling process more sampling and ideal. In the future modeling process, the corresponding constraints need to be considered, Making simulation closer to reality and enhancing the robustness of actual control. In addition, most of the existing task planning methods are aimed at single or specific conflict scenarios and lack a holistic system solution for an integrated task environment.
C. Control Layer
The control layer controls the UAV to fly according to the planned path, which is the basis of UAV cluster research. By establishing control system frame-works [74]–[77] and designing corresponding controllers [78]–[80], it is possible to solve the reconstruction of different types of drones in cluster formations [81]–[86], cluster search And tracking [90]–[92] and cluster anti-collision and other aspects [93]–[95]. An overall summary of the classification of the control layer is given in TABLE 4.
1) System Control Platform
Automation equipment clusters are increasing in popularity and scale. Usually, there are two main ideas for their control: centralized and distributed control. Centralized platforms can achieve higher output quality, but will result in better network traffic and limited scalability, while decentralized systems are more scalable, but less complex. Justin Hu proposed the concept of the HiveMind platform, a scalable and high-performance centralized coordinated control platform for UAV clusters [74]. The UAV cluster network ensures the connectivity between high-speed UAV nodes and simplifies the design of various cluster applications. By establishing a new distributed cluster model for the UAV group network, the connectivity between all nodes in the cluster network is ensured. Compared with the traditional fly ad hoc network, the network throughput is increased by 1.4 times [75]. In response to the problem of group control, based on the concept of the appearance of group agents, [76] established a multi-layer group control scheme inspired by group intelligence, and then researched a comprehensive sensing and communication method to adjust how the UAV Calculate the distance and the deflection angle from the neighbor, and finally carried out a series of experiments on the simulator and cluster prototype based on OMNeT++ to evaluate the effectiveness of the scheme. The traditional UAV dynamic model usually relies on sensor input and a priori knowledge of the environment and targets. To overcome these limitations, a two-layer quasi-distributed control framework based on the simulation of urban blocks is shown in FIGURE 8, which was used to achieve continuous control of the UAV group in two designated monitoring stages [77].
![FIGURE 7. - High-level operations for the first scenario in the centralized platform [74].](/mediastore/IEEE/content/media/6287639/8948470/9214446/zhou7-3028865-large.gif)
![FIGURE 8. - Two-layer quasi-distributed control framework [77].](/mediastore/IEEE/content/media/6287639/8948470/9214446/zhou8-3028865-large.gif)
2) Controller Design Technology
In the design process of drones, the design of the controller is crucial. Selma B et al. proposed a robust intelligent controller based on an adaptive network fuzzy inference system (ANFIS) and improved ant colony optimization (IACO) to control the behavior of three-degree-of-freedom four-rotor aircraft. Using the ANFIS controller to reproduce the desired trajectory of the quadrotor in the two-dimensional vertical plane, this method reduces the learning error and improves the quality of the controller [78]. For the full-attitude maneuvering control of the suspension load of multiple UAVs under uncertain conditions, a controller that can handle the uncertain parameters of the crane system is used, and the limit of evaluation using the planned trajectory is given [79]. When the UAV moves along a specific path, the effective control of the airborne gimbal system directly affects the performance of tracking ground moving targets. Reference [80] modeled the gimbal system based on the non-linear Hammerstein block structure to effectively use the model predictive controller (MPC) for control, and also improved the real-time target tracking performance under external interference.
3) Flight and Formation Control Technology
Common methods of multi-UAV formation control include consensus theory, leader-follower strategy, behavior method, virtual structure method, differential game, finite-time control theory, and so on. For the UAV formation maintenance control problem of multi-agent system consistency, based on the Routh-Hurwit stability criterion, the stability criterion of the system equilibrium point and Hopf bifurcation conditions are obtained. At the same time, the model prediction controller of the follower can predict the leader’s movement state, so that the UAV formation remains stable [81]. Faced the problem of time-varying formation control of singular multi-agent systems with switched topologies, [82] designed a distributed formation controller based on output factors, through the impulse-free and equivalent exchange of singular multi-agent systems, An algorithm for solving distributed controller is designed, and the effectiveness of the method is verified by numerical simulation. Zhang J proposed a cooperative guidance control method based on the backstepping method to quickly form the desired formation and achieve a multi-UAV steady state. Compared with the model prediction method and the Laplacian method, the proposed method not only enables the UAV formation to quickly form the desired formation, but also provides a fast dynamic response and a small tracking error when tracking the virtual leader [83]. Close-formed rotary flight will expand the flying range of the UAV group, but due to some complex conditions, such as uneven fuel configuration and irregular formation, it will pose a huge challenge to the conventional rotary method [84]. On the basis of building a leader-follower reciprocating model, a distributed formation rotation algorithm is proposed, which coordinates the UAV group to fly in a compact and straight-line formation, increasing the formation range under the above complex conditions. Reference [85] proposed a distributed formation control algorithm for multi-rotor UAVs, which allows the UAV to achieve a balanced configuration on a predetermined shape in 2D or 3D shape at the same time. Based on the online MPC method and disturbance observer, we combined the formal algorithm with the flight control structure, and implemented a complete framework on the low-power computing unit. Based on the traditional MPC method, [86] presents an event-triggered model predictive control (MPC) scheme, which takes into account the prediction state error and the convergence of the cost function, and at the same time for each local level optimal control problem, developed a no-fly zone strategy based on safety distance and integrated it into the local cost function to make it more computationally efficient.
4) Collaborative Search and Tracking Technology
In terms of the theory of unmanned aerial vehicle cluster cooperative control, due to the autonomous characteristics of unmanned aerial vehicles, unmanned swarm collaborative methods are gradually mapped to multi-agent systems for research. Board researches the collaborative problem of UAV clusters, compared with the centralized control, distributed control can give full play to the autonomous capabilities of UAVs [87]. Jakob Foerster et al. broke through the issues of central planning and distributed decision-making, and effectively improved the limitations of single-agent observation [88]. Is based on the reconfiguration method of rule-driven agent-driven real-time distributed control system, and discusses the task coordination problem of the distributed intelligent system [89].
Target search for UAVs in unknown environments, it needs to consider its limitations and characteristics of effective strategies and control methods. Inspired by the population evolution model, [90] studied a collaborative UAV collaborative target search method based on an improved bean optimization algorithm (BOA), which presents effective search capabilities, distributed collaborative interaction, and emerging group intelligence. When the target has the searcher’s position awareness and mobility, it will increase the difficulty of searching. Using detection information and target motion prediction, a multi-UAV search path planning optimization model is established, and by using an efficient HPSO algorithm to solve the model, the target discovery probability is maximized [91]. For the tracking of multiple targets, [92] constructed a decentralized multi-target tracking system for a cooperative unmanned aerial vehicle (UAV) with limited sensing capabilities. This system combines clustering algorithms, optimal the sensor manager and the optimal path planner to achieve the expected tracking effect.
5) Obstacle Avoidance Technology
Under the high altitude density and increasingly complex flight conditions, it is very important to avoid collisions between drones and drones. However, this problem has not been fully resolved, especially in self-organized flight clusters. Reference [93] introduced a new type of self-organizing UAV group flight collision method, based on the Reynolds rule, the self-organizing flight rule model of the UAV group was established, and the new collision avoidance rules between UAV groups were derived. This method improves the collision avoidance efficiency between UAV clusters, and obtains a smooth collision avoidance trajectory, and its collision avoidance behavior is more in line with actual flight requirements. When a minority group encounters an obstacle, it is easy to fall into a local minimum state, making the group state worse. The use of a cluster obstacle avoidance algorithm with local interaction of obstacle information can improve the disadvantage poor consensus of the local obstacle avoidance algorithm, and promote the consensus of the UAV group while overcoming the obstacles [94]. However, the existing collision avoidance methods still have some theoretical and practical problems. The traditional artificial potential field method is limited to single UAV trajectory planning and cannot guarantee collision. Reference [95] studied an optimized artificial potential field algorithm for multi-UAV work in 3D dynamic space, introducing distance factor and jumping strategy methods to achieve trajectory planning and collision avoidance for UAV systems.
6) Research Trends and Future Insights
Drone control systems provide drones with the ability to fly accurately and adapt to complex environments, but there are still some deficiencies in current research.
First of all, in terms of controller design, due to the aerodynamic complexity and control coupling characteristics of fixed-wing UAVs, and lack of controllability, it is difficult to establish accurate dynamic models. At the same time, the UAV will be affected by wind factors during flight, affecting flight speed. Therefore, it is necessary to establish an accurate dynamic model with the help of some external experimental conditions, and at the same time consider the influence of wind factors.
However, in the field of UAV cluster formation flying and formation reconstruction technology, there are few research results on cooperative control such as formation maintenance and formation transformation of large-scale clusters (more than 100 aircraft). At the same time, how to be based on the performance constraints of the cluster platform, such as flight problems in uncertain communication environments, will become a challenge in the future.
Then for the problem of collaborative search and tracking of drones, due to the low cost requirements of the cluster system, usually using low-cost sensor equipment will cause noise and inaccuracy in obtaining information. Future work will explore how to develop low cost and accuracy High sensor. Moreover, the previous algorithms tend to study the path tracking of straight lines and circles. For the curve path tracking problem, there is no similar comparative comparison.
Finally, regarding the obstacle avoidance problem of the UAV cluster, it is necessary to further increase the response speed to the close-range threat scenario and reduce the avoidance time. To ensure that the UAV can successfully complete the hovering task, it is also necessary to develop an autonomous collision-free hovering control algorithm for the UAV. At the same time, airborne processing requires drone operations, such as dynamic sensing and avoidance algorithms and image processing. Designing a low-power, powerful airborne processor is a research focus in the future.
D. Communication Layer
The key to whether the UAV cluster can achieve the predetermined combat effectiveness lies in the acquisition and transmission of information. The efficient operation of UAV communications is the key to obtaining ownership of the battlefield. From the perspective of UAV communication and network, the relevant mission parameters, data requirements of the network, and the applicability o the existing technology that can support aerial networks are summarized [96]. The main problems of current communication layer research include communication architecture [97]–[99], communication, networking technology [100]–[107], airbase station technology [108]–[114] and security communication technology [115]–[123].
1) Communication Architecture
The communication architecture is one of the cores of UAV networking design. A suitable network structure can improve the efficiency and reliability of communication data and the execution of upper-level tasks. Through collaborative communication and relay technology, UAV clusters can expand the effective coverage of IoT services through multiple relay nodes. FIGURE 9 shows a hierarchical network structure of unmanned aerial vehicles is mentioned in [97].
![FIGURE 9. - Typical layered UAV network scenario for forest fire prevention and inspection [97].](/mediastore/IEEE/content/media/6287639/8948470/9214446/zhou9-3028865-large.gif)
Typical layered UAV network scenario for forest fire prevention and inspection [97].
The minimum number of upper-layer unmanned aerial vehicles is theoretically proved by a closed-form coverage boundary, and the average delay and average link delay are verified by numerical simulation. It has better performance in terms of packet distribution rate. In the future intelligent transportation system, a diagram of the ICT-centric mobile simulation framework is illustrated in FIGURE 10. Mobility and communication simulation is integrated into a system process, which can make good use of the unique mobility potential of its drones to improve the performance of the service [98].
![FIGURE 10. - The ict-centric mobile simulation framework [98].](/mediastore/IEEE/content/media/6287639/8948470/9214446/zhou10-3028865-large.gif)
Besides, due to the different tasks of drones in the network, the communication needs of each UAVs need to be considered. As shown in FIGURE 11, from the perspective of game theory, the joint channel slot selection problem in a multi-drone network is described as a weighted interference mitigation game, and then a distributed logarithmic linear algorithm is applied to achieve the desired optimization, which Overcome the constraints of the dynamic communication requirements of each UAV [99].
![FIGURE 11. - Illustration of the multi-UAV network, showing the selection of each UAV [99].](/mediastore/IEEE/content/media/6287639/8948470/9214446/zhou11-3028865-large.gif)
2) Communication and Networking Technology
The key to whether the UAV can achieve its intended combat effectiveness lies in the acquisition and transmission of information. The performance of the UAVs communication network is the key to gaining the right to information on the battlefield. At the same time, due to the high probability of line-of-sight (LOS) link and on-demand deployment, the UAV can improve the overall performance of the ground communication system. Chen Q et al. studied a relay system in which multiple UAVs established a UAV relay network through orthogonal frequency division multiple access (OFDMA) to help some transmitters communicate with corresponding receivers [100]. In a cellular network composed of drones, each drone can sense and transmit data from multiple tasks to the base station. By introducing the Information Age to quantify the “freshness” of base station data, and using a task scheduling algorithm to optimize the sensing time, transmission time, the trajectory of the UAV to complete a specific task [101]. However, UAV nodes usually face design problems and power limitations, which in turn affect the routing mechanism. Lasari H N et al. proposed the concept of cross-layer design and efficient power algorithms to improve the performance of UAV networks [102]. The modular dynamic clustering method based on the improved Louvain method can be used for efficient UAV-assisted mobile communication, which is promising in solving the energy consumption of UAVs and reducing the transmission power of mobile devices [103].
The networked multi-agent system (NMDS) uses local control rates based on the spatial information of the nearby environment and neighboring agents to exhibit a sudden swarming behavior, but there is currently no method of interaction between the operator and a large number of agents. Reference [104] researched a robust and estimated method for operators to indirectly configure the propagation and observation of the system state to help the operator provide the best input to the NMDS at the appropriate time. The deployment of large-scale UAV clusters based on the advantages of clusters will lead to high competition and excessive congestion of spectrum resources, resulting in mutual interference. Multi-cluster flying ad hoc networks (FANET) under different network topologies have interference-aware online spectrum access problems. The online distributed algorithm based on the optimal response (IOCPCBR) algorithm can effectively reduce the interference of the UAV cluster and reduce the cost of channel switching per time slot [105]. Albu-Salih AT and others proposed a new framework to improve the data collection efficiency of wireless sensor networks and multiple drones, and minimize the total travel distance, travel time and network energy consumption of drones. The framework can effectively collect sensor data while meeting the constraints of UAV energy and duration [106]. To solve the joint optimization problem of drone location, time slot allocation and computing task allocation, [107] proposed a mobile edge computing (MEC) server enabled by UAV (UAV). Users provide MEC services.
3) Airbase Station Technology
Drones have the flexibility and ability to establish line-of-sight wireless connections. In scenarios that require rapid deployment (for example, in the case of natural disasters, emergencies, and sports events), drones are used as aerial base stations to supplement and support existing ground communication infrastructure. At present, UAV-wireless network (UAWN) has been successfully applied to UAV-cellular unloading (UAV-CO), UAV-emergency communications (UAV-EC), UAV-Internet of Things (UAV-IoT), etc. TABLE 5 summarize the application of the UAV-wireless network.
a: UAV-Cellular Unloading
In the field of UAV-CO, [108] has established an analysis framework for analyzing the signal-to-interference-noise ratio coverage probability of drone-assisted cellular networks of cluster user equipment (UE). The location modeling with the ground base station is a poison point process, which verifies the impact of the UAV height and path loss function on system performance. Reference [109] analyzes the coverage performance of UAV-assisted terrestrial honeycomb and introduces a user-centric collaborative drone clustering scheme. Under the condition of a suitable collaborative cylinder, by optimizing the cache size and drone the density maximizes the coverage of the system. Meanwhile, due to the fast mobility and highly dynamic topology of the UAV network, it is necessary to design a clustered layered routing protocol to provide scalability [110].
b: UAV-Emergency Communications
In the field of UAV-EC, drones can be used as assisted emergency communications in emergency scenarios such as firefighting and disaster rescue. The information is transmitted to the control center. Zhang Q et al. proposed a joint communication and calculation optimization solution for the UAV group scenario supported by mobile edge computing technology (MEC). Under the communication and calculation constraints, the transmission efficiency is improved and the response delay is reduced [111].
c: UAV-Internet of Things
In the field of UAV-IoT, the deployment of drones can facilitate the data transmission of IoT devices. During the transmission of uplink data between the Internet of Things equipment and the base station, the use of drones as a communication relay can enhance the signal reception strength of the base station. Using a distributed UC algorithm, IoT devices are integrated into multiple UCs. By solving the system optimization model, the optimal deployment and transmission power of each UC relay UAV is obtained [112]. When the throughput optimization problem involves the constraints of the drone’s flight speed and the IoT device’s upstream transmission power, the use of multi-agent deep learning (DQL) strategy and a anoverl algorithm can solve the 3D path planning and Joint optimization of channel resources [113]. Current communication systems are based on traditional information theory principles to transmit long messages, and achieving ultra-high reliability of short messages is the core challenge of future infinite communication systems. Ranhja and others rely on multi-hop drone relay links to provide ultra-reliable and low-latency (URLLC) command packets between ground-level Internet of Things (IoT) devices to reduce the overall decoding error rate and enhance the reliability of receiving short data packets [114].
4) Secure Communication Technology
In the actual UAV communication network, although the nature of the powerful LoS link allows UAV communication to provide ubiquitous high data rate wireless services, it also makes the communication between the UAV and the ground user more easily intercepted [115]. Therefore, various serious challenges have been proposed for the security of UAV communications [116]. In [117], various communication protocols between UAVs are discussed and a new UAVs secure communication protocol is proposed. To improve the efficiency of safe communication, the [118]–[121] system uses multi-UAV cooperative communication. A jamming drone can fly close to a potential eavesdropper, and then transmit artificial noise signals to interfere with the eavesdropper [118]. At the same time, to improve the safety performance of the system, [119] and [120] proposed a cooperative interference method, which uses artificial interference transmission to protect the communication of the drone, from the existence of a single eavesdropper other neighboring drones [119]. Fair consideration in the safe communication of two drones, investigation of the joint power distribution and trajectory design to maximize the minimum secrecy rate for each user, one drone dispatches confidential information to the ground, and the other cooperative drone transmits interference signals [121]. Since in [118]–[121], the role of the UAV is fixed, it can only provide communication/interference functions in the entire time range. In response to the above shortcomings, [122] proposed a multi-purpose drone, which can be dynamically used as a communication drone or jamming drone, providing a high flexibility for safe drone communication. Reference [123] studied the confidentiality interruption performance of the low-altitude UAV group secure communication system using opportunistic relays, and verified the influence of backhaul reliability, drone cooperation and eavesdropping probability on the confidentiality interruption probability.
5) Research Trends and Future Insights
The communication between cluster drones is mainly used for the exchange of status and load information between drones. When the number of drone clusters is huge, there are many types of tasks, fast flight speed, frequent changes in relative space-time relationships, and the effectiveness of information transmission, making the communication networking between clusters very challenging.
The main research directions of current communication networking can be divided into four aspects.
DELAY-TOLERANT NETWORKING (DTN): Solve networking problems in a dynamic environment;
NETWORK FUNCTION VIRTUALIZATION (NFV): Enable network infrastructure virtualization;
SOFTWARE-DEFINED NETWORKING (SDN): Provides separation between the control plane and data plane;
LOW POWER AND LOSSY NETWORKS (LLT): Effective networking under limited resources.
In the research of aerial base stations, since FANET uses the same wireless communication frequency bands
as satellite communications and GSM networks, it will cause frequency congestion. Therefore, it is necessary to standardize the FANET communication frequency bands and design corresponding congestion control algorithms.
In the field of unmanned aerial vehicle communication, in addition to measures to interfere with unmanned aerial vehicles to ensure safe communication, future research work can also be done from the perspective of signal distortion monitoring and multi-unmanned cooperative countermeasures.
E. Application Layer
At present, drones have been widely used in leisure entertainment and express delivery services, and may even be expanded to military reconnaissance, agriculture, environmental monitoring, emergency rescue and other application fields in the future. Smart drones are the next major revolution in drone technology and are expected to provide new opportunities in various fields under the premise of reducing costs and risks. TABLE 6 lists the application of UAV swarm intelligence.
1) Intelligent Transportation Field
[124] introduced the civilian applications of drones and their challenges, and discussed the current research trends, and proposed the challenges faced by civilian applications of drones: battery life, collision avoidance, network resource congestion, and security. In the field of intelligent transportation, multi-rotor drones have recently been recognized as one of the advanced technologies used in smart cities. It can monitor the transportation network, channel traffic and monitor events in real-time [125]. Reference [126] introduced a UAV intelligent traffic monitoring system based on five generation (5G) technology, which overcomes the static characteristics and other limitations of traditional systems and greatly reduces the incidence of traffic accidents. Reference [127] proposed a UAV-based vehicle detection and tracking system, which can generate vehicle dynamic information in real-time. Reference [128] uses a convolutional neural network method to improve the accuracy and performance of moving vehicle detection. Reference [129] integrate video data collected by drones with traffic simulation models to enhance the real-time traffic monitoring. To optimize the problem of the limited battery capacity of UAVs, [130] developed a universal management framework for UAVs for intelligent transportation systems (ITS), which effectively utilized the UAV fleet. In addition, in the intelligent transportation delivery system, the deployment of drones is still in its infancy. Reference [131] proposed an optimization from the four aspects of communication routing methods, optimal path planning of drones, dynamic task allocation, and hybrid delivery solutions Good candidate for delivery efficiency and reduced air traffic.
2) Environment Monitoring Field
In the field of environmental monitoring, use drones to collect sensors data to monitor water and air quality in real-time [132]. Reference [133] introduces the concept of the internet of drone things and gives its application in some special scenarios. Besides, the location of pollution sources in chemical plants has become a hot research topic. An air pollution source tracking algorithm based on artificial potential field method and particle swarm optimization algorithm can accurately find the pollution source in a short time [134]. To monitor the degradation of tropical rain forests, [135] studied a new method for real-time automatic detection of non-forest and eroded areas in tropical rain forests. Continuous segmentation through multiple thresholds will produce a binary map that can clearly distinguish the forest and eroded areas.
3) Electromagnetic Spectrum Monitoring Field
In the field of electromagnetic spectrum monitoring, to monitor private black broadcasts, a group of unmanned aerial vehicles carrying sensors that receive strong signals are used to locate intermittently launched RF transmitters [136].
4) Agricultural Field
Reference [137] emphasize the importance of UAVs in agriculture, by using UAVs in agricultural monitoring and observation to increase crop yields. In terms of agricultural plant protection, spraying pesticides by drones has the advantages of safety, good prevention and control effects, low cost, and no geographical restrictions. By establishing the overall framework of the plant protection drone control system and using Pixhawk open-source flight control, the optimal spray control requirements of the plant protection drone are achieved [138].
5) Emergency Field
Reference [139] introduced a multipurpose UAV for mountain rescue, which can adapt to mountain environments such as low temperature, low altitude, and strong wind to perform rescue missions. A drone search and rescue scheme based on human body monitoring and geographic positioning is proposed in [140], when the UAV scans the wounded in the target area, it will deliver medical supplies to it through geolocation. By using a UAV modular architecture for search tasks, it is possible to realize the collaborative control of multiple UAVs and transmit the video information stream in real time [141]. Besides, in fire or chemical accident scenarios, taking into account the spatial correlation inherent in the investigation phenomenon, Katharina Glock introduced a concept of mission planning, routing drones through a set of sampling points, using interpolation methods to collect samples to predict hazards Distribution of substances throughout the affected area [142].
Discussions
A. Limitations
1) Charming Challenge
The battery capacity of the drone is a key factor in achieving continuous missions. But as the battery capacity increases, its weight will increase, which will cause the UAV to consume more energy for specific tasks. Researchers are developing optimized hybrid battery solutions [143]–[145].
Reference [146] introduced an autonomous battery maintenance mechatronics system, which can quickly replace the depleted battery and a supplementary battery of the drone, while charging several other batteries. This allows the battery maintenance system to have a low drone downtime, arbitrarily expandable operating time, and a compact footprint. Reference [147] proposed a new type of battery charging exchange station that converts used batteries into rechargeable batteries while keeping the micro aircraft in an active state to ensure continuous operation of mission parameters and data.
2) Manufacturing Cost Challenge
Cluster systems often achieve quality advantages with scale advantages, and the damage of a single individual will not affect the system. Therefore, large-scale cluster systems often strictly limit the cost of single-machine systems, including platforms, loads, airborne processors, and communication equipment. Small drones also have certain requirements for load weight. To pursue high performance, existing loads are often expensive and bulky, and are not suitable for use on drones. Therefore, the development of low-cost, lightweight platforms and loads that meet the needs of clusters is of great significance to the formation of cluster task capabilities.
3) Secure Challenge
The widely used drones may also be used maliciously and even threaten national security. Reference [148] has designed a traceable privacy protection protocol. This solution provides a feasible and safe management platform for the application of drones in sensitive areas. At the same time, to ensure the safe operation of drones, it is necessary to establish a common term for experts in the avionics and telecommunications industries to better understand the needs of the two fields [149]. In the future, the safety regulations of the drone industry need to be further improved and supplemented to better serve human beings.
B. Latest Technology Trend
In addition to the technological development trends proposed at each level of the drone cluster architecture, we also combined some of the latest development technologies today, and proposed some possible future directions of drone cluster intelligence.
1) UAV Swarm Digital Twin System
The concept of digital twins was first defined by Michael Grieves of the University of Michigan in the United States. In 2003, he proposed “virtual digital expression equivalent to physical products” [150]. Digital twins create virtual models of physical objects in digital form to simulate their behavior. The virtual model can understand the state of the entity through perceptual data, thereby predicting, estimating, and analyzing dynamic changes. Through the establishment of the UAV cluster digital twin system, it is possible to understand the complex commands of the commander, perform fuzzy reasoning based on the perception of itself, partners and the transmitted external information to obtain the optimal strategy and actions in the current situation, and carry out task execution autonomously.
2) Artificial Intelligence Promotes Bionic Intelligent UAV Swarm
With the development of artificial intelligence, UAV cluster control will become more intelligent in the future. Design a distributed control framework for UAV clusters based on artificial intelligence, so that the UAVs in the system can only form a self-organized intelligent interaction network with other UAVs through cluster data link technology under the local perception ability, and trigger in the external environment Under the hood, implement complex behavior patterns, have learning capabilities, and emerge intelligence at the group level. In the artificial intelligence assisted drone network, [151] proposed an artificial intelligence assisted drone for the dynamic environment to assist the next generation network. Multiple drones are used as aerial base stations to collect information about users and remote traffic needs. Learn from the environment, and take action based on user feedback, and then quickly adapt to the dynamic environment to improve the network performance, reliability, and agility of the system.
3) 6G Air-Ground Integrated Network Communication Technology
The air-ground integrated network is a key component of the future 6G network, which can support seamless and almost even hyperlinks. A novel architecture called UaaS (UAV even service) is proposed for the air-ground integrated network [152]. This architecture will use machine learning (ML) technology to enhance the key driving force of edge intelligence. In the future, the UAV network can be used to intelligently Provide wireless communication services, edge computing services and edge caching services, to take full advantage of the flexible deployment of drones and various machine learning technologies. The 6G networked drone cluster is bound to make the advantages of drone formation and formation reconstruction, task coordination, heterogeneous drone coordination and human-machine collaboration, etc., to the extreme.
Conclusion
From the perspective of a layered control framework, this article divides the key technologies currently studied in UAV cluster intelligence into five levels: decision-making, path planning, control, communication, and application. Meanwhile, the research trend and future insights is described at various layers. Finally, this article discusses the limitation of swarm technology and looks forward to the future from three possible aspects. The development trend of human-machine cluster intelligent technology is expected to play a certain role in the development of future UAV cluster systems.