Contents Introduction
Autonomous and automated driving has become one of the most challenging technological problems over the last decade. Engineers aim to improve traffic with respect to safety, comfort, efficiency and effectiveness simultaneously. However, these goals are in conflict [1], which makes finding satisfactory solutions so difficult. Significant international research and development efforts for automated driving functions have yet to yield reliable, high-performance autonomous vehicles that can operate in a broad range of traffic situations.
Approaches aligned with these efforts are information sharing, connected driving, and infrastructure-supported driving. For example, infrastructure might be able to provide information that is impossible or very difficult to be acquired by the vehicle itself. This raises many unresolved questions, including what information is best shared, in what form and quality must it be made available, and how safety and security can be ensured. These questions are addressed in this paper by putting the collaboration of infrastructure and vehicle on a solid basis.
The acquisition of all necessary information for automated driving tasks may take a joint effort of different stakeholders and organizations. The group of stakeholders in automated driving is diverse and contains not only car manufacturers but also infrastructure managers, road service providers, traffic managers, map vendors, vehicle operators, policymakers, and legislature among others. Each of these stakeholders is involved in different automation tasks which encompass making certain decisions. Automating a decision requires that information about all relevant aspects of the task are available to the decision making entity. An infrastructure based digital twin [2] of a traffic system is not only a digital model of a real world traffic system, but also enables decision making that affects the real world in a closed loop by providing information to decision making entities. Such digital twins or model based controllers with live updates have already been proposed in [1], [3], [4], [5], [6], [7] among others. It is worth pointing out that all considered stakeholders are in a symbiotic relationship that is rooted in the quality of the content of the digital twin. This work aims to address the problem of what aspects of the world should be represented in a digital twin and how, so that a set of decisions can be made.
The decision making process for humans differs significantly from the process for algorithms/machines in several ways. Humans are very good at dealing with partial, fuzzy information and can fill in the blanks by experience and context knowledge [8]. If information is of insufficient quality, a risk-lowering (not necessarily minimizing) action can often be found. In contrast, a machine requires information in a formalized format meeting quality standards. Incomplete or missing information is often a show-stopping problem and the automated acquisition of missing context is often infeasible, e.g., [9].
For machines as for human drivers, a driving task can be performed if the capability of the (automated) driver exceeds the complexity of the given task [10]. The complexity of a task is mainly determined by factors that can not be controlled by the driver and describes how much and which information about their surroundings, environment and situational context must be available to the driver. In contrast, the capabilities of the (automated) driver are determined by the available information, its utilized algorithms and hardware. The capabilities determine a vehicle’s or automated driving function’s Operational Design Domain (ODD). Both the algorithms and the built-in hardware can be considered operating constraints that can not be modified in a meaningful way outside of software updates (which can be performed by the manufacturer only) or hardware replacements (which can be performed by a mechanic). However, information can be made available during a vehicle’s (or driver’s) operation and is therefore of major significance for the success of automated driving functions.
Automated driving systems can acquire information from three different sources:
on-board sensors (e.g., video cameras, radar and lidar sensors),
communication networks (e.g., Vehicle To Everything (V2X)-communication or other connectivity services),
databases and data stored in memory (e.g., maps for navigation).
This work aims at providing a conceptual as well as technical background that enables sharing information across traffic participants and infrastructure providers. In Section III, the fundamental idea of communicating at information level is argued for, the necessity of a reference information model is addressed, and a set of requirements for information models is proposed. Lastly, the relationship between the ODD of a function or vehicle, a reference information model and the infrastructure support level (ISAD) is discussed. Section IV addresses requirements on a technical realization of a digital twin based infrastructure support system. In Section V, a solution concept that meets these requirements is proposed and in Section VI the solution concept is implemented for several real-world demonstration use cases where information modeling was performed for relevant information groups, validating the introduced concept for the digital twin based decision support platform. Finally, in Section VII an extensive discussion is provided that covers general open issues as well as specific technical aspects of the implementation.
Related Work - open for discussion
The concept of Digital Twins (DTs) is well established in smart manufacturing and became even more relevant with advancements in the Internet of Things (IoT). An overview of the current standards landscape for DTs and the IoT is provided in [13], which cover aspects of resource description, discovery and access. These aspects are relevant to applications outside of manufacturing as well. In [14] an overview of works that aim at technology transfer from manufacturing to smart city applications and the application of DTs in traffic an fleet management specifically, is provided. Works on DT applications in the context of smart cities and federated learning are surveyed in [15]. At the same time a vast body of research exists on autonomous vehicles and enabling technologies such as 5G [16] which often hinge on the utilization of a suitable DT. However, smart road applications at the intersection of automated driving, collective perception and smart traffic management are not explicitly covered by these surveys.
Generally speaking, technological frameworks, protocols and standards for, DTs description and data exchange exist and can be utilized in many different DT applications. However, each DT is a custom design when it comes to data/information modelling and interoperability of different digital twins. A first step in the direction of requirements formulation for information models is given in [17] where high-level requirements are identified and described from the perspective of manufacturing. However, no requirements on information itself and its representation are given.
With many different participants, both in traffic and as infrastructure providers, the amount of data that is potentially available for sharing with all other participants exceeds practical limits. To overcome this problem, it is necessary to establish a suitable level of abstraction for message content that ideally reduces both data volume and computational effort for message interpretation. To the authors’ knowledge this issue has not been addressed previously and is therefore addressed in this work.
Methodology
Based on a vehicle’s ODD, a set of scenarios can be determined that the vehicle can cope with. Such scenarios include factors of the environmental conditions, the state of traffic control and management, the traffic regulations and laws, the composition and behavior of traffic participants, the vehicle’s and driver’s intentions, conditions and capabilities including the capability to acquire information with on-board systems. A smart road service aims at supplementing a vehicle’s perception of the world. Therefore, communication between traffic participants and infrastructure is central. In order to improve aspects of traffic (e.g., safety, flow, density, etc.), two questions must be addressed: First, what content should be communicated? Second, how should the content be represented?
A. Abstraction Levels for Content Representation
Content may potentially be presented at any level of abstraction, e.g., signal, data, information, knowledge or even action (loosely following the Data, Information, Knowledge and Wisdom (DIKW) model [18]). However, communication is not practical at all levels. These levels are discussed using a 100 km/h speed limit on a highway, traditionally communicated via a physical traffic sign, as a working example. Table 1 maps speed limit aspects to each abstraction level.
One could consider each level representing a different natural language. The same meaning (semantic) might then be communicated in any of these languages. Message syntax would then be the specific wording in that language. There would be advantages or disadvantages in using a particular language.
Presenting content on the signal level would involve transmitting all features and characteristics of the traffic sign. Each participant must combine these received features and classify the particular traffic sign. This process is not robust: occlusion, incomplete feature sets, or contradicting features pose problems. Also, equivalent traffic signs may differ in appearance among countries.
Presenting content on the data level would involve transmitting the position and type of the traffic sign. This seems familiar since it mimics the established style of communication from road authority to the human driver. Furthermore, it reduces robustness problems and is less verbose compared to the signal level. But, inferring the speed limit requires additional data, e.g., another (later) traffic sign ending the speed limit.
Presenting content on the information level would involve transmitting a complete description of the speed limit, including the obvious maximum velocity but also the geographic, situational, and temporal scope as well as addressed vehicle types. This leaves little room for interpretation due to inference/classification failures or errors caused by incomplete data or signals such as missing the corresponding cancellation sign.
Presenting content on the knowledge level would involve transmitting a message like ‘your speed should not exceed 100 km/h’, but, ‘because you pull a particular trailer, your speed should not exceed 80 km/h’. This requires that the sender knows something about the receiver. Generally speaking, communication on the knowledge level requires the sender to have access to additional characteristics like vehicle type, driver’s license type, toll sticker, axis load, etc., with disadvantages in the technical and legal areas. In this working example, converting information into knowledge is equivalent to the mental step that a human driver takes when recognizing a speed limit sign at a location (information) and concluding that “this particular sign is applicable to me”.
Presenting content on the action level would involve transmitting recommendations, like ‘you should slow down’, or even (remote) commands that might reach through to a connected vehicle’s braking system. This essentially means that the sender would perform tasks that are traditionally performed by the driver (or an automated driving (AD) function). This is in conflict with international agreements which state that the driver [19] or the AD function [20] has to be in control of their vehicle at all times. Besides legal questions, it is doubtful that proper consideration of the full situational context can be ensured by other entities than the driver.
The discussion above indicates that presenting content at the information level is most advantageous. This approach ensures feasibility and upwards-compatibility with respect to (future) sensor systems and avoids legal issues regarding driver or AD function responsibility. The responsibility to correctly derive knowledge from information and situational context remains with the driver, an AD function or, generally, the receiver. At the same time it does not merely replicate the established way to communicate (via traffic signs), which is difficult to standardize for international applicability [21] and error prone regarding completeness and correctness of the transmitted information.
B. Short Survey of Information Models for Smart Road Services
An information model for smart road services declares and structures all conceivable relevant information. The notion of an information model is sometimes also referred to in the literature as data model or ontology.
Given that smart roads and autonomous driving is a highly dynamic and young field of research, the landscape of standards, formats, tools, and methods has not yet converged to a homogeneous, universally accepted, and well-structured state of the art. The platform connected automated driving1 collects about 170 published standards and more than 30 standards in development, among those, e.g., Taxonomies for ODD [22], [23], [24], Scenario representation [25], Safety and Edge Cases [26]. Standardization efforts for communication in traffic at the information level in recent years were fruitful and yielded a group of published standards, e.g., [27]. These documents define groups of C-ITS messages suitable to express different types of information relevant for smart road services.
Since these standards focus on applications within a declared scope, they, by design, only partially cover the spectrum of potentially relevant information. In the context of smart roads and Autonomous Driving (AD), diverse situational aspects are relevant for different stakeholders. Stakeholders include traffic participants, vehicle or component manufacturers, insurance, road network operation and maintenance, multi modal traffic planning, and test field operation. Each stakeholder is interested in specific information or aspects of smart road services, based on their respective responsibilities to perform (a group of) tasks.
Currently, there is no universally accepted information model covering all conceivable aspects of smart road services. Towards the vision of such a unified reference information model, a concise set of requirements is proposed in Section III-C that serves as a starting point.
C. Requirements on Information Models
Different smart road services focus on different sub-sets of information. This might introduce a tendency to application specific sub-sets that express the same information represented in different (and possibly incompatible) ways. A unified reference information model would support the convergence of development efforts and interoperability of systems. Towards such a universally acceptable reference information model, a requirement-driven approach appears best suited.
Table 2 proposes requirements on information to facilitate development and standardization efforts towards a reference information model. Thereby, the focus is on information content (what), and not on systems, sensors, hardware, or communication technology (how), which are covered in Section IV.
The first two requirements in Table 2 are closely related to the discussion in Section III-A, where an argument for the information level is made. Requirement
Assume these requirements can be met and an agreeable reference information model can be defined as a super-set of relevant information. Then, one can investigate how this reference model relates to the Infrastructure Support Levels for Automated Driving (ISAD) [29], [30].
D. Reference Information Model, Odd and ISAD
A starting point to explore the relationship between a function’s or system’s ODD and ISAD, is the definition of ODD. The ODD (of a function or system) describes the Operational Conditions (OC), in which the function or system is designed to operate. Any scenario might be fully inside, outside or partially inside the ODD, depending on the involved tasks and decisions that a system has to cope with. Each task or decision requires characteristic information to deal with.
This implies, that for any scenario to be within a system’s ODD, all necessary information must be made available to the system. Let this set of information be denoted necessary information. Note that necessary information is specific to both the scenario and the system.
As stated in Section I, this information might be acquired by a vehicle’s on-board sensors or static databases, or via communication. Let information communicated from the infrastructure be denoted smart road service information. Then, ISAD describes the set of smart road service information together with its quality, that is provided by the infrastructure at any given time and location.
It is beneficial to view necessary information and smart road service information as subsets of a common superset. This might seem trivial, but ensures compatible definitions of information among road users and infrastructure. Assuming this information superset can be unified across relevant scenarios, it can be denoted as reference information model, see Fig. 1.
Necessary information for certain tasks can be covered by different information sources, e.g., task X (light blue) can be covered solely by on-board sensor information (dash-dotted) even if smart road service (dashed) could support the task as well. Whereas task Y (dark blue) can only be fulfilled by utilizing smart road services in addition to on-board sensor information.
A reference information model harmonizes the views of both sender (e.g., infrastructure) and receiver (e.g., road user) of information, meeting the requirements stated above. Therefore, it aims to be valid across stakeholders, manufacturers and countries. Three application strategies for smart road service information for ADAS/AD systems can then be utilized, see Table 3, which are illustrated by means of a working example.
Imagine an ADAS for lane keeping. The system relies on on-board camera sensors to detect driving lanes. On-board camera performance deteriorates with declining visibility that might be caused by bad weather conditions.
The example relies on the concepts capability of a system and complexity of the task (or decisions) [10]. Any task can be associated with a scenario-specific complexity, whereas an ADAS/AD function has a certain capability to perform a task. If the capability exceeds the complexity of the task, it can be completed, otherwise it can not be completed safely.
In this working example, a task is lane detection. Its complexity is related to, among other things, the visibility range in relation to driving speed. A given lane detection subsystem exhibits a certain capability, namely, it will work safely up to a specific range of view.
If the infrastructure communicates degraded visibility conditions in a specific region, a vehicle approaching this region might predictively assess, that its lane detection capability will be below the complexity of the task. It therefore might re-route to avoid disengagements caused by leaving its ODD. This is an example for strategy S1 in Table 3.
If the infrastructure provides lane geometry information in the region with degraded visibility, the vehicle would be able to perform the lane keeping task despite degraded visibility. Thus, the systems capabilities and ODD are effectively expanded by smart road service information. This is an example for strategy S2.
Another option would be to automatically reduce the maximum driving speed for all traffic participants on the Section in question, such that the driving speed is appropriate for the visibility range. Thus, the scenario complexity is reduced to a level where the system’s capability exceeds the scenario’s complexity. This is an example of strategy S3.
The concept and definition of ISAD was proposed in the EU project INFRAMIX.2 ISAD also includes a Quality of Service description that encompasses attributes of the information such as accuracy, reliability and age of the information. However, the current concept of ISAD and its implications for infrastructure providers and managers are still disputed [34], [35], [36]. It is to be expected that the concept of ISAD will evolve with better understanding of the requirements. Note, the current concept of ISAD does not strictly comply with the requirements presented in Section III-C.
Requirements on a Technical Realization
For a practical realization, a broad spectrum of stakeholders are to be considered, including drivers (human or automated), traffic managers, infrastructure operators, public transport providers, and urban planners. Performing their respective tasks, each group makes heterogeneous decisions based on specific information. Hence, a digital twin with this purpose can be called Cooperative, Connected, Automated Mobility (CCAM) Decision Support Platform (DSP). In order to capture relevant requirements, a thorough stakeholder analysis has been performed [37], modelling stakeholder groups, their tasks, and the information necessary to perform each task. This allows to deduce on stakeholder and task level,
which information is required
in what accuracy, and
how recent it needs to be.
Guided by these context models, requirements for a technical realization were identified and are presented in Table 4. The requirement descriptions make use of the terms frontend and backend. A backend is a collection of algorithms that turn data into information. Data can be acquired from different (possibly redundant) sources and may be processed in various ways, including filtering, smoothing, aggregating, combining, estimating, forecasting, and augmenting. A frontend is a system that uses information to extract and present use-case-specific knowledge or make decisions. This may involve visualization, communication, assessment, contextualization, weighing, recommendation or even automated decision making. A backend can be considered a producer of information, a frontend can be considered a consumer of information.
The requirements cover a wide range of different areas and are demanding in their entirety. Some of the requirements mentioned are common in the field of machine and factory automation, where time determinism and guarantees are to be achieved. Others demand flexibility and reliability, as can be achieved through fault-tolerant redundant and distributed systems. In addition, security, and interoperability are important issues.
It is obvious that the number and diversity of requirements demands a mature and proven technology. It is also obvious that not every application can be realized cost-effectively by one and the same system. Thus, a suitable platform architecture is to be aimed for.
Solution Concept
Based on the identified requirements and the level of abstraction discussed above, this Section proposes an architecture and a solution strategy that, in the authors opinion, allows compliance with all stated requirements.
A. Architecture
Central questions in the definition of a (platform) architecture are the distribution of tasks to individual components, as well as finding suitable interfaces between these components. The architecture design presented here is based on the DIKW pyramid and the abstraction levels from Section III-A. This allows a simple representation of the basic data flow, shown in Figure 2 with exemplary front- and backend configurations.
Architecture of CCAM DSP realizations. Right margin: Mapping to information pyramid.
In this model, content generally flows from bottom to top. Backends use potentially multiple data sources (e.g., sensors, databases, C-ITS messages) to process information. This might involve filtering, noise reduction, sensor fusion, failure handling, missing value replacement, error bounds estimation or prediction as examples. As the provider of information in this scheme, a backend must know the relevant part of the information model (cf. Section III-B) and implement a compliant parameterization. A backend can also be sensorless and act as a virtual sensor, relying only on information provided by other backends and aggregating it into higher level information, as indicated in Fig. 2 by the rightmost backend. For example, a traffic simulation may rely on traffic flow information as boundary conditions and may provide optimized traffic control regimes as information to the CCAM DSP.
Up to this point, the information layer can be referred to as a digital shadow according to [2], since it is an automatically created image of the real world. A digital twin additionally requires automated feedback into the real world, which can be a task of a frontend. Note that providing automated feedback into the real world may create control loops, showing that a control system can be implemented in accordance with this architecture.
A frontend tailors the provided information to the needs of a specific stakeholder. Its tasks include sharing relevant aspects over a network, visualization, monitoring and evaluation, the suggestion of certain options for action as a decision support system, to a fully automated decision and initiation of corresponding actions. The tasks of a frontend also include compliance with relevant standards and formats, e.g., when information aspects are to be made available via standardized network protocols or C-ITS messages that are broadcast by Roadside Unit (RSU). Additionally, use-case specific role and authorization concepts must be provided by the frontend.
Note that the architecture does not dictate that a frontend must take all of its input from the information layer. For example, if a surveillance system requires the integration of live camera images, these can of course be tapped directly by the frontend and do not have to be made available in the information layer. On the contrary: a camera image does not fulfill the requirements for information (Section III-C) but is rather to be classified as a signal.
The information layer is of central importance in this architecture. If the information models are sufficiently standardized, front- and backends can be reused as modular components to create concrete system realizations (provided that the specific technical constraints are sufficiently similar). This once again highlights the need for clean and standardized information models (cf. Section III-B).
The proposed architecture enables modular development and operation of such a CCAM DSP, Fig. 3 depicts the stakeholder model. The CCAM DSP User is the entity that makes decisions based on information made available by a frontend and possibly additional information collected from other sources. This stakeholder may be, e.g., an ADAS or a highway control manager. The CCAM DSP Operator makes sure that the CCAM DSP is up and running as expected. The manufacturer of a CCAM DSP integrates and deploys all components of a CCAM DSP that are supplied by frontend and backend developers such that the information pool complies with relevant standards, developed by a standardization body. Lastly, a sensor operator makes sure that relevant sensors are up and running as expected and thereby ensures that necessary measurement data are provided to the backend module.
For the demonstrators presented in this work, most stakeholders are represented exclusively by the authors with two exceptions: Sensor Operator and CCAM DSP User, which were also represented by the operators of the existing data sources used and the project principals, respectively, see Fig. 4 for details.
Overview of the demonstrator setups with backend data sources (bottom row), extracted information groups (text) and frontend implementation (top row).
B. Technical Basis
As described, the information layer is the pivotal point of the proposed architecture. Therefore, the selection of an appropriate technological foundation is crucial for a successful implementation. At first glance, many technologies from the areas of Industrial Internet of Things, Semantic Web, Cloud Computing, or Databases are candidates. Among those, Open Platform Communications Unified Architecture (OPC UA)3 meets the full spectrum of requirements from Section IV. The existing body of research also suggests, that tools developed for manufacturing such as OPC UA can advantageously used for digital twins in smart city applications as well [14]. In [38] an argument is made for OPC UA and in [17] a set of requirements on information models is provided. Therefore, we consider and favor OPC UA as an implementation technology.
OPC UA is not a protocol, data format, tool, or single standard, but rather an ecosystem that includes all of these for demanding applications, interoperable and vendor-independent. It defines a standardized service-oriented architecture that enables platform-independent data exchange, e.g., [39]. It is the de-facto standard in process and machine automation and has some properties that suit the application at hand. In the following, this statement is underpinned by checking OPC UA against the requirements in Table 4.
As shown, OPC UA meets the above requirements in theory. In order to investigate the suitability in practice, several demonstrators were set up.
Demonstration
Within this Section, three implementations for CCAM Decision Support Platforms are presented. The first implementation is located in Austria and covers a highway Section with a tunnel construction site, intending to support automated vehicles when travelling along the ODD-critical section. The second implementation is based in Germany and consists of a digital information representation of traffic data collected by the Test Bed Lower Saxony. Its two use cases are to support automated vehicles and infrastructure managers, respectively. The third implementation covers COPE4 [32], a collective perception system for protecting vulnerable road users on urban intersections. It is located at an urban intersection in Hallein, Austria.
An in-depth description of each demo implementation, its use cases and a technical description of the implementation is provided in the following Sections. Fig. 4 and Table 5 provide an overview of the implementations and use-cases. Note, that these implementations are proof-of-concept prototypes and are not in any way optimized for commercial deployment. The hardware specs merely indicate, that a prototype can run on consumer grade computers and achieve acceptable latencies. The algorithm complexity of a backend or frontend depends exclusively on the particular implementation. Hence, the algorithm complexity given in Table 5 reflects the complexity of the implemented backends in these demonstration implementations and is not to be mistaken for a general limit or target.
A. AUSTRIA: A10
This first demonstration covers a Section of the A10 highway in Austria where a tunnel renovation is planned. The CCAM DSP operator is the highway operator and manager (ASFINAG) which, as a possible future scenario, aims to support automated vehicles as users, that pass through the construction site. A thorough analysis of relevant driving tasks, their respective anticipated demand for supplementary information, as well as the potential hazard in case of an automated driving function failure was performed. Relevant information groups that are best provided by the infrastructure were identified. These include the topography of the highway section, points of particular interest, e.g., tunnel portals, lane geometry, restrictions and availability, and the applicable rules and regulations, e.g., the speed limit.
This demo implementation addresses S1 and S2 as described in Table 3 by providing additional information to vehicles. Scenarios that were previously out-of-ODD for an ADAS/AD system may now be inside-ODD.
Utilized data sources from which information is extracted are the Graphenintegrations-Plattform (GIP) data Austria, which provides open government data (quality controlled and versioned) of the Austrian road network and several content streams provided by ASFINAG, in particular the DATEX II profiles TRAFFIC SIGNS STATIC, TRAFFIC SIGNS DYNAMIC, PLANNED EVENTS and UNPLANNED EVENTS. The first two contain data of traffic signs (e.g., location, type of sign, vehicle class it applies to, numeric information about the speed limits) from which the speed limit for each vehicle class at all points of the highway can be derived. The last two streams contain data on planned construction sites and incidents, e.g., location, duration, lane blockings and physical restrictions. GIP data are updated approximately every two months and are therefore considered quasi-static data. In contrast, the content streams are updated every 15 minutes and can be considered dynamic data.
The OPC UA server is based on open62541, 5 an open source implementation for OPC UA Servers. A toolchain that supports information model generation in MATLAB® and produces a valid information model file for compilation has been developed. Two backends are implemented that combine data from different sources, extract information, cf. Table 2, and provide the live information to the OPC UA server via a information-specific UDP packet format. The highest algorithm complexity is
A frontend is implemented in MATLAB® that reads information from the OPC UA server and visualizes selected information in a dashboard. In a full implementation, another frontend would read information, convert it into C-ITS messages and cyclically broadcast them via road-side units to the user, i.e., the automated vehicle (omitted in the demonstration).
B. Germany: Test Bed Lower Saxony (TBLS) A2
The second demonstration aims to provide information for two different user groups and includes content captured by the Test Bed Lower Saxony (TBLS). The TBLS is a research infrastructure built by the German Aerospace Center (DLR) on public roads A39, A7, A2, and a number of rural roads in Lower Saxony (Germany) in 2020 [40]. It consists of physical road infrastructure (camera-based object detection and V2X RSUs), a virtual replica (models and scenarios for simulation), and a digital content catalog. The digital content catalog contains, among other sources, content provided by the Traffic Management Center (TMC) in Lower Saxony.6 The content of the TMC consists of three different types: loop detector content, dynamic traffic sign content, and weather content. The demonstration use cases Intelligent Speed Advisory, Traffic Volume Dashboard and Temperature Dashboard as shown in Fig. 4 are implemented based on this content.
Technically, each loop detector and weather station sends a measured reality update every minute that includes a station identifier and the measurement values. The content of the dynamic traffic signs is updated when the content of the sign changes (on demand) and is also containing an identifier and payload information. A static database provides the meta information (like location) about the infrastructure points for, e.g., to geographically locate the measurement.
The OPC UA implementation for the Test Bed Lower Saxony is based on Python [41] in version 3.8. It is divided in three modules: OPC UA server, OPC UA updater (backend) and OPC UA frontend.
The OPC UA modules are based on the open-source Python library opcua-asyncio.7 The server component consists of the data/information model and the OPC UA interfaces. The update module fetches updates via Apache Kafka, converts data into information, and stores information in the information model within the server module.
To generate the information, the incoming content is distinguished according to the perspective of the user. For example, content provided by the loop detectors is viewed from two different perspectives:
infrastructure perspective
automated vehicle perspective
A loop detector sends a measurement update of the vehicles passed and their speed for a particular time stamp. From an infrastructure perspective, the quality of the measurement, the number of updates received in the past, and aged or missed measurements are of interest to determine the status of the infrastructure and to plan maintenance if necessary. Thus, the relevant information, such as the number of updates received in the last hour and the corresponding failure rate along the location of the loop detector, is contained in the information pool.
the perspective of an Autonomous Vehicle (AV), e.g., the information about traffic volume along its route is of interest to determine optimal routes and velocities. Thus, it mainly addresses S1 (see Table 3) by providing additional information to vehicles. Therefore, for AVs the information about the traffic volume and speed along the course of the road is relevant rather than the single measurement values of nearby loop detectors. Hence, the measurements are modeled as two sorted arrays containing the information about the loop detector’s track kilometers and the respective traffic volume.
The other data sources (dynamic traffic signs and weather content) are modeled analogously from an AV and infrastructure perspective.
The frontend is realized using a dashboard visualizing the information model. To transmit the information from the OPC UA server to the frontend, the subscription mechanism of the OPC UA server is used.
Fig. 5 shows the dashboard for the loop detector data. The dashboard client allows to chose the visualization perspective of the information model (vehicle or infrastructure perspective). It provides the quantity and velocity of passenger cars, trucks, or all vehicles combined.
Frontend visualization for loop detector traffic flow content. The color indicates the number of vehicles per hour.
Fig. 6 combines both perspectives into one visualization of the dynamic traffic signs. The visualization is limited to the information of the dynamic traffic signs regarding speed limits (information on, e.g., the “no overtaking” sign is omitted for the prototype). The information for an AV is shown for each lane and representing the value, beginning, and end of the limitation. For simplicity, the information is not restricted to specific vehicle types. Whereas, the information for the infrastructure perspective is displayed with its location and status.
Visualization of dynamic traffic sign information: combined vehicle and infrastructure perspective. Blue points indicate a dynamic traffic sign with its infrastructural information (e.g., last update). Continuous lines represent the allowed speed on the given lane (yellow - 100 kph speed limit, green - no limit).
C. Austria: Cope - Collective Perception
The Austrian project COPE aims at protecting Vulnerable Road Users (VRU) at urban intersections by utilizing collective perception. This is a safety-relevant instance of S1, cf. Table 3, where strict requirements concerning information dynamics, latency and information quality apply.
At an urban intersection equipped with sensors (video cameras) traffic participants are detected utilizing object recognition and object tracking. In addition, V2X-enabled vehicles can provide telemetry. A backend generates and supplies information about all recognized traffic participants (location, orientation, velocity, acceleration, type, etc.) based on sensor an tracking data. Augmenting the traffic participant list, another backend provides information about collision risk by applying behaviour models to predict future locations of traffic participants. The highest algorithm complexity is
Conclusion
The term digital twin seems to have become a projection screen for a wide variety of topics and applications. Focusing on the context of CCAM and smart road services, this work proposes a CCAM acrlong DSP, which is a digital twin [2] in the general sense, albeit with a specific scope, crisp requirements, versatile architecture and mature technologies that enable practical implementations.
The main contributions of this work can be summarized as follows:
An appropriate abstraction level for communication between traffic participants and smart road services is established. A set of requirements on information is proposed that facilitates the development of interoperable information models.
The relationship between information, ODD and ISAD is clarified. Three taxonomic strategies to utilize smart road information to support CCAM are identified.
The requirements on a digital twin, i.e., a CCAM DSP, for road and traffic infrastructure are analysed and formulated comprehensively. Relevant stakeholders involved in road traffic, including asset management and maintenance, traffic management and control as well as CCAM are considered.
An architecture for a DSP is proposed and a technological basis for its realization (OPC UA) is identified, which is capable of meeting the defined requirements.
The overall solution concept is validated with three different implementations and several demonstration use cases for each implementation.
Not all open questions have been addressed and solved with the presented approach, but the proposed architecture is considered to be a sound basis for technical realizations. Different implementations may differ significantly (e.g., tooling and SDKs, hardware and operating systems, networking and degree of distribution, real-time capabilities, security, safety relevance, redundancy). However, architecture and information modelling principles are shown to be sufficiently generic to support a broad variety of use cases. The demonstrators presented show the feasibility of the architecture, but do not represent optimized or production-ready systems.
Whereas standards for V2X messages are relatively far developed, e.g., [27], other details of frontend-to-user-communication such as, e.g., push/pull-mechanisms and efficient querying, are not generally specified. For both frontends and backends, reusability of modules increases with a growing body of harmonized standards.
Standardised information models can support the harmonized formulation of laws and regulations and build a foundation for a technology-independent description of information sources. It is critical that information modelling is done in a way that facilitates effective cooperation between infrastructure and ADAS/AD functions. A practical roll-out strategy might be to develop partial reference information models that are sufficient for a set of intended tasks and decisions. Such a model can be extended once additional tasks are implemented and corresponding necessary information becomes relevant. The proposed requirements on information model support the development of (partial) suitable reference models as a joint effort of involved stakeholders.
Outlook
A number of follow-up initiatives can be derived when utilizing CCAM DSP for smart road services:
The requirements on information models proposed in this work are expected to contribute significantly to the joint effort of standardization across all stakeholder groups. In upcoming activities, the proposed requirements should be expanded and integrated into the standardization efforts.
The demonstrators presented show the proof-of-concept of the proposed DT architecture but do not represent optimized implementations or concrete proposals for specific applications. Future work should propose concrete, production-ready data models and implementations for specific applications.
In future CCAM projects, we propose that diverse CCAM DSP user groups (e.g., back-end and front-end providers, infrastructure operators) should cooperate on the topic This encourages cooperative information modelling, standardization, the description of data quality and meta requirements on information for particular tasks, matching user expectations to frontend developer and DSP operator assumptions. In such a group aspects that remained unsolved within the proof-of-concept could be treated, i.e., evolving a modelling guideline for an effective DT with lower or higher latency demands depending of the intended range of applications or identification of possible set-ups for the communication structures and functional distribution between the frontend and the DSP users for specific applications. Furthermore, the presented demonstrators only showed the general functionality of the described approach using OPC UA for the implementation of a DSP. In future activities a comprehend approach could look at a higher variety of test bed data sources and CCAM applications in order to enhance the concept on a broader base of information. This would allow as well to further define hardware requirements in relation to information model structure and size.
In general, the task of interface standardization will play a major role for the evolution of high-level ISAD applications. Future work, such as the GAIA-X 4 Future Mobility8 project family, will focus on providing a standardized service infrastructure for smart road services. The GAIA-X service layer could serve as the access point for a digital twin of a certain road Section and thus, for all active CCAM services on that section. This way, an easy access to CCAM through a DSP as proposed in this paper could be implemented.
Another topic of high urgency is the definition and resolution of binding laws and regulations. This applies to both, the operation of infrastructure and the machine-readable encoding of existing laws. For the latter, a first step was taken, e.g., in the lex2Vehicle project [21], but further research and development is necessary.