A 12-nm Autonomous Driving Processor With 60.4 TOPS, 13.8 TOPS/W CNN Executed by Task-Separated ASIL D Control

This article proposes an automotive safety integrity level (ASIL) D, which is the highest automotive integrity level specified in ISO 26262, and a target application processor for driving automation systems with a high convolutional neural network (CNN) processing performance of 60.4 trillion operations per second (TOPS) with high power efficiency of 13.8 TOPS/W. To achieve both performance and power efficiency, a stream processing architecture specialized for CNN hardware module is adopted to implement many low-power arithmetic units. In addition, 6 MB of local memories are implemented to reduce data transfers between dynamic random access memory (DRAM) and the CNN hardware, thereby improving the execution efficiency of the arithmetic units and saving power consumption caused by the data transfer. In order to support high safety integrity level with power efficiency, we provide safety mechanisms with high failure detection rates only for the hardware used in applications that require ASIL D support. The configuration of two CNN modules can switch between lockstep and single operation depending on the use case, thus achieving a tradeoff between performance and ASIL while maintaining power efficiency. In addition, the freedom from interference (FFI) concept defined in ISO 26262 is achieved by providing mechanisms to prevent low safety level tasks from interfering with high safety level tasks running concurrently inside the processor.

I. Introduction

The widely used classification of driving automation systems is the six levels defined by the SAE International in J3016 [9], [10], as shown in Table I. In level 0 to level 2 driving automation systems or driver-assistance features, the driver must constantly monitor the driving environment and take responsibility for driving safety. Level 3 and above will be an automated driving system, and the system will assume the driving responsibilities that the human driver had in level 2 and below. Therefore, the system, when deployed to market, will require accurate and high-speed recognition, judgment, and operation. Research on automated driving has been conducted across multiple disciplines, including recognition of the surrounding environment, localization, planning, and control [13]–[16]. Convolutional neural networks (CNNs) have made a breakthrough in the field of pattern recognition [11], various CNNs have been proposed, and their performance for pattern recognition has been greatly improved [6], [19]–[22]. However, it requires a lot of computation. If the CNN processes the information input from each sensor about 30 times per second, the amount of processing can be several hundred giga operations per second (GOPS) to 1 tera operation per second (TOPS) per sensor. In automated driving systems, the number of installed sensors has increased to about 20, and the data from each sensor are processed by multiple CNNs for different applications, such as recognition and segmentation. To output vehicle operation commands based on the information from the sensors, these multiple CNNs need to be processed in real time within a time frame of about 100 ms. Considering these, a total CNN processing performance of over 100 TOPS is required. For example, Tesla has installed two 72 TOPS chips in its automated driving system to achieve a total of 144 TOPS of CNN processing performance [24]. However, the high power consumption associated with the high computing performance of CNNs requires expensive cooling systems, such as water cooling, and hinders their widespread use. Since the air-cooled system is low-cost and lightweight, power consumption of 25 W or less is required to make it feasible. Comparing the computational performance and power consumption of existing CNN processors [1]–[5], [7], [12], [17], as shown in Fig. 1, a performance in the range of 100–120 TOPS with an energy efficiency of 10 TOPS/W is the sweet spot that balances cost and performance at present.TABLE I Levels of Driving Automation

Level	What the features do:	What the driver must do
0	Warning and momentary assistance	The driver is driving whenever the features are engaged. The driver must constantly supervise the features.
1	Steering or brake/acceleration support
2	Steering and brake/acceleration support
3	Driving vehicle under limited conditions	The driver is not driving when the features are engaged. When the feature requests, the driver must drive.
4	Driving vehicle under limited conditions	The driver is not driving when the features are engaged. The feature will not require the driver to take over driving.
5	Driving vehicle under all conditions

Note that this is an excerpt from the J3016 automated-driving graphic.

Fig. 1.

CNN performance/watt target of this work shown on a scatter plot of performance versus power for AI accelerators and processors.