Controllable Diffusion Models for Safety-Critical Driving Scenario Generation

Safety-critical driving scenarios are essential to the development and validation of autonomous driving algorithms. Currently, most of the data is acquired in naturalistic scenarios, resulting in a sparsity of the safety-critical cases. Consequently, synthetic scenario generation based on deep models becomes crucial to validate the risk and reduce the cost. However, previous works either fail to generate realistic scenarios with high quality or can hardly follow instructions to generate desired scenarios. In this paper, we propose a controllable diffusion model that operates on a naturalistic driving scenario to generate safety-critical cases with high fidelity and controllability. In particular, our proposed method encompasses a pre-trained text-to-image diffusion model and a bounding box to incorporate the conditions of category and position of generated objects, respectively. To enhance generation quality and mitigate boundary artifacts, we introduce a mask-aware adapter to better integrate the generated objects into the driving scenarios. Moreover, we propose a transformer encoder and region-guided cross attention to fuse the additional coordinate inputs into the diffusion model, while encouraging more interaction between different generated objects. Comparative analysis demonstrates that our method outperforms exciting work by offering more realistic, diverse and controllable synthetic scenarios and allowing for multiple objects generation with complex spatial relationship.