An Effective Macro Placement Algorithm Based On Curiosity-Driven Reinforcement Learning

In mixed-size circuit designs, due to the numerous connections with standard cells and huge sizes of macros, the positions of macros greatly affect the placement of standard cells and then the layout quality. In this work, we use an on-policy reinforcement learning method to train an agent from scratch and make the agent learn macro placement. To satisfy the non-overlap constraint of macro placement, we design a fast non-overlapping algorithm. To get better feature extraction of global and local state information, we adopt a convolutional neural network (CNN) and an edge-based graph neural network (Edge-GNN). In addition, we add intrinsic reward as the curiosity in the reward design to encourage the agent to explore, helping the agent better understand how to place macros effectively. Experiment results show that our proposed framework is more effective in optimizing the wirelength of the placement compared with a simulated annealing method.