Abstract:
HD map reconstruction is crucial for autonomous driving. LiDAR-based methods are limited due to expensive sensors and time-consuming computation. Camera-based methods usu...Show MoreMetadata
Abstract:
HD map reconstruction is crucial for autonomous driving. LiDAR-based methods are limited due to expensive sensors and time-consuming computation. Camera-based methods usually need to perform road segmentation and view transformation separately, which often causes distortion and missing content. To push the limits of the technology, we present a novel framework that reconstructs a local map formed by road layout and vehicle occupancy in the bird’s-eye view given a front-view monocular image only. We propose a front-to-top view projection (FTVP) module, which takes the constraint of cycle consistency between views into account and makes full use of their correlation to strengthen the view transformation and scene understanding. In addition, we apply multi-scale FTVP modules to propagate the rich spatial information of low-level features to mitigate spatial deviation of the predicted object location. Experiments on public benchmarks show that our method achieves various tasks on road layout estimation, vehicle occupancy estimation, and multi-class semantic estimation, at a performance level comparable to the state-of-the-arts, while maintaining superior efficiency.
Published in: IEEE Transactions on Pattern Analysis and Machine Intelligence ( Volume: 46, Issue: 9, September 2024)
Funding Agency:
College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China
Wenxi Liu received the PhD degree from the City University of Hong Kong, in 2014. He is a professor with the College of Computer and Data Science, Fuzhou University, China. His research interests include crowd analysis, image segmentation, and image processing.
Wenxi Liu received the PhD degree from the City University of Hong Kong, in 2014. He is a professor with the College of Computer and Data Science, Fuzhou University, China. His research interests include crowd analysis, image segmentation, and image processing.View more
College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China
Qi Li received the BE degree from Fuzhou University. He is currently working toward the doctoral degree with the College of Computer and Data Science, Fuzhou University, China. His research interest lies primarily in computer vision.
Qi Li received the BE degree from Fuzhou University. He is currently working toward the doctoral degree with the College of Computer and Data Science, Fuzhou University, China. His research interest lies primarily in computer vision.View more
College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China
Weixiang Yang received the ME degree from Fuzhou University, in 2022. His research interests lies primarily in image segmentation.
Weixiang Yang received the ME degree from Fuzhou University, in 2022. His research interests lies primarily in image segmentation.View more
College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China
Jiaxin Cai is currently working toward the master's degree with the College of Computer and Data Science, Fuzhou University, China. His research interest lies primarily in computer vision.
Jiaxin Cai is currently working toward the master's degree with the College of Computer and Data Science, Fuzhou University, China. His research interest lies primarily in computer vision.View more
College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China
Yuanlong Yu received the PhD degree in electrical engineering from the Memorial University of Newfoundland, St. Johns, NL, Canada, in 2010. Since 2011, he worked as a postdoctoral fellow respectively with the Memorial University of Newfoundland and Dalhousie University, Halifax, NS, Canada. Since 2013, he has been working as a professor with Fuzhou University, Fuzhou, China. His main interests are computer vision, machine...Show More
Yuanlong Yu received the PhD degree in electrical engineering from the Memorial University of Newfoundland, St. Johns, NL, Canada, in 2010. Since 2011, he worked as a postdoctoral fellow respectively with the Memorial University of Newfoundland and Dalhousie University, Halifax, NS, Canada. Since 2013, he has been working as a professor with Fuzhou University, Fuzhou, China. His main interests are computer vision, machine...View more
ShanghaiTech University, Shanghai, China
Yuexin Ma received the PhD degree from the Department of Computer Science, University of Hong Kong, in 2019. Now, she is an assistant professor with ShanghaiTech University, China, leading 4DV Lab. Her research interests lie on the computer vision and deep learning. Particularly, she is interested in 3D scene understanding, multi-modal perception, human-machine cooperation, and autonomous driving. She has published dozens...Show More
Yuexin Ma received the PhD degree from the Department of Computer Science, University of Hong Kong, in 2019. Now, she is an assistant professor with ShanghaiTech University, China, leading 4DV Lab. Her research interests lie on the computer vision and deep learning. Particularly, she is interested in 3D scene understanding, multi-modal perception, human-machine cooperation, and autonomous driving. She has published dozens...View more
Singapore Management University, Singapore
Shengfeng He (Senior Member, IEEE) received the BSc and MSc degrees from the Macau University of Science and Technology, in 2009 and 2011, respectively, and the PhD degree from the City University of Hong Kong, in 2015. He is an associate professor with the School of Computing and Information Systems, Singapore Management University. He was on the faculty of the South China University of Technology, from 2016 to 2022. His...Show More
Shengfeng He (Senior Member, IEEE) received the BSc and MSc degrees from the Macau University of Science and Technology, in 2009 and 2011, respectively, and the PhD degree from the City University of Hong Kong, in 2015. He is an associate professor with the School of Computing and Information Systems, Singapore Management University. He was on the faculty of the South China University of Technology, from 2016 to 2022. His...View more
University of Hong Kong, Pok Fu Lam, Hong Kong
Jia Pan received the PhD degree in computer science from the University of North Carolina at Chapel Hill, Chapel Hill, NC, USA, in 2013. He is currently an associate professor with the Department of Computer Science, University of Hong Kong, Hong Kong. He is also a member of the Centre for Garment Production Limited, Hong Kong. His research interests include robotics and artificial intelligence as applied to autonomous sy...Show More
Jia Pan received the PhD degree in computer science from the University of North Carolina at Chapel Hill, Chapel Hill, NC, USA, in 2013. He is currently an associate professor with the Department of Computer Science, University of Hong Kong, Hong Kong. He is also a member of the Centre for Garment Production Limited, Hong Kong. His research interests include robotics and artificial intelligence as applied to autonomous sy...View more
Contents College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China
Wenxi Liu received the PhD degree from the City University of Hong Kong, in 2014. He is a professor with the College of Computer and Data Science, Fuzhou University, China. His research interests include crowd analysis, image segmentation, and image processing.
Wenxi Liu received the PhD degree from the City University of Hong Kong, in 2014. He is a professor with the College of Computer and Data Science, Fuzhou University, China. His research interests include crowd analysis, image segmentation, and image processing.View more
College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China
Qi Li received the BE degree from Fuzhou University. He is currently working toward the doctoral degree with the College of Computer and Data Science, Fuzhou University, China. His research interest lies primarily in computer vision.
Qi Li received the BE degree from Fuzhou University. He is currently working toward the doctoral degree with the College of Computer and Data Science, Fuzhou University, China. His research interest lies primarily in computer vision.View more
College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China
Weixiang Yang received the ME degree from Fuzhou University, in 2022. His research interests lies primarily in image segmentation.
Weixiang Yang received the ME degree from Fuzhou University, in 2022. His research interests lies primarily in image segmentation.View more
College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China
Jiaxin Cai is currently working toward the master's degree with the College of Computer and Data Science, Fuzhou University, China. His research interest lies primarily in computer vision.
Jiaxin Cai is currently working toward the master's degree with the College of Computer and Data Science, Fuzhou University, China. His research interest lies primarily in computer vision.View more
College of Computer and Data Science, Fuzhou University, Fuzhou, Fujian, China
Yuanlong Yu received the PhD degree in electrical engineering from the Memorial University of Newfoundland, St. Johns, NL, Canada, in 2010. Since 2011, he worked as a postdoctoral fellow respectively with the Memorial University of Newfoundland and Dalhousie University, Halifax, NS, Canada. Since 2013, he has been working as a professor with Fuzhou University, Fuzhou, China. His main interests are computer vision, machine learning, visual attention, and cognitive robotics.
Yuanlong Yu received the PhD degree in electrical engineering from the Memorial University of Newfoundland, St. Johns, NL, Canada, in 2010. Since 2011, he worked as a postdoctoral fellow respectively with the Memorial University of Newfoundland and Dalhousie University, Halifax, NS, Canada. Since 2013, he has been working as a professor with Fuzhou University, Fuzhou, China. His main interests are computer vision, machine learning, visual attention, and cognitive robotics.View more
ShanghaiTech University, Shanghai, China
Yuexin Ma received the PhD degree from the Department of Computer Science, University of Hong Kong, in 2019. Now, she is an assistant professor with ShanghaiTech University, China, leading 4DV Lab. Her research interests lie on the computer vision and deep learning. Particularly, she is interested in 3D scene understanding, multi-modal perception, human-machine cooperation, and autonomous driving. She has published dozens of papers on the IEEE Transactions on Pattern Analysis and Machine Intelligence, International Journal of Computer Vision, CVPR, ICCV, ECCV, IJCAI, AAAI, etc.
Yuexin Ma received the PhD degree from the Department of Computer Science, University of Hong Kong, in 2019. Now, she is an assistant professor with ShanghaiTech University, China, leading 4DV Lab. Her research interests lie on the computer vision and deep learning. Particularly, she is interested in 3D scene understanding, multi-modal perception, human-machine cooperation, and autonomous driving. She has published dozens of papers on the IEEE Transactions on Pattern Analysis and Machine Intelligence, International Journal of Computer Vision, CVPR, ICCV, ECCV, IJCAI, AAAI, etc.View more
Singapore Management University, Singapore
Shengfeng He (Senior Member, IEEE) received the BSc and MSc degrees from the Macau University of Science and Technology, in 2009 and 2011, respectively, and the PhD degree from the City University of Hong Kong, in 2015. He is an associate professor with the School of Computing and Information Systems, Singapore Management University. He was on the faculty of the South China University of Technology, from 2016 to 2022. His research interests include visual understanding and generative models. He is a senior member of CCF. He serves as the lead guest editor of the International Journal of Computer Vision and the associate editor of the Neurocomputing.
Shengfeng He (Senior Member, IEEE) received the BSc and MSc degrees from the Macau University of Science and Technology, in 2009 and 2011, respectively, and the PhD degree from the City University of Hong Kong, in 2015. He is an associate professor with the School of Computing and Information Systems, Singapore Management University. He was on the faculty of the South China University of Technology, from 2016 to 2022. His research interests include visual understanding and generative models. He is a senior member of CCF. He serves as the lead guest editor of the International Journal of Computer Vision and the associate editor of the Neurocomputing.View more
University of Hong Kong, Pok Fu Lam, Hong Kong
Jia Pan received the PhD degree in computer science from the University of North Carolina at Chapel Hill, Chapel Hill, NC, USA, in 2013. He is currently an associate professor with the Department of Computer Science, University of Hong Kong, Hong Kong. He is also a member of the Centre for Garment Production Limited, Hong Kong. His research interests include robotics and artificial intelligence as applied to autonomous systems, particularly for navigation and manipulation in challenging tasks such as effective movement in dense human crowds and manipulating deformable objects for garment automation.
Jia Pan received the PhD degree in computer science from the University of North Carolina at Chapel Hill, Chapel Hill, NC, USA, in 2013. He is currently an associate professor with the Department of Computer Science, University of Hong Kong, Hong Kong. He is also a member of the Centre for Garment Production Limited, Hong Kong. His research interests include robotics and artificial intelligence as applied to autonomous systems, particularly for navigation and manipulation in challenging tasks such as effective movement in dense human crowds and manipulating deformable objects for garment automation.View more
