Automatic Digitization of Engineering Diagrams using Deep Learning and Graph Search

A Piping and Instrumentation Diagram (P&ID) is a type of engineering diagram that uses symbols, text, and lines to represent the components and flow of an industrial process. Although used universally across industries such as manufacturing and oil & gas, P&IDs are usually trapped in image files with limited metadata, making their contents unsearchable and siloed from operational or enterprise systems. In order to extract the information contained in these diagrams, we propose a pipeline for automatically digitizing P&IDs. Our pipeline combines a series of computer vision techniques to detect symbols in a diagram, match symbols with associated text, and detect connections between symbols through lines. For the symbol detection task, we train a Convolutional Neural Network to classify certain common symbols with over 90% precision and recall. To detect connections between symbols, we use a graph search approach to traverse a diagram through its lines and discover interconnected symbols. By transforming unstructured diagrams into structured information, our pipeline enables applications such as diagram search, equipment-to-sensor mapping, and asset hierarchy creation. When integrated with operational and enterprise data, the extracted asset hierarchy serves as the foundation for a facility-wide digital twin, enabling advanced applications such as machine learning-based predictive maintenance.

1. Introduction

Engineering designs, such as Piping and Instrumentation Diagrams (P&IDs), are used throughout the entire lifecycle of a facility. Created during the engineering and design phase, they are used to communicate requirements throughout the construction and operational phases of an industrial facility. In many jurisdictions, primarily for safety considerations, there are legal requirements to keep these designs up to date with any changes. Additionally, process engineers rely primarily on P&IDs to understand an industrial process and as a tool to plan and implement process changes. These factors make P&IDs a valuable, up to date ground-truth data source for the configuration of an industrial facility. However, P&IDs are often archived as hundreds of CAD files per facility with limited or no metadata describing the components, connections between components, or connections between P&IDs.