Performance of ChatGPT on CMRP: Potential for Assisting Maintenance and Reliability Professionals Using Large Language Models

Amid the growth of Industry 4.0 and digital transformation, Smart Maintenance and AI-assisted maintenance solutions are gaining significant traction. However, most studies focus on the application of sophisticated AI algorithms in maintenance optimization while overlooking how AI can directly assist maintenance and reliability (M&R) professionals in a more comprehensible manner. This study fills this gap by investigating the potential of language models, specifically the GPT-3.5 and GPT-4 models developed by OpenAI, in aiding M&R practitioners. We used the Certified Maintenance & Reliability Professional (CMRP) exam, accredited by the American National Standards Institute (ANSI) to assess aptitude within 5 Pillars of the Society for Maintenance & Reliability Professionals Body of Knowledge, as a testing ground to evaluate and compare the performance and reasoning capabilities of the two AI models to that of human experts. Our findings suggest that while GPT-4 outperformed GPT-3.5 in providing self-consistent explanations and overall scoring, both models exhibited potential weaknesses, particularly in arithmetic operations and handling domain-specific terminologies. Despite these challenges, our results demonstrate the potential of these AI models as tools to support various M&R tasks, paving the way for more practical and comprehensive AI assistance in smart maintenance.