The Tables Have Turned: GPT-3 Distinguishing Passwords from Honeywords

In the field of information security, there has been a noteworthy trend toward leveraging machine learning models to develop and exploit security solutions. The emergence of Generative Pre-trained Transformer: version 3 (GPT-3), a pre-trained language model developed by OpenAI, has generated considerable excitement due to its unprecedented ability to generate different solutions. In the realm of timely detecting threats on a password-file, the generation of realistic yet fictitious passwords or honeywords has long been recognized as a crucial aspect of security solutions. However, meeting this requirement has proven to be a persistent challenge. In the face of this crisis, researchers have recently proposed employing GPT-3 as a means to surpass this barrier. This paper presents an analysis of how GPT-3 can potentially undermine the effectiveness of this security solution by accurately distinguishing genuine passwords from a set of honeywords it generates. The experiments conducted for this study reveal that GPT-3 can accurately guess a significant percentage of actual passwords, reaching as high as 53.45% with just three attempts. Though we emphasize the careful use of GPT-3 for generating honeywords, one of the primary findings in this study strongly indicates that GPT-3 can effectively be transformed into an attack mechanism, thus altering the dynamics of the present notion.