CARMEN: A Method for Automatic Evaluation of Poems

Automatic poem generation has been a challenging topic in Natural Language Processing research. However, their evaluation is still largely based on methods that involve evaluation by human judges, comparing the results with poems written by humans, or using metrics that were not designed for this purpose. In order to fill this gap, this work proposes a specific metric for automatic evaluation of poems, capable of quantitatively evaluating morphological characteristics of the text, such as rhyme and meter, of different types within this literary genre. The tests carried out demonstrate that the metric developed presents an advance for the evaluation of this type of text.