Generating Authorship Embeddings with Transformers

Authorship attribution and profiling tools provide useful instruments with wide areas of application, such as disinformation spreaders detection. Models developed until now to fulfil these tasks usually rely on manually crafted features or on a training process restricted and limited by the number of authors involved. Besides, current methods have limited capacity to generate a broad representation of the author, without considering a great variety of features that can be extracted from their texts. In this paper, we propose a contrastive training method to generate representative embeddings of the authorship of a text and able to generalize to unseen authors. The core of this method is the Transformer architecture, which is known to generate very powerful semantically-aware text representations. Using a pretrained RoBERTa-large model, we evaluate our method on the environment of the standardized Gutenberg corpus, detecting the authorship of literary works. Representations generated by our proposal can be used to extract representative authorship embeddings and to visualize meaningful relationships between authors, genres and books. Furthermore, the embedding method achieves zero-shot 79% accuracy and 94% top-5 accuracy when tasked to distinguish a text piece from a set of 100 authored texts. Our code is readily available on GitHub
^1
1
https://github.com/jahuerta92/authorship-embedding