Forged handwriting verification: a public domain dataset for training machine learning models

Handwriting verification is an important task in the legal framework where the paternity of handwritten wills or con-tracts needs to be ascertained. Due to the lack of automated tools, the court currently relies exclusively on handwriting experts, who often show a wide degree of uncertainty in their responses. Machine learning models, and in particular artificial Neural Networks (NNs) might be a valuable aid to experts by providing an objective and automated instrument for handwriting analysis, especially when expert witnesses are undecided. However, at the state of the art there is a scarcity of datasets which are suitable for training NNs for the handwriting verification task, preventing the development of models accurate enough to be introduced into forensic practice. In this paper, a dataset of 3320 genuine and 3320 forged handwritten samples from 166 subjects is presented, considering two scenarios: copy (imitation of handwriting maintaining the same text, 3320 samples) and spontaneous production (imitation of handwriting generating a new text, 3320 samples). We provide baseline results for deep siamese convolutional neural networks, that are deep learning models widely adopted in similar tasks. In the proposed dataset, such NNs were able to reach accuracies over 75% in identifying forged samples in the copy scenario and of almost 82% in the spontaneous production scenario. Finally, a sample of 550 humans were tested in the same classification task. The experiments show that NNs perform significantly better than humans in the spontaneous production scenario, which is more complex than the copy one. We publicly release our dataset to encourage the future development of advanced Deep Learning models for the forged handwriting verification task.

I. Introduction

Nowadays, more and more technological tools support human writing: thanks to laptops, smartphones, and tablets we can write documents, mail friends, and post thoughts on social networks. However, handwriting still plays an important role in society. There are some fundamental characteristics of human handwriting which make it unique, unambiguous, and in a one-to-one relationship with its author [3], [5], [24]. For this reason, the handwritten signature remains the main mean by which the authenticity of contracts, agreements, legacies, and testimonies between human beings is guaranteed. Technological advances in cybersecurity and digital forensic science, like the advent of electronic signatures, have not yet eliminated the practice of handcrafting legal documents, such as contracts and wills, with all the consequent limitations. Indeed, handwritten documents are frequently brought to court to verify the authenticity of the authorship and, then, to ascertain their legal validity, making handwriting verification an important task in the legal framework. For handwriting verification, courts currently rely exclusively on the opinion of handwriting expert witnesses, also known as Forensic Document Examiners (FDEs) [11], [34]. This opens up the problem of the reliability and the admissibility as scientific evidence according to Daubert criteria, of the expert witnesses' testimony about the authenticity of a written text [21], [32]. Sita, Found, and Rogers [27] compared the performance of FDEs and a group of non-experts in a signatures comparison task. The two groups were asked to decide about the authenticity of a pool of signatures, labeling each signature as genuine or forged. They could also decide not to express any opinion. Results highlighted that FDEs make fewer errors (3.4%), especially in recognizing the fake signatures. How-ever, they also showed a wide degree of uncertainty in their responses, providing more inconclusive answers than non-experts. In other words, the experts tend to stay on the safe side, preferring not to express any opinion instead of taking the risk of committing mistakes. On the contrary, naive subjects tend to be overly confident in their answers, thus committing more mistakes but providing fewer inconclusive answers [27]. Dror et al. [9] measured the consistency among a group of FDEs in a decision-making task that required to compare the unknown questioned signature with a sample of known genuine signatures. The study revealed a lack of consistency in conclusions among the FDEs, pointing out that there is low reliability in their decision-making process. Ultimately, even expert witnesses are not 100% accurate in judging the authenticity of handwriting, preventing courts to pronounce sentences “beyond any reasonable doubt”.