Automated nuclei detection in serous effusion cytology with stacked sparse autoencoders

Serous effusion is frequently encountered specimen type in (cyto)pathological assessment. However, this assessment is time-consuming and leads to variability among pathologists. The cell nuclei is seen as the corner stone for diagnostic purposes in automatic analysis of cytopathological images. In this paper, a stacked sparse autoencoder (SSAE) is proposed for nuclei detection in serous effusion cytology. SSAE is an unsupervised deep learning method which learns high-level features from just pixel intensities alone in order to identify distinguishing features of nuclei. High-level features, obtained via the autoencoder, are then subsequently fed to a softmax classifier which categorizes each patch as nuclei or non-nuclei. With a detection rate of 98.3% based on images of serous effusion cytology, proposed method shows good performance.