Classifying Cancer Stage with Open-Source Clinical Large Language Models

Cancer stage classification is important for making treatment and care management plans for oncology patients. Information on staging is often included in unstructured form in clinical, pathology, radiology and other free-text reports in the electronic health record system, requiring extensive work to parse and obtain. To facilitate the extraction of this in-formation, previous NLP approaches rely on labeled training datasets, which are labor-intensive to prepare. In this study, we demonstrate that without any labeled training data, open-source clinical large language models (LLMs) can extract pathologic tumor-node-metastasis (pTNM) staging information from real-world pathology reports. Our experiments compare LLMs and a BERT-based model fine-tuned using the labeled data. Our findings suggest that while LLMs still exhibit subpar performance in Tumor (T) classification, with the appropriate adoption of prompting strategies, they can achieve comparable performance on Metastasis (M) classification and improved performance on Node (N) classification.

I. Introduction

Although deaths due to cancer have continued to drop in the United States (U.S.), an estimated 2 million people were diagnosed with cancer in 2023 [1]. Cancer was one of the leading causes of death in the U.S. in 2021, second only to heart disease, and provisional mortality statistics indicate that this remained unchanged in 2022 and 2023 [2]. Diagnosing, treating, and monitoring cancer is an interdisciplinary effort that involves multiple health specialties, including medical and surgical oncologists, pathologists, radiologists, interventional radiologists, pharmacists, and nurses among others. All these providers interface with the patient at different times in their medical journey, thereby creating vast amounts of clinical data containing rich clinical insights. Knowledge of the patient's cancer stage is a critical piece of diagnostic and prognostic information for guiding treatment planning. An important type of staging data available from pathology reports is the Tumor-Node-Metastasis (TNM) stage, and specifically the pathologic TNM (pTNM) stage (determined after surgery, when the tumor has been excised and tissue samples obtained for analysis [3]). TNM is a staging system that allows for a standardized format for presenting information about different cancers. It includes information on the size and extent of the main tumor (T), how much it has spread to the lymph nodes (N), and whether it has spread further to distant sites in the body (M) [4]. These categories can also be further subdivided to provide additional information.