Reinforcement Learning With Safe Exploration for Adaptive Plasma Cancer Treatment

Cold atmospheric plasma (CAP) jet is an ionized gas with a rich combination of reactive oxygen/nitrogen species, charged particles, and photons. By both in vitro and in vivo experiments, it has been demonstrated that CAP can be successfully utilized in cancer treatments. However, the therapeutic effects of CAP depend on various factors, such as plasma discharge voltage, gas composition, treatment duration, and type of cancer cells. This article presents an adaptive plasma system, where the CAP treatment conditions are adjusted online depending on the actual cancer cell response. In particular, we utilize safe $Q$ -learning to schedule CAP cancer treatment autonomously while prohibiting excessive treatments caused by large uncertainties. As it is capable of learning the dynamic characteristics of the particular cancer cell under treatments in situ, we can treat cancer successfully without the complete prior knowledge of cancer characteristics, where the uncertainties of the learned dynamics are carefully accounted. The efficacy of the proposed algorithm is illustrated by numerical examples with an empirical cancer dynamic model constructed from in vitro experiments.

I. Introduction

Cold atmospheric plasma (CAP) jet is generated by ionization that is initialized when noble gas, such as helium and argon, pass through an electric field. In particular, CAP or nonthermal plasma jet refers to the case when the ion temperature is close to the room temperature [1]. There have been increasing interests in CAP specifically for potential application in cancer treatments. It is studied that the rich environments of reactive species, charged particles, photons, and UV, provided by CAP jet trigger cell death pathway selectively for cancer cells while leaving healthy cells unharmed. This has been illustrated by both in vitro and in vivo under various conditions [2]. The CAP jet is capable of eliminating cancer cells in vitro and reducing the size of tumor in vivo [3], [4].