Audio Based Action Recognition for Monitoring Elderly Dementia Patients

Dementia is a disease that causes a decrease in memory and thinking. This condition has an impact on their lifestyle, social skills, and their daily activities. Generally, people with dementia live with care without a family companion. Treatment for dementia patients can be in the form of therapy for activities of daily living. Due to the limitations of the medical side in supervision, one way to find out the activities carried out by patients can be done by using voice. The system that will be used is Audio Based Action Recognition which can monitor the activities of dementia patients through voice data. The voice data is processed by sound signal processing using a deep learning Convolutional neural network (CNN) method to get the output about activities carried out by dementia patients. The system will record the voice of the dementia patient every 5 seconds. Then do feature extraction from audio data to get spectrogram data. Spectrogram data will be labeled according to the action wanted to identify. Then the system performs training and action classification using CNN. Action data of dementia patients recognized will be sent to the server to be used as a habit analysis of dementia patients. In testing, the system recognized the action with the pre-trained model for ResNet101 getting 69% accuracy, DenseNet 87%, InceptionV3 89%, MobileNet 95%, and the author's design model got 98% accuracy in a noise level of 30db.