Understanding and Mitigating Privacy Leaks from Third-Party Smart Speaker Apps

Smart speaker systems provide a platform for third-party developers to develop their own apps. This paper looks at ways a malicious third-party developer can create a valid smart speaker app to eavesdrop on users. We find that using the provided APIs together with machine learning techniques, we are able to overhear 50% of what the user says, capture at most 70% of triggered words from advertisements, and determine the advertisement category with overall accuracy up to 81.5%. To mitigate this threat, we propose a strategy for users to limit the success of this adversary. We run a series of experiments to look at the impact of a number of factors at the home on what the 3
^rd
party app can hear. We find that word capture appears to peak when the speaker is placed 3m or less from the audio source, and audio is playing at an approximate volume of 70dB or more. The inclusion of background noise actually improves data collection performance by helping to keep the session alive longer, especially when the user is pausing or not speaking.