Towards Comparative Evaluation of DDoS Defences

DDoS defence evaluation provides a way to capture the usefulness of defensive solutions to one of the most notorious Internet attacks of our computing generation. An alternative approach to evaluation offers a valuable mechanism by which different DDoS defences can be commensurably and objectively compared. Such a development would not only enable individual organizations to make better informed decisions on which defences to implement but could also aid collaborations to realize global solutions; and reveal insights into aspects requiring further investigation. We present CED3 (pronounced “Seed”), a DDoS defence evaluation framework designed to facilitate the commensurable comparison between DDoS defences in a way that captures their strengths and weaknesses. Firstly, CED3 introduces the notion of true effectiveness, which addresses the problem, identified in the literature, of previously validated defences subsequently being shown to be ineffective when evaluated under different attack conditions. CED3 leverages a structured theoretical analysis process to drive empirical data acquisition in order to enhance consistency, transparency and, ultimately, longevity of evaluation conclusions. Lastly, CED3 introduces the concept of defence maps, which applies the idea of true effectiveness to “scopes” in order to communicate the strengths and weaknesses of defences in a way that allows them to be visually compared. We demonstrate the CED3 framework by applying it to comparatively evaluate three DDoS defences. Using results obtained from extensive simulations in NS-3, we show how the strengths and weaknesses of different defences can be visually compared. We conclude by discussing the merits and limitations of CED3.