Evaluating the Impact of Flow Length on the Performance of In-Switch Inference Solutions

As modern networks evolve into complex systems to support next-generation applications with strict latency requirements, in-switch machine learning (ML) has emerged as a candidate technology for minimizing ML inference latency. Multiple solutions, mostly based on Decision Tree (DT) and Random Forest (RF) models, have been proposed in that regard for inference at packet level (PL) or flow level (FL) or simultaneously at PL and FL. Such heterogeneity in the inference target leads to the use of varying performance metrics for evaluating the solutions, rendering a fair comparison between them difficult. In this paper, we perform a comprehensive evaluation of 5 leading solutions for DT/RF-based in-switch inference. We replicate and deploy the solutions into a real-world testbed with Intel Tofino switches, and run experiments with measurement data from 4 datasets. We then evaluate their performance using (i) the original metric used in the solution's paper, and (ii) a novel FL metric which evaluates every solution at FL. This FL metric enables us to delve into an extensive analysis of how the solutions perform on flows of different lengths in diverse use cases. Results show that while some solutions perform similarly across use cases and flow sizes, others show inconsistent behaviours that we discuss.