CONFPILOT: A Pilot for Faster Configuration by Learning from Device Manuals

The command line interface (CLI) is widely used to configure and manage network devices. However, as heterogeneous devices are introduced into the network, the CLI-based method is becoming time-consuming and inefficient because much effort is required to learn proprietary configuration languages of different vendors or consult online documents. In this work, we present CONFPILOT, an assistant system that can accelerate configuration by automatically converting natural language intents into commands. Our solution is based on a retrieval-augmented generation framework that features a unified parser that parses device manuals into a searchable configuration library, a vendor-agnostic retriever that finds the most relevant
$k$
syntaxes through a two-stage coarse-to-fine process, and a reliable generator that predicts syntactically correct commands via a pointer-generator network and syntax-guided decoding. In a nutshell, CONFPILOT frees engineers from most time-consuming efforts by learning directly from device manuals to generate configuration commands. Our evaluation and user study show, CONFPILOT can speed up the configuration process by 60x compared to manual lookup while maintaining acceptable exact match accuracy. Furthermore, CONFPILOT can quickly adapt to new vendors and devices with little human effort and time cost.