SQLPass: A Semantic Effective Fuzzing Method for DBMS

Fuzzing, as an effective method for software system defect testing and bug mining, utilizes random data generated by mutation to execute programs to trigger potential bugs in the tested program. However, due to the strict syntax and semantic checks in DBMSs, existing mutation-based testing methods are difficult to generate test cases with correct syntax and semantics. To ensure the syntax and semantic correctness of the test case generated by mutation, this paper proposes the concepts of weak semantic correlation nodes and semantic relationship tables. At the same time, a set of mutation operators for weak semantic correlation nodes in the syntax tree is designed, and each time the “optimal” mutation operator is selected to replace, delete, and insert the target node subtree in the test case syntax tree. In response to semantic errors during the mutation, this paper checks and corrects them through a pre-extracted semantic relationship table, further ensuring the correctness of the semantics of the test cases generated by the mutation. This paper utilizes this semantic effective fuzzing method for DBMSs to implement a new fuzzing framework for DBMSs, SQLPass. We evaluated SQLPass on four popular DBMSs: SQLite3, MySQL, MariaDB and PostgreSQL. In our experiment, SQLPass achieves 5.7%-94.2% higher semantic correctness than state-of-the-art tools, and explores 1.3%-52% more code coverage than other tools. Notably, it discovered four unknown bugs on SQLite3 and MariaDB and has submitted them to the vendor for confirmation.