2011 | Xuejun Yang, Yang Chen, Eric Eide, John Regehr
This paper presents Csmith, a randomized test-case generator for finding bugs in C compilers. Csmith generates C programs that avoid undefined and unspecified behaviors in the C99 standard, ensuring they have a single interpretation. By using differential testing, Csmith compares the outputs of multiple compilers to detect discrepancies. Over three years, Csmith found more than 325 previously unknown bugs in mainstream C compilers, including GCC, LLVM, and commercial tools. Every compiler tested crashed or silently miscompiled valid input. The bugs found were significant, with many being classified as high-priority. Csmith's test cases are expressive, covering a wide range of C features, and are designed to explore atypical combinations of language constructs. Csmith's safety mechanisms ensure that generated programs avoid undefined and unspecified behaviors, using static analysis and runtime checks. The paper also discusses the results of experiments comparing Csmith's bug-finding power to other tools and the distribution of bugs across compiler stages. The findings highlight the importance of testing in improving compiler quality and the need for more comprehensive testing methods.This paper presents Csmith, a randomized test-case generator for finding bugs in C compilers. Csmith generates C programs that avoid undefined and unspecified behaviors in the C99 standard, ensuring they have a single interpretation. By using differential testing, Csmith compares the outputs of multiple compilers to detect discrepancies. Over three years, Csmith found more than 325 previously unknown bugs in mainstream C compilers, including GCC, LLVM, and commercial tools. Every compiler tested crashed or silently miscompiled valid input. The bugs found were significant, with many being classified as high-priority. Csmith's test cases are expressive, covering a wide range of C features, and are designed to explore atypical combinations of language constructs. Csmith's safety mechanisms ensure that generated programs avoid undefined and unspecified behaviors, using static analysis and runtime checks. The paper also discusses the results of experiments comparing Csmith's bug-finding power to other tools and the distribution of bugs across compiler stages. The findings highlight the importance of testing in improving compiler quality and the need for more comprehensive testing methods.