December 1990 | Barton P. Miller, Lars Fredriksen and Bryan So
This study investigates the reliability of Unix utilities by systematically testing them with random input strings. The project involved four main steps: (1) creating programs to generate random characters and test interactive utilities; (2) using these programs to test a large number of utilities on random input strings to see if they crashed; (3) identifying the input strings that caused crashes; and (4) analyzing the causes of these crashes, including the "gets finger" bug. The study found that many Unix utilities were vulnerable to crashes due to improper handling of input, such as buffer overflows and pointer errors. These findings highlight the importance of thorough testing and the need for more rigorous testing procedures to ensure the reliability of system utilities.
The study tested 88 utility programs across seven versions of Unix, and more than 24% of these programs crashed under various input conditions. The results showed that many of the crashes were caused by common programming errors, such as incorrect pointer usage, failure to check return codes, and improper handling of input functions. The study also found that some of the crashes were caused by input that was carelessly typed, and that some programs were not designed to handle noisy input, such as that from a dial-up line.
The study developed tools, including the fuzz program for generating random input strings and ptyjig for testing interactive utilities, to automate the testing process. These tools were used to test a wide range of Unix utilities, including commonly used ones like vi, emacs, and csh. The results of the tests were analyzed to identify the common causes of crashes and to suggest ways to improve the reliability of Unix utilities.
The study also discussed the importance of formal verification in ensuring the correctness of programs, but noted that it is not yet practical for large systems. Instead, the study proposed a more practical approach to testing, which involved using random input to simulate a wide range of possible scenarios. This approach was found to be effective in discovering bugs that might not be easily found through traditional testing methods.
The study concluded that the reliability of Unix utilities is a critical issue that requires ongoing testing and improvement. The findings of the study provide valuable insights into the common errors that can occur in Unix utilities and highlight the importance of thorough testing and error handling in software development. The study also emphasized the need for a more comprehensive approach to testing, which includes not only testing individual utilities but also considering the interactions between different programs and the overall system.This study investigates the reliability of Unix utilities by systematically testing them with random input strings. The project involved four main steps: (1) creating programs to generate random characters and test interactive utilities; (2) using these programs to test a large number of utilities on random input strings to see if they crashed; (3) identifying the input strings that caused crashes; and (4) analyzing the causes of these crashes, including the "gets finger" bug. The study found that many Unix utilities were vulnerable to crashes due to improper handling of input, such as buffer overflows and pointer errors. These findings highlight the importance of thorough testing and the need for more rigorous testing procedures to ensure the reliability of system utilities.
The study tested 88 utility programs across seven versions of Unix, and more than 24% of these programs crashed under various input conditions. The results showed that many of the crashes were caused by common programming errors, such as incorrect pointer usage, failure to check return codes, and improper handling of input functions. The study also found that some of the crashes were caused by input that was carelessly typed, and that some programs were not designed to handle noisy input, such as that from a dial-up line.
The study developed tools, including the fuzz program for generating random input strings and ptyjig for testing interactive utilities, to automate the testing process. These tools were used to test a wide range of Unix utilities, including commonly used ones like vi, emacs, and csh. The results of the tests were analyzed to identify the common causes of crashes and to suggest ways to improve the reliability of Unix utilities.
The study also discussed the importance of formal verification in ensuring the correctness of programs, but noted that it is not yet practical for large systems. Instead, the study proposed a more practical approach to testing, which involved using random input to simulate a wide range of possible scenarios. This approach was found to be effective in discovering bugs that might not be easily found through traditional testing methods.
The study concluded that the reliability of Unix utilities is a critical issue that requires ongoing testing and improvement. The findings of the study provide valuable insights into the common errors that can occur in Unix utilities and highlight the importance of thorough testing and error handling in software development. The study also emphasized the need for a more comprehensive approach to testing, which includes not only testing individual utilities but also considering the interactions between different programs and the overall system.