Selecting Empirical Methods for Software Engineering Research

Selecting Empirical Methods for Software Engineering Research

2008 | Steve Easterbrook, Janice Singer, Margaret-Anne Storey, and Daniela Damian
Selecting empirical methods for software engineering research is challenging because the benefits and challenges of each method are not well understood. This chapter describes various empirical methods, examines their goals, and analyzes the types of questions they best address. It also reviews the theoretical foundations, practical considerations, and data collection aspects of these methods. Together, this information provides a basis for understanding and selecting appropriate methods for empirical software engineering research. Despite widespread interest in empirical software engineering, there is little guidance on which methods are suitable for which research problems. Many researchers choose inappropriate methods due to a lack of understanding of the underlying goals or limited knowledge of alternatives. This chapter discusses key questions to consider when selecting a method, from philosophical considerations to practical ones. It characterizes key empirical methods applicable to software engineering and explains their strengths and weaknesses. Software engineering is a multidisciplinary field that crosses social and technological boundaries. To understand how software engineers construct and maintain complex systems, we need to study not only tools and processes but also social and cognitive processes. This requires the study of human activities, including how individuals and teams develop software and coordinate efforts. Because of the importance of human activities in software development, many research methods are drawn from disciplines that study human behavior. These methods have known flaws and can only provide limited evidence. However, viable research strategies use multiple methods to address their weaknesses. This chapter identifies and compares five classes of research methods most relevant to software engineering: controlled experiments, case studies, survey research, ethnographies, and action research. It discusses the trade-offs involved in choosing between these methods but does not provide a recipe for building research strategies. The selection of methods depends on various local factors, including resources, access to subjects, and researcher skills. Two fictional researchers, Jane and Joe, are used to illustrate the steps involved in deciding which method or methods to use. Jane is a new PhD student interested in the effectiveness of a novel fisheye-view file navigator, while Joe is a researcher interested in how developers use UML diagrams during software design.Selecting empirical methods for software engineering research is challenging because the benefits and challenges of each method are not well understood. This chapter describes various empirical methods, examines their goals, and analyzes the types of questions they best address. It also reviews the theoretical foundations, practical considerations, and data collection aspects of these methods. Together, this information provides a basis for understanding and selecting appropriate methods for empirical software engineering research. Despite widespread interest in empirical software engineering, there is little guidance on which methods are suitable for which research problems. Many researchers choose inappropriate methods due to a lack of understanding of the underlying goals or limited knowledge of alternatives. This chapter discusses key questions to consider when selecting a method, from philosophical considerations to practical ones. It characterizes key empirical methods applicable to software engineering and explains their strengths and weaknesses. Software engineering is a multidisciplinary field that crosses social and technological boundaries. To understand how software engineers construct and maintain complex systems, we need to study not only tools and processes but also social and cognitive processes. This requires the study of human activities, including how individuals and teams develop software and coordinate efforts. Because of the importance of human activities in software development, many research methods are drawn from disciplines that study human behavior. These methods have known flaws and can only provide limited evidence. However, viable research strategies use multiple methods to address their weaknesses. This chapter identifies and compares five classes of research methods most relevant to software engineering: controlled experiments, case studies, survey research, ethnographies, and action research. It discusses the trade-offs involved in choosing between these methods but does not provide a recipe for building research strategies. The selection of methods depends on various local factors, including resources, access to subjects, and researcher skills. Two fictional researchers, Jane and Joe, are used to illustrate the steps involved in deciding which method or methods to use. Jane is a new PhD student interested in the effectiveness of a novel fisheye-view file navigator, while Joe is a researcher interested in how developers use UML diagrams during software design.
Reach us at info@study.space