The Hearsay-II speech-understanding system, developed during a five-year DARPA-sponsored research program, addresses the challenges of speech understanding by integrating symbolic reasoning with acoustic and linguistic knowledge. The system aims to reconstruct the speaker's intention from the acoustic signal, which involves interpreting speech through a series of transformations from intentions to semantic and syntactic structures. Ambiguity and uncertainty arise at each step, necessitating a structured approach to resolve these issues. Hearsay-II employs a problem-solving framework that reconstructs intentions from hypothetical interpretations at various levels of abstraction. It allocates processing resources to the most promising incremental actions, using a focus-of-control mechanism to identify the most valuable potential actions. The system's architecture includes components for generating and evaluating speech hypotheses and a mechanism to control the search for solutions. The paper discusses the characteristics of the speech problem, the types of uncertainty in that domain, and the structure of Hearsay-II. It also compares Hearsay-II with other speech-understanding systems and analyzes its performance. The system has been adapted for various applications, demonstrating its adaptability and effectiveness in handling complex problems involving diverse sources of knowledge.The Hearsay-II speech-understanding system, developed during a five-year DARPA-sponsored research program, addresses the challenges of speech understanding by integrating symbolic reasoning with acoustic and linguistic knowledge. The system aims to reconstruct the speaker's intention from the acoustic signal, which involves interpreting speech through a series of transformations from intentions to semantic and syntactic structures. Ambiguity and uncertainty arise at each step, necessitating a structured approach to resolve these issues. Hearsay-II employs a problem-solving framework that reconstructs intentions from hypothetical interpretations at various levels of abstraction. It allocates processing resources to the most promising incremental actions, using a focus-of-control mechanism to identify the most valuable potential actions. The system's architecture includes components for generating and evaluating speech hypotheses and a mechanism to control the search for solutions. The paper discusses the characteristics of the speech problem, the types of uncertainty in that domain, and the structure of Hearsay-II. It also compares Hearsay-II with other speech-understanding systems and analyzes its performance. The system has been adapted for various applications, demonstrating its adaptability and effectiveness in handling complex problems involving diverse sources of knowledge.