Using NLG for speech synthesis of mathematical sentences

Using NLG for speech synthesis of mathematical sentences

Tokyo, Japan, 28 Oct - 1 Nov, 2019 | Alessandro Mazzei, Michele Monticone, Cristian Bernareggi
This paper explores the design and evaluation of a Natural Language Generation (NLG) system for converting mathematical expressions into natural language sentences. The primary goal is to address the limitations of using LaTeX for accessibility, which requires knowledge of LaTeX, is slow, and is prone to errors. The authors propose a system that generates mathematical sentences in Italian, focusing on the linguistic status of mathematical expressions, the use of standard NLG architecture, and the simplification of speech synthesis for mathematical content. The paper outlines the following key components: 1. **From LaTeX to CMML**: The first step involves converting LaTeX expressions into Content MathML (CMML) using an external tool like LatexML, followed by post-processing to enhance the CMML. 2. **Syntactic Structure of Mathematical Expressions**: The authors analyze the syntactic structures of mathematical operators and define eight categories for representing complex mathematical expressions. 3. **Building Mathematical Sentences with NLG**: The NLG process includes document planning, sentence planning, and realization. The sentence planner uses a recursive algorithm to generate sentence plans based on the CMML structure, considering precedence and ambiguity issues. 4. **Evaluation**: A human-based evaluation was conducted with four visually impaired participants, using a web-based questionnaire to assess the comprehensibility of the generated sentences. The evaluation metrics included Exact Match and SPICE, showing that the system performs well on simple expressions but needs improvement for more complex ones. The paper concludes with future directions, including expanding the lexicon to support English, integrating acoustic signals for parenthetical representations, and integrating the system into a dialogue architecture for better user interaction.This paper explores the design and evaluation of a Natural Language Generation (NLG) system for converting mathematical expressions into natural language sentences. The primary goal is to address the limitations of using LaTeX for accessibility, which requires knowledge of LaTeX, is slow, and is prone to errors. The authors propose a system that generates mathematical sentences in Italian, focusing on the linguistic status of mathematical expressions, the use of standard NLG architecture, and the simplification of speech synthesis for mathematical content. The paper outlines the following key components: 1. **From LaTeX to CMML**: The first step involves converting LaTeX expressions into Content MathML (CMML) using an external tool like LatexML, followed by post-processing to enhance the CMML. 2. **Syntactic Structure of Mathematical Expressions**: The authors analyze the syntactic structures of mathematical operators and define eight categories for representing complex mathematical expressions. 3. **Building Mathematical Sentences with NLG**: The NLG process includes document planning, sentence planning, and realization. The sentence planner uses a recursive algorithm to generate sentence plans based on the CMML structure, considering precedence and ambiguity issues. 4. **Evaluation**: A human-based evaluation was conducted with four visually impaired participants, using a web-based questionnaire to assess the comprehensibility of the generated sentences. The evaluation metrics included Exact Match and SPICE, showing that the system performs well on simple expressions but needs improvement for more complex ones. The paper concludes with future directions, including expanding the lexicon to support English, integrating acoustic signals for parenthetical representations, and integrating the system into a dialogue architecture for better user interaction.
Reach us at info@study.space
Understanding Using NLG for speech synthesis of mathematical sentences