This paper presents a natural language generation (NLG) system for producing mathematical sentences, which are natural language sentences expressing the semantics of mathematical expressions. The system aims to improve accessibility for visually impaired individuals who rely on speech synthesis to understand mathematical content. The system converts mathematical expressions from LaTeX into a natural language sentence using a two-step process: first, converting LaTeX to Content MathML (CMML), and then generating a spoken version of the mathematical sentence using an NLG module.
The system addresses the limitations of using LaTeX for mathematical expressions, such as its verbose nature, typographical focus, and difficulty in conveying semantics. The system uses a linguistic-based NLG architecture, which allows for more natural and accurate speech synthesis of mathematical expressions. The system was evaluated by four blind users for its ability to produce understandable mathematical sentences in Italian.
The system's architecture includes a preprocessing step to enhance CMML generated from LaTeX, followed by a sentence planner that generates a syntactic structure for the mathematical expression. The system then uses a realizer to generate the spoken version of the sentence. The system was tested with different strategies for handling operator precedence, including the use of parentheses and pauses.
The evaluation showed that the system's performance varied depending on the user and the complexity of the mathematical expression. The results indicated that formant-based synthesizers performed better than neural network-based ones, and that simple expressions were understood well, while more complex ones required improvement. The study highlights the importance of considering the unique characteristics of mathematical language in speech synthesis and the need for further research to improve accessibility for visually impaired individuals.This paper presents a natural language generation (NLG) system for producing mathematical sentences, which are natural language sentences expressing the semantics of mathematical expressions. The system aims to improve accessibility for visually impaired individuals who rely on speech synthesis to understand mathematical content. The system converts mathematical expressions from LaTeX into a natural language sentence using a two-step process: first, converting LaTeX to Content MathML (CMML), and then generating a spoken version of the mathematical sentence using an NLG module.
The system addresses the limitations of using LaTeX for mathematical expressions, such as its verbose nature, typographical focus, and difficulty in conveying semantics. The system uses a linguistic-based NLG architecture, which allows for more natural and accurate speech synthesis of mathematical expressions. The system was evaluated by four blind users for its ability to produce understandable mathematical sentences in Italian.
The system's architecture includes a preprocessing step to enhance CMML generated from LaTeX, followed by a sentence planner that generates a syntactic structure for the mathematical expression. The system then uses a realizer to generate the spoken version of the sentence. The system was tested with different strategies for handling operator precedence, including the use of parentheses and pauses.
The evaluation showed that the system's performance varied depending on the user and the complexity of the mathematical expression. The results indicated that formant-based synthesizers performed better than neural network-based ones, and that simple expressions were understood well, while more complex ones required improvement. The study highlights the importance of considering the unique characteristics of mathematical language in speech synthesis and the need for further research to improve accessibility for visually impaired individuals.