Know What You Don’t Know: Unanswerable Questions for SQuAD

Know What You Don’t Know: Unanswerable Questions for SQuAD

11 Jun 2018 | Pranav Rajpurkar*, Robin Jia*, Percy Liang
The paper introduces SQuAD 2.0, an updated version of the Stanford Question Answering Dataset (SQuAD), which aims to address the limitations of existing datasets by focusing on unanswerable questions. SQuAD 2.0 combines existing SQuAD data with over 50,000 new unanswerable questions generated by crowdworkers to ensure relevance and the presence of plausible answers. These questions are designed to look similar to answerable ones but are impossible to answer based on the context. The dataset is challenging for current models, with a state-of-the-art model achieving only 66.3% F1 score, compared to human accuracy of 89.5% F1. The paper also compares SQuAD 2.0 with automatically generated unanswerable questions, finding that SQuAD 2.0 is more difficult. The authors conclude that SQuAD 2.0 will encourage the development of models that can better understand when a question cannot be answered, leading to deeper language understanding.The paper introduces SQuAD 2.0, an updated version of the Stanford Question Answering Dataset (SQuAD), which aims to address the limitations of existing datasets by focusing on unanswerable questions. SQuAD 2.0 combines existing SQuAD data with over 50,000 new unanswerable questions generated by crowdworkers to ensure relevance and the presence of plausible answers. These questions are designed to look similar to answerable ones but are impossible to answer based on the context. The dataset is challenging for current models, with a state-of-the-art model achieving only 66.3% F1 score, compared to human accuracy of 89.5% F1. The paper also compares SQuAD 2.0 with automatically generated unanswerable questions, finding that SQuAD 2.0 is more difficult. The authors conclude that SQuAD 2.0 will encourage the development of models that can better understand when a question cannot be answered, leading to deeper language understanding.
Reach us at info@study.space