The paper introduces the Pointer Sentinel Mixture Model, a novel architecture for neural sequence models that combines the strengths of standard softmax classifiers and pointer networks. This model is designed to improve language modeling performance by effectively handling rare or out-of-vocabulary (OOV) words and long-term dependencies. The Pointer Sentinel-LSTM model achieves state-of-the-art perplexity on the Penn Treebank dataset while using fewer parameters than traditional softmax LSTM models. To evaluate the model's ability to handle longer contexts and more realistic vocabularies, the authors also introduce the WikiText corpus, a new benchmark dataset for language modeling. The Pointer Sentinel Mixture Model uses a mixture gate to decide when to use the softmax vocabulary or the pointer component, allowing it to leverage the strengths of both. The model's effectiveness is demonstrated through experiments on the Penn Treebank and WikiText-2 datasets, showing significant improvements over existing models, particularly in handling rare words. The paper also includes a qualitative analysis of the Pointer Sentinel model's behavior, highlighting its ability to use the pointer component for both rare and frequent words.The paper introduces the Pointer Sentinel Mixture Model, a novel architecture for neural sequence models that combines the strengths of standard softmax classifiers and pointer networks. This model is designed to improve language modeling performance by effectively handling rare or out-of-vocabulary (OOV) words and long-term dependencies. The Pointer Sentinel-LSTM model achieves state-of-the-art perplexity on the Penn Treebank dataset while using fewer parameters than traditional softmax LSTM models. To evaluate the model's ability to handle longer contexts and more realistic vocabularies, the authors also introduce the WikiText corpus, a new benchmark dataset for language modeling. The Pointer Sentinel Mixture Model uses a mixture gate to decide when to use the softmax vocabulary or the pointer component, allowing it to leverage the strengths of both. The model's effectiveness is demonstrated through experiments on the Penn Treebank and WikiText-2 datasets, showing significant improvements over existing models, particularly in handling rare words. The paper also includes a qualitative analysis of the Pointer Sentinel model's behavior, highlighting its ability to use the pointer component for both rare and frequent words.