October 31–November 4, 2018 | Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Céspedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, Ray Kurzweil
The paper introduces the Universal Sentence Encoder (USE), a TensorFlow Hub model for sentence embedding that demonstrates strong transfer performance across various natural language processing (NLP) tasks. The encoder is available in two variants: the Transformer and the Deep Averaging Network (DAN). The Transformer model achieves high accuracy but requires more computational resources, while the DAN model is more efficient but with slightly lower accuracy. The paper reports the trade-offs between model complexity and resource consumption, showing that sentence-level transfer learning outperforms word-level transfer learning and models without transfer learning. The models are trained using unsupervised data from web sources and supervised data from the Stanford Natural Language Inference (SNLI) corpus. The evaluation covers multiple tasks, including sentiment analysis, subjectivity detection, and semantic textual similarity, demonstrating the effectiveness of the encoder with minimal training data. The paper also assesses model bias using word embedding association tests (WEAT) and finds that the encoders reproduce human-like biases in some cases. Overall, the USE models provide a flexible and efficient solution for sentence-level embedding tasks in NLP.The paper introduces the Universal Sentence Encoder (USE), a TensorFlow Hub model for sentence embedding that demonstrates strong transfer performance across various natural language processing (NLP) tasks. The encoder is available in two variants: the Transformer and the Deep Averaging Network (DAN). The Transformer model achieves high accuracy but requires more computational resources, while the DAN model is more efficient but with slightly lower accuracy. The paper reports the trade-offs between model complexity and resource consumption, showing that sentence-level transfer learning outperforms word-level transfer learning and models without transfer learning. The models are trained using unsupervised data from web sources and supervised data from the Stanford Natural Language Inference (SNLI) corpus. The evaluation covers multiple tasks, including sentiment analysis, subjectivity detection, and semantic textual similarity, demonstrating the effectiveness of the encoder with minimal training data. The paper also assesses model bias using word embedding association tests (WEAT) and finds that the encoders reproduce human-like biases in some cases. Overall, the USE models provide a flexible and efficient solution for sentence-level embedding tasks in NLP.