24 Sep 2017 | Jiwei Li1, Will Monroe1, Tianlin Shi1, Sébastien Jean2, Alan Ritter3 and Dan Jurafsky1
This paper proposes using adversarial training for open-domain dialogue generation, inspired by the Turing test. The system is trained to generate dialogue responses that are indistinguishable from human-generated ones. The task is framed as a reinforcement learning problem, where a generative model produces dialogue sequences, and a discriminator distinguishes between human and machine-generated dialogues. The discriminator's output serves as a reward for the generative model, encouraging it to generate responses that closely resemble human dialogues.
The paper introduces a model for adversarial evaluation, where success in fooling an adversary is used as a dialogue evaluation metric. Experimental results show that the adversarially-trained system generates higher-quality responses than previous baselines. The system uses adversarial training to improve dialogue generation, leading to more interactive, interesting, and non-repetitive responses compared to standard SEQ2SEQ models.
The adversarial training approach involves a generative model and a discriminative model. The generative model is trained to produce dialogue responses, while the discriminative model evaluates whether a response is human-generated or machine-generated. The system uses policy gradient methods to train the generative model, with the discriminator's output serving as a reward. The paper also introduces a method called REGS (Reward for Every Generation Step) to address the issue of using a single reward for all tokens in a generated sequence.
The paper discusses the challenges of adversarial evaluation, including the potential for a poor discriminative model to lead to misleading results. It proposes strategies to test the reliability of the evaluator, including manually designed test cases. The paper also compares the performance of different models, including standard SEQ2SEQ models and the proposed adversarial training approach.
The experimental results show that the adversarial training approach leads to significant improvements in dialogue generation, with higher adversarial success and better human evaluation scores. The paper concludes that adversarial training is a promising approach for dialogue generation, and further research is needed to explore its potential in other NLP tasks.This paper proposes using adversarial training for open-domain dialogue generation, inspired by the Turing test. The system is trained to generate dialogue responses that are indistinguishable from human-generated ones. The task is framed as a reinforcement learning problem, where a generative model produces dialogue sequences, and a discriminator distinguishes between human and machine-generated dialogues. The discriminator's output serves as a reward for the generative model, encouraging it to generate responses that closely resemble human dialogues.
The paper introduces a model for adversarial evaluation, where success in fooling an adversary is used as a dialogue evaluation metric. Experimental results show that the adversarially-trained system generates higher-quality responses than previous baselines. The system uses adversarial training to improve dialogue generation, leading to more interactive, interesting, and non-repetitive responses compared to standard SEQ2SEQ models.
The adversarial training approach involves a generative model and a discriminative model. The generative model is trained to produce dialogue responses, while the discriminative model evaluates whether a response is human-generated or machine-generated. The system uses policy gradient methods to train the generative model, with the discriminator's output serving as a reward. The paper also introduces a method called REGS (Reward for Every Generation Step) to address the issue of using a single reward for all tokens in a generated sequence.
The paper discusses the challenges of adversarial evaluation, including the potential for a poor discriminative model to lead to misleading results. It proposes strategies to test the reliability of the evaluator, including manually designed test cases. The paper also compares the performance of different models, including standard SEQ2SEQ models and the proposed adversarial training approach.
The experimental results show that the adversarial training approach leads to significant improvements in dialogue generation, with higher adversarial success and better human evaluation scores. The paper concludes that adversarial training is a promising approach for dialogue generation, and further research is needed to explore its potential in other NLP tasks.