19 Jan 2017 | Tim Salimans, Andrej Karpathy, Xi Chen, Diederik P. Kingma
PixelCNN++ is an advanced version of the PixelCNN generative model, which introduces several modifications to improve its performance and simplify its structure. The key modifications include:
1. **Discretized Logistic Mixture Likelihood**: Instead of using a 256-way softmax, the model uses a discretized logistic mixture likelihood, which speeds up training and simplifies the model structure.
2. **Conditioning on Whole Pixels**: The model now conditions on whole pixels rather than individual sub-pixels (R/G/B), reducing complexity.
3. **Downsampling for Multi-Resolution Processing**: Downsampling is used to efficiently capture structure at multiple resolutions, similar to dilated convolutions.
4. **Additional Short-Cut Connections**: These connections help recover information lost during downsampling and improve optimization.
5. **Regularization with Dropout**: Dropout is applied to prevent overfitting and improve model quality.
The authors evaluate these modifications on the CIFAR-10 dataset, achieving state-of-the-art log-likelihood results and generating high-quality images. The modified model is available at <https://github.com/openai/pixel-cnn>.PixelCNN++ is an advanced version of the PixelCNN generative model, which introduces several modifications to improve its performance and simplify its structure. The key modifications include:
1. **Discretized Logistic Mixture Likelihood**: Instead of using a 256-way softmax, the model uses a discretized logistic mixture likelihood, which speeds up training and simplifies the model structure.
2. **Conditioning on Whole Pixels**: The model now conditions on whole pixels rather than individual sub-pixels (R/G/B), reducing complexity.
3. **Downsampling for Multi-Resolution Processing**: Downsampling is used to efficiently capture structure at multiple resolutions, similar to dilated convolutions.
4. **Additional Short-Cut Connections**: These connections help recover information lost during downsampling and improve optimization.
5. **Regularization with Dropout**: Dropout is applied to prevent overfitting and improve model quality.
The authors evaluate these modifications on the CIFAR-10 dataset, achieving state-of-the-art log-likelihood results and generating high-quality images. The modified model is available at <https://github.com/openai/pixel-cnn>.