This paper introduces a new decoding method called Permute-and-Flip (PF) decoder, which offers robustness properties similar to standard sampling decoding but is provably up to 2x better in its quality-robustness tradeoff than sampling and never worse than other decoders. The PF decoder is also designed with a cryptographic watermarking scheme, analogous to Aaronson's Gumbel watermark, that allows for arbitrarily low false positive rates and high recall when generated text has high entropy. The PF decoder significantly outperforms naive sampling in terms of perplexity while retaining the same robustness and detectability, making it a promising new approach for LLM decoding. The paper also presents a watermarking scheme for PF decoding that is computationally indistinguishable from the non-watermarked version and has precisely controlled false positive rates. Experiments show that the PF watermark achieves the best balance of detection accuracy and perplexity compared to other watermarking methods. The PF decoder is also robust against small perturbations to the logits, making it suitable for applications where robustness is important. The paper concludes that the PF decoder is a promising new approach for LLM decoding, with the added benefit of watermarking capabilities.This paper introduces a new decoding method called Permute-and-Flip (PF) decoder, which offers robustness properties similar to standard sampling decoding but is provably up to 2x better in its quality-robustness tradeoff than sampling and never worse than other decoders. The PF decoder is also designed with a cryptographic watermarking scheme, analogous to Aaronson's Gumbel watermark, that allows for arbitrarily low false positive rates and high recall when generated text has high entropy. The PF decoder significantly outperforms naive sampling in terms of perplexity while retaining the same robustness and detectability, making it a promising new approach for LLM decoding. The paper also presents a watermarking scheme for PF decoding that is computationally indistinguishable from the non-watermarked version and has precisely controlled false positive rates. Experiments show that the PF watermark achieves the best balance of detection accuracy and perplexity compared to other watermarking methods. The PF decoder is also robust against small perturbations to the logits, making it suitable for applications where robustness is important. The paper concludes that the PF decoder is a promising new approach for LLM decoding, with the added benefit of watermarking capabilities.