The paper discusses the importance of order in sequence-to-sequence (seq2seq) models when dealing with input and output data that are not naturally sequential. While seq2seq models have been successful in tasks involving sequences, they face challenges when handling unordered sets, such as sorting numbers or modeling the joint probability of random variables. The authors propose an extension of the seq2seq framework that can handle input and output sets more effectively by considering different orderings during training. They introduce a loss function that searches for the optimal order during training, improving performance on tasks like sorting and estimating joint probabilities. The paper also shows that the order of input and output data significantly affects model performance, even when no natural order is present. Experiments on benchmark tasks and artificial datasets demonstrate that the proposed methods outperform traditional seq2seq models in various scenarios. The key idea is that while the chain rule allows for flexible modeling of sequences, handling unordered data requires careful consideration of the order in which elements are processed. The authors also introduce a model that uses attention mechanisms to handle variable-length structures and demonstrate its effectiveness in tasks involving sets. The results show that the proposed approach can learn optimal orderings without prior knowledge, making it more versatile for a wide range of applications.The paper discusses the importance of order in sequence-to-sequence (seq2seq) models when dealing with input and output data that are not naturally sequential. While seq2seq models have been successful in tasks involving sequences, they face challenges when handling unordered sets, such as sorting numbers or modeling the joint probability of random variables. The authors propose an extension of the seq2seq framework that can handle input and output sets more effectively by considering different orderings during training. They introduce a loss function that searches for the optimal order during training, improving performance on tasks like sorting and estimating joint probabilities. The paper also shows that the order of input and output data significantly affects model performance, even when no natural order is present. Experiments on benchmark tasks and artificial datasets demonstrate that the proposed methods outperform traditional seq2seq models in various scenarios. The key idea is that while the chain rule allows for flexible modeling of sequences, handling unordered data requires careful consideration of the order in which elements are processed. The authors also introduce a model that uses attention mechanisms to handle variable-length structures and demonstrate its effectiveness in tasks involving sets. The results show that the proposed approach can learn optimal orderings without prior knowledge, making it more versatile for a wide range of applications.