This paper explores the importance of order in sequence-to-sequence (seq2seq) models, particularly when dealing with input and output sets that are not naturally ordered. The authors demonstrate that the order in which data is presented to the model can significantly impact its performance. They propose an extension to the seq2seq framework that handles input sets in a principled way, using attention mechanisms to integrate information from variable-length structures. Additionally, they introduce a loss function that searches over possible orders during training to deal with the lack of structure in output sets. The paper includes empirical evidence from various tasks, such as language modeling, parsing, sorting numbers, and estimating the joint probability of unknown graphical models, to support their claims. The authors also discuss the challenges and limitations of current approaches and propose solutions to address them.This paper explores the importance of order in sequence-to-sequence (seq2seq) models, particularly when dealing with input and output sets that are not naturally ordered. The authors demonstrate that the order in which data is presented to the model can significantly impact its performance. They propose an extension to the seq2seq framework that handles input sets in a principled way, using attention mechanisms to integrate information from variable-length structures. Additionally, they introduce a loss function that searches over possible orders during training to deal with the lack of structure in output sets. The paper includes empirical evidence from various tasks, such as language modeling, parsing, sorting numbers, and estimating the joint probability of unknown graphical models, to support their claims. The authors also discuss the challenges and limitations of current approaches and propose solutions to address them.