This paper explores the behavior of transformer layers through the analogy of painters in an assembly line. The study investigates how different layers of a pretrained transformer model function, particularly focusing on whether they share a common representation space, whether all layers are necessary, and how layer order and execution affect performance. The research uses a series of experiments on frozen transformer models, including Llama2 and BERT, to test various hypotheses about the layers' behavior. The findings suggest that middle layers of a transformer share a common representation space, but the "outer" layers (first and last few) have a distinct representation. Middle layers are not redundant, as replacing them with the center layer leads to significant performance degradation. The order of layers matters, with some tasks, such as mathematical and reasoning tasks, being more sensitive to layer order than semantic tasks like Winogrande or HellaSwag. However, the model remains robust to changes in layer order, with both random and reversed layer orders showing graceful degradation. The study also tests whether layers can be executed in parallel, with results showing that this is possible for most tasks, except for math-heavy benchmarks. Looping parallelized layers improves performance, with the optimal number of iterations proportional to the number of parallelized layers. Among the variants tested, randomizing layer order and looped parallel execution cause the least damage to performance. The results suggest that transformers can be modified in various ways without catastrophic failure, and that some modifications, such as parallel execution or random layer order, can improve efficiency while maintaining performance. The study also highlights the importance of the residual connections in enabling shared representation spaces among layers. Overall, the findings provide insights into the structure and behavior of transformer layers, with implications for model optimization and architectural improvements.This paper explores the behavior of transformer layers through the analogy of painters in an assembly line. The study investigates how different layers of a pretrained transformer model function, particularly focusing on whether they share a common representation space, whether all layers are necessary, and how layer order and execution affect performance. The research uses a series of experiments on frozen transformer models, including Llama2 and BERT, to test various hypotheses about the layers' behavior. The findings suggest that middle layers of a transformer share a common representation space, but the "outer" layers (first and last few) have a distinct representation. Middle layers are not redundant, as replacing them with the center layer leads to significant performance degradation. The order of layers matters, with some tasks, such as mathematical and reasoning tasks, being more sensitive to layer order than semantic tasks like Winogrande or HellaSwag. However, the model remains robust to changes in layer order, with both random and reversed layer orders showing graceful degradation. The study also tests whether layers can be executed in parallel, with results showing that this is possible for most tasks, except for math-heavy benchmarks. Looping parallelized layers improves performance, with the optimal number of iterations proportional to the number of parallelized layers. Among the variants tested, randomizing layer order and looped parallel execution cause the least damage to performance. The results suggest that transformers can be modified in various ways without catastrophic failure, and that some modifications, such as parallel execution or random layer order, can improve efficiency while maintaining performance. The study also highlights the importance of the residual connections in enabling shared representation spaces among layers. Overall, the findings provide insights into the structure and behavior of transformer layers, with implications for model optimization and architectural improvements.