This paper explores the computational power of neural networks, specifically recurrent first-order neural networks, also referred to as processor nets. The study focuses on networks composed of a finite number of synchronously evolving processors, where each processor updates its state using a "sigmoidal" scalar nonlinearity applied to a linear combination of previous states and an external input. The authors prove that such networks can simulate all Turing Machines, including multi-tape Turing Machines, using only first-order (linear) connections and rational weights. This simulation can be done in linear time, and a network with approximately 1,000 processors can compute a universal partial-recursive function. The results also extend to non-deterministic Turing Machines, showing that non-deterministic rational nets can simulate them in linear time. The paper discusses the implications of these results for undecidability and complexity issues in neural networks. It shows that determining whether a given neuron ever assumes the value "0" is effectively undecidable, while using linear activation functions makes the problem decidable. The study also highlights the computational equivalence between first-order and higher-order networks up to polynomial time. The authors describe a detailed construction of a processor net that can simulate a two-stack Turing Machine, using a combination of linear operations and sigmoidal functions. This construction is extended to multi-tape Turing Machines, demonstrating that the results can be generalized. The paper also addresses the issue of inputs and outputs, showing how to encode and decode information within the network. The study concludes with an analysis of the size of the network required to compute a recursively computable partial function, showing that a universal processor net can be constructed with approximately 1,000 processors. The paper also discusses the possibility of reducing the number of processors through more efficient constructions and the use of multi-tape Turing Machines. The results have significant implications for the understanding of the computational capabilities of neural networks and their potential applications in computing.This paper explores the computational power of neural networks, specifically recurrent first-order neural networks, also referred to as processor nets. The study focuses on networks composed of a finite number of synchronously evolving processors, where each processor updates its state using a "sigmoidal" scalar nonlinearity applied to a linear combination of previous states and an external input. The authors prove that such networks can simulate all Turing Machines, including multi-tape Turing Machines, using only first-order (linear) connections and rational weights. This simulation can be done in linear time, and a network with approximately 1,000 processors can compute a universal partial-recursive function. The results also extend to non-deterministic Turing Machines, showing that non-deterministic rational nets can simulate them in linear time. The paper discusses the implications of these results for undecidability and complexity issues in neural networks. It shows that determining whether a given neuron ever assumes the value "0" is effectively undecidable, while using linear activation functions makes the problem decidable. The study also highlights the computational equivalence between first-order and higher-order networks up to polynomial time. The authors describe a detailed construction of a processor net that can simulate a two-stack Turing Machine, using a combination of linear operations and sigmoidal functions. This construction is extended to multi-tape Turing Machines, demonstrating that the results can be generalized. The paper also addresses the issue of inputs and outputs, showing how to encode and decode information within the network. The study concludes with an analysis of the size of the network required to compute a recursively computable partial function, showing that a universal processor net can be constructed with approximately 1,000 processors. The paper also discusses the possibility of reducing the number of processors through more efficient constructions and the use of multi-tape Turing Machines. The results have significant implications for the understanding of the computational capabilities of neural networks and their potential applications in computing.