This paper introduces a curriculum-based reinforcement learning quantum architecture search (CRLQAS) algorithm to address the challenges of deploying variational quantum algorithms (VQAs) in noisy quantum environments. The algorithm is designed to efficiently explore the search space of quantum circuits, optimize gate parameters, and adapt to hardware noise. CRLQAS incorporates a 3D tensor-based encoding of quantum circuits, an episode halting mechanism to find shorter circuits, and a novel variant of simultaneous perturbation stochastic approximation (SPSA) for faster convergence. The algorithm is tested on quantum chemistry tasks, demonstrating superior performance compared to existing QAS methods in both noiseless and noisy environments. The CRLQAS algorithm uses a curriculum-based approach to guide the agent in learning optimal quantum circuits. It introduces illegal actions to prune the search space, random halting to encourage shorter circuits, and a tensor-based binary encoding of the circuit structure. The algorithm also employs a modified SPSA algorithm with adaptive momentum and variable sample budgets to improve convergence and robustness in the presence of noise. The paper evaluates the performance of CRLQAS on quantum chemistry tasks, including the ground-state energy estimation of molecules such as hydrogen (H₂), lithium hydride (LiH), and water (H₂O). The results show that CRLQAS achieves chemical accuracy with significantly lower energy errors compared to existing QAS methods. The algorithm is also shown to be efficient in simulating noisy quantum circuits using the Pauli-transfer matrix (PTM) formalism, which allows for faster computation and better performance under physical noise. The study highlights the importance of addressing hardware noise in quantum computing and demonstrates the effectiveness of CRLQAS in overcoming these challenges. The algorithm's ability to adapt to different noise levels and find optimal circuits makes it a promising approach for quantum architecture search in noisy environments. The results suggest that CRLQAS can be applied to a wide range of quantum computing tasks, including combinatorial optimization, quantum machine learning, and quantum reinforcement learning.This paper introduces a curriculum-based reinforcement learning quantum architecture search (CRLQAS) algorithm to address the challenges of deploying variational quantum algorithms (VQAs) in noisy quantum environments. The algorithm is designed to efficiently explore the search space of quantum circuits, optimize gate parameters, and adapt to hardware noise. CRLQAS incorporates a 3D tensor-based encoding of quantum circuits, an episode halting mechanism to find shorter circuits, and a novel variant of simultaneous perturbation stochastic approximation (SPSA) for faster convergence. The algorithm is tested on quantum chemistry tasks, demonstrating superior performance compared to existing QAS methods in both noiseless and noisy environments. The CRLQAS algorithm uses a curriculum-based approach to guide the agent in learning optimal quantum circuits. It introduces illegal actions to prune the search space, random halting to encourage shorter circuits, and a tensor-based binary encoding of the circuit structure. The algorithm also employs a modified SPSA algorithm with adaptive momentum and variable sample budgets to improve convergence and robustness in the presence of noise. The paper evaluates the performance of CRLQAS on quantum chemistry tasks, including the ground-state energy estimation of molecules such as hydrogen (H₂), lithium hydride (LiH), and water (H₂O). The results show that CRLQAS achieves chemical accuracy with significantly lower energy errors compared to existing QAS methods. The algorithm is also shown to be efficient in simulating noisy quantum circuits using the Pauli-transfer matrix (PTM) formalism, which allows for faster computation and better performance under physical noise. The study highlights the importance of addressing hardware noise in quantum computing and demonstrates the effectiveness of CRLQAS in overcoming these challenges. The algorithm's ability to adapt to different noise levels and find optimal circuits makes it a promising approach for quantum architecture search in noisy environments. The results suggest that CRLQAS can be applied to a wide range of quantum computing tasks, including combinatorial optimization, quantum machine learning, and quantum reinforcement learning.