This paper introduces a method to tune a sequence-based generative model for molecular *de novo* design using deep reinforcement learning. The model is trained to generate molecules with specified desirable properties, such as avoiding certain elements (e.g., sulfur) and generating analogues to query structures. The method uses an augmented episodic likelihood, which combines prior likelihood and a user-defined scoring function, to fine-tune the model. The model is tested on various tasks, including generating sulfur-free molecules, creating analogues to the drug Celecoxib, and generating compounds predicted to be active against the dopamine receptor type 2 (DRD2). The results show that the model can generate a high fraction of valid and predicted active molecules, even when the training set does not include the target compounds. The study demonstrates the potential of using reinforcement learning to improve the generative capabilities of recurrent neural networks in molecular design.This paper introduces a method to tune a sequence-based generative model for molecular *de novo* design using deep reinforcement learning. The model is trained to generate molecules with specified desirable properties, such as avoiding certain elements (e.g., sulfur) and generating analogues to query structures. The method uses an augmented episodic likelihood, which combines prior likelihood and a user-defined scoring function, to fine-tune the model. The model is tested on various tasks, including generating sulfur-free molecules, creating analogues to the drug Celecoxib, and generating compounds predicted to be active against the dopamine receptor type 2 (DRD2). The results show that the model can generate a high fraction of valid and predicted active molecules, even when the training set does not include the target compounds. The study demonstrates the potential of using reinforcement learning to improve the generative capabilities of recurrent neural networks in molecular design.