RL-VLM-F is a method that automatically generates reward functions for agents to learn new tasks using only a text description of the task goal and the agent's visual observations. It leverages feedback from vision language foundation models (VLMs) to generate preference labels over pairs of image observations, which are then used to learn a reward function. This approach eliminates the need for extensive human effort in manually crafting reward functions and outperforms prior methods that use large pretrained models for reward generation. RL-VLM-F is tested on various tasks, including classic control, rigid, articulated, and deformable object manipulation, and shows superior performance compared to baselines such as VLM Score, CLIP Score, BLIP-2 Score, and RoboCLIP. The method uses a two-stage prompting strategy to generate preference labels, which improves the accuracy and effectiveness of the reward function. The results demonstrate that RL-VLM-F can produce effective rewards and policies that solve diverse tasks, and it significantly outperforms prior methods in terms of performance and efficiency. The approach is applicable to a wide range of tasks and can be extended to more complex scenarios with the integration of advanced VLMs. The method also provides a practical pathway for applying reinforcement learning in real-world settings where obtaining reward functions is challenging.RL-VLM-F is a method that automatically generates reward functions for agents to learn new tasks using only a text description of the task goal and the agent's visual observations. It leverages feedback from vision language foundation models (VLMs) to generate preference labels over pairs of image observations, which are then used to learn a reward function. This approach eliminates the need for extensive human effort in manually crafting reward functions and outperforms prior methods that use large pretrained models for reward generation. RL-VLM-F is tested on various tasks, including classic control, rigid, articulated, and deformable object manipulation, and shows superior performance compared to baselines such as VLM Score, CLIP Score, BLIP-2 Score, and RoboCLIP. The method uses a two-stage prompting strategy to generate preference labels, which improves the accuracy and effectiveness of the reward function. The results demonstrate that RL-VLM-F can produce effective rewards and policies that solve diverse tasks, and it significantly outperforms prior methods in terms of performance and efficiency. The approach is applicable to a wide range of tasks and can be extended to more complex scenarios with the integration of advanced VLMs. The method also provides a practical pathway for applying reinforcement learning in real-world settings where obtaining reward functions is challenging.