The paper presents a novel approach to Recommender Systems (RS) by modeling the recommendation process as a Markov Decision Process (MDP). The authors argue that traditional RS models, which treat recommendations as static predictions, are insufficient for capturing the sequential nature of user behavior. By using MDPs, the system can consider both the long-term effects and expected value of each recommendation, leading to more informed and effective recommendations. The core contribution of the paper is the development of an $n$-gram predictive model to generate an initial MDP. This model represents user behavior as a Markov chain, where states correspond to sequences of user selections. The authors detail enhancements to the $n$-gram model, including skipping, clustering, and finite mixture modeling, to improve its accuracy. These enhancements help address data sparsity and computational complexity issues. The paper also formalizes the recommendation problem as an MDP, defining states, actions, rewards, and transition functions. The initial transition function is estimated using the predictive model, and the system is initialized with this function to ensure it performs well upon deployment. The authors then describe how to solve the MDP to determine the optimal policy for recommending items. The evaluation section uses real data from an Israeli online bookstore, *Mitos*, to compare the proposed MDP-based approach with other models, including Microsoft's Commerce Server 2002 Predictor and non-sequential Markov chains. The results show that the MDP-based approach outperforms these models in terms of both recommendation score and exponential decay score, demonstrating the effectiveness of the proposed method. Finally, the authors discuss future work, including deploying the system in a live environment, exploring alternative initialization methods, and improving the predictive model by learning weighting functions and mixture weights from data.The paper presents a novel approach to Recommender Systems (RS) by modeling the recommendation process as a Markov Decision Process (MDP). The authors argue that traditional RS models, which treat recommendations as static predictions, are insufficient for capturing the sequential nature of user behavior. By using MDPs, the system can consider both the long-term effects and expected value of each recommendation, leading to more informed and effective recommendations. The core contribution of the paper is the development of an $n$-gram predictive model to generate an initial MDP. This model represents user behavior as a Markov chain, where states correspond to sequences of user selections. The authors detail enhancements to the $n$-gram model, including skipping, clustering, and finite mixture modeling, to improve its accuracy. These enhancements help address data sparsity and computational complexity issues. The paper also formalizes the recommendation problem as an MDP, defining states, actions, rewards, and transition functions. The initial transition function is estimated using the predictive model, and the system is initialized with this function to ensure it performs well upon deployment. The authors then describe how to solve the MDP to determine the optimal policy for recommending items. The evaluation section uses real data from an Israeli online bookstore, *Mitos*, to compare the proposed MDP-based approach with other models, including Microsoft's Commerce Server 2002 Predictor and non-sequential Markov chains. The results show that the MDP-based approach outperforms these models in terms of both recommendation score and exponential decay score, demonstrating the effectiveness of the proposed method. Finally, the authors discuss future work, including deploying the system in a live environment, exploring alternative initialization methods, and improving the predictive model by learning weighting functions and mixture weights from data.