The book provides a comprehensive overview of Markov chains and their applications in various fields, including queueing models, stochastic automata networks, and numerical methods. It is divided into several chapters that cover both theoretical foundations and practical computational techniques. - **Chapter 1** introduces Markov Chains, including discrete-time and continuous-time Markov Chains, nonnegative matrices, stochastic matrices, and cyclic stochastic matrices. It also discusses state clustering indicators and queueing models. - **Chapter 2** focuses on direct methods for solving linear systems, such as Gaussian elimination, LU decomposition, and inverse iteration. It explores the implementation considerations and stability analysis. - **Chapter 3** delves into iterative methods, including the power method, Jacobi, Gauss-Seidel, SOR, and symmetric SOR methods. It also covers block iterative methods and preconditioned power iterations. - **Chapter 4** discusses projection methods, such as simultaneous iteration, Krylov subspaces, GMRES, Arnoldi, Lanczos, and conjugate gradients. It provides examples and implementation details. - **Chapter 5** introduces block Hessenberg matrices and recursive methods for solving queueing problems, including the matrix-geometric approach and explicit solutions. - **Chapter 6** covers decompositional methods for NCD Markov Chains, stochastic complementation, and iterative aggregation/disaggregation methods. It includes convergence properties and numerical experiments. - **Chapter 7** focuses on p-cyclic Markov Chains, directed graphs, and numerical methods for periodic and cyclic matrices. It also discusses reduced schemes and IAD methods. - **Chapter 8** addresses transient solutions, including uniformization, ODE solvers, and Krylov subspace methods. - **Chapter 9** explores stochastic automata networks, both noninteracting and interacting, and methods for computing probability distributions. The book also includes a section on software tools, such as MARCA (MARkov Chain Analyzer) and XMARCA, which are useful for modeling and solving queueing networks and Markov Chains.The book provides a comprehensive overview of Markov chains and their applications in various fields, including queueing models, stochastic automata networks, and numerical methods. It is divided into several chapters that cover both theoretical foundations and practical computational techniques. - **Chapter 1** introduces Markov Chains, including discrete-time and continuous-time Markov Chains, nonnegative matrices, stochastic matrices, and cyclic stochastic matrices. It also discusses state clustering indicators and queueing models. - **Chapter 2** focuses on direct methods for solving linear systems, such as Gaussian elimination, LU decomposition, and inverse iteration. It explores the implementation considerations and stability analysis. - **Chapter 3** delves into iterative methods, including the power method, Jacobi, Gauss-Seidel, SOR, and symmetric SOR methods. It also covers block iterative methods and preconditioned power iterations. - **Chapter 4** discusses projection methods, such as simultaneous iteration, Krylov subspaces, GMRES, Arnoldi, Lanczos, and conjugate gradients. It provides examples and implementation details. - **Chapter 5** introduces block Hessenberg matrices and recursive methods for solving queueing problems, including the matrix-geometric approach and explicit solutions. - **Chapter 6** covers decompositional methods for NCD Markov Chains, stochastic complementation, and iterative aggregation/disaggregation methods. It includes convergence properties and numerical experiments. - **Chapter 7** focuses on p-cyclic Markov Chains, directed graphs, and numerical methods for periodic and cyclic matrices. It also discusses reduced schemes and IAD methods. - **Chapter 8** addresses transient solutions, including uniformization, ODE solvers, and Krylov subspace methods. - **Chapter 9** explores stochastic automata networks, both noninteracting and interacting, and methods for computing probability distributions. The book also includes a section on software tools, such as MARCA (MARkov Chain Analyzer) and XMARCA, which are useful for modeling and solving queueing networks and Markov Chains.