An introduction to pyMarmote and pyMarmoteMDP for Markovian modeling - A tutorial

The tutorial provides an introduction to the capabilities of software environment Marmotevia its Python interface: pyMarmote/pyMarmoteMDP.Marmote is a programming library for modeling with Markov chains, analyzing and "solving" these chains. It provides the objects for building continuous-time and discrete-timeMarkov chains on discrete but possibly complicated state spaces. Once defined, Markovchains can be analyzed with a variety of methods, including structural analysis, Monte-Carlosimulation and numerical solution for criteria such as transient and stationary distributions,or average hitting times.The extention MarmoteMDP provides a library for modeling with Markov Decision Processes. It provides algorithms for numerically determining optimal policies for all classicaloptimization criteria. It also features capabilities for the structural analysis of the resultingpolicies and value functions.The presentation consists in showing how to use Marmote in a series of thematicPython notebooks. These notebooks will be shared so as to allow motivated attendants topractice themselves.