Does Your Accurate Process Predictive Monitoring Model Give Reliable Predictions?

The evaluation of business process predictive monitoring models usually focuses on accuracy of predictions. While accuracy aggre gates performance across a set of process cases, in many practical sce narios decision makers are interested in the reliability of an individual prediction, that is, an indication of how likely is a given prediction to be eventually correct. This paper proposes a first definition of business process prediction reliability and shows, through the experimental evalu ation, that metrics that include features defining the variability of a pro cess case often give a better prediction reliability indication than metrics that include the probability estimation computed by the machine learn ing model used to make predictions alone