An Empirical Study to Quantify the SetUp and Maintenance Benefits of Adopting WebDriverManager

Test automation brings several benefits but also presents significant problems that often force developers/testers to carry out tiring and costly manual tasks. Among these tiring tasks while using Selenium WebDriver for testing Web applications, driver management (i.e., version discovery, download, setup and maintenance) ranks in a top position. Recently, an open source Java library, named WebDriverManager, that carries out automatically the driver management process for Selenium WebDriver has been proposed to alleviate the burden of the developers. This library appears to be very promising but until now no one has experimentally evaluated its effectiveness. In this paper, we present an empirical study aimed at understanding whether the use of WebDriverManager allows to reduce both initial and long-term setup efforts of a multibrowser test suite. The results are in favor of WebDriverManager as it allows an average saving of more than 33% of the time for an initial setup which translated annually corresponds to many hours of bare manual driver management to which must be added the hours for understanding that the outdated driver is the cause of the problems reported by the test suite. Having almost weekly, as we estimated, outdated drivers brings also to a reduction in the perceived reliability of E2E test suites. The adoption of WebDriverManager can help to drastically reduce all these problems.