Tests as Documentation: a First Attempt at Quality Evaluation

We present a novel method, and its associated supporting tool, for automatically singling out sloppy tests; that is, tests that run successfully on (some) incorrect implementations, that violate the property they are expected to verify. Our freely available tool is written in C#, but the technique is language agnostic and can be easily applied to other languages.