Perfect Software--And Other Illusions about Testing by Gerald M. WeinbergExhaustive testing of software is generally impossible; in all but the most simple of cases the number of possible paths is effectively infinite. By that I mean that while the number is finite, it is so large that only a tiny percentage could ever be tested. So in practice the number is infinite. Since humans are prone to error and the number of possible paths where a bug can be introduced into a program is also very large, the bug-free (perfect) program is an unattainable goal. It is no fun, but that is just the way things are.
Being driven by optimism and necessity, humans still create millions of lines of code every year and many of these programs are released into the untamed wilderness known as the customer base. Therefore, conscientious developers have no choice but to develop a testing program that will have a reasonable chance of identifying the most significant and likely bugs in the software.
As Weinberg points out, some people do make the choice to do very little testing, sometimes relying on an approach like If we get lucky, no one will experience that bug. Others simply redefine the bug as a feature and move on. Neither of these approaches is ethical nor will it provide you with legal cover. In a very witty yet understated style, Weinberg describes some of the circumstances he has encountered where people simply denied there was a problem and if that failed tried to hoist the problem onto someone else.
As is the case in most areas of software development, the solution lies more in the realm of group psychology rather than technical expertise. Having authored the classis The Psychology of Computer Programming Weinberg is an expert at finding what solutions are available to this seemingly impossible problem. It starts with eliminating all illusions and delusions and accepting the inherent limitations of what software testing can do. After that, one develops a testing process that is executed concurrently with the development of the software. The first step is to test the modules by creating a set of reasonable coverage test values. Varying from module to module, this set covers some of the most likely entries as well as the bizarre ones and the ones on the limits.
Once the tests on the modules are complete, the next step is to integrate the modules into a functional unit. Once the unit is stable and can be run, an additional set of reasonable coverage test operations is created and executed. In each case, detailed knowledge of what the software is supposed to is used to develop the sets of test values.
Weinberg is also forceful in debunking the current state of the automated testing of software, a tactic that he dismisses. The strongest argument against using software to test software is grounded in human psychology, namely the easily adopted assumption that if it came from a computer then it must be right. This belief in the infallibility of technology has a strong tendency to dumb-down humans and that is to be avoided if at all possible.
As stated before, the quality software developer has no choice but to test their product. If you want to maximize your likelihood of releasing a quality product, then your first step is for everyone in your team to read and discuss this book. Doing so will clear a lot of the yucky stuff out of your path. So while you may still step in it, the pile you squish is either of your own making or one of the few that Weinberg does not cover.
Published in the online Journal of Object Technology reprinted with permission
Download Perfect Software And Other Illusions about Testing
I loved this book. It is a must read for anyone who is managing or working in software testing. It explains the true issues with software testers, developers and managers, all who have their own perspective on software and deadlines. It also helps accentuate the need for intelligent testing and human decision making. I will recommend it to my fellow testers!! When I read this book, I had more than 10 years of experience in software development and testing, and I immensely benefited from studying this book.
Gerald Marvin Weinberg October 27, — August 7, was an American computer scientist , author and teacher of the psychology and anthropology of computer software development. Gerald Weinberg was born and raised in Chicago. Weinberg started working in the computing business at IBM in at the Federal Systems Division Washington, where he participated as Manager of Operating Systems Development in the Project Mercury — , which aimed to put a human in orbit around the Earth. Further Weinberg was an author at Dorset House Publishing since , consultant at Microsoft since , and moderator at the Shape Forum since He was a member of the Society for General Systems Research since the late s.