Debugging and Testing

"When the software is 95% complete there's still 25% to go" - anonymous

"You can easily save yourself two or three hours of design time by spending 20 or 30 hours in the lab afterwards"

A bug is any discrepancy between intended program behavior and the way it would actually work, under any feasible circumstance. This definition is both subjective, because it involves human intention, and hypothetical, in the sense that a bug can be there even if one of those circumstances hasn't yet occurred.

Bugs can enter at any stage of the software-development process. There can be discrepancies between the customer's intentions and the written specifications, between the programmer's understanding of the specifications and the customer's, or between the programmer's intentions and the code he/she writes because of a misunderstanding either features of the language or of modules produced by other programmers. These bugs are present before the first line of the program has been entered into the editor. Therefore, as a programmer, you should make sure that the specifications make sense and that you understand them. Also, you should attempt to understand your tools, both your programming language and the software written by others that is involved in your project.

Rule #1: No amount of debugging is going to replace good design

Finding your bug is a process of confirming many things you believe are true -- until you find one which is NOT true.

From information theory, we know that the test that gives the most information is the one that cuts the possibilities in half. Unfortunately, most students work from definitive hypotheses, i.e., ones that say, ``The cause of the bug must be such and such.'' They perform tests which, if the hypothesis is correct, fix the bug, but, if the hypothesis is false, give no information. This is an inefficient search strategy. (Try to imagine how you would find a particular number between 1 and 1000 by asking questions of the form, ``Is it ...?'' A much more efficient approach would be the standard binary search.)

More than half the bugs introduced by professional programmers are introduced during the debugging process. Similarly, students introduce many bugs and destroy the structure of their code in the process of making changes associated with such testing their hypotheses. It is important to use CVS carefully and frequently so that you can back out of any changes done to test a particular hypothesis.

The binary search principle:

First try to confirm that everything is OK at about 1/2 of your program. If so, try at 3/4, if not, try at 1/4.

Creating a stack trace can sometimes be helpful to figure out how did you get to a certain point in the program:

java.lang.Exception e = new java.lang.Exception();

e.printStackTrace();

In class, we demonstrated debugging using Eclipse and we'll have a tutorial on how to use JUnit. You can download them, free, for Unix and WinNT, but you need a separate JDK, which you can download from Sun. All of this stuff is installed on all Owlnet machines, both those running WinNT and Unix (but you'll need to be running Solaris 8 to make Eclipse work). The there's also JBuilder on the Owlnet.

The example code we showed in class is in the comp314 home directory on Owlnet (/home/comp314). The packages that we used are below that (e.g., /home/comp314/comp314/util/Debug.java and /home/comp314/junit/framework/Assert.java hold the classes for comp314.util.Debug and junit.framework.Assert). You probably want to configure /home/comp314 to be in your classpath.

In class, we showed:

How and where to set breakpoints to study your program.
How to use Assert statements within your program to catch errors.
How to manage debugging statements without commenting and uncommenting your code.
How to do unit testing with JUnit.

Stuff we didn't talk about, but which is still worth knowing:

How to use CVS from within Eclipse. (Tonight's tutorial just shows how to do it with Emacs.)
How to do refactoring from Eclipse. (While not exactly a debugging technique, refactoring lets you manage your code efficiently, moving around methods among classes if you feel it's necessary, to keep your program clear and concise.)

In class, I showed you how to evaluate expressions. This is pretty straightforward (or, as straightforward as anything is in Eclipse). You need to be looking at the Debug View. Then, the window in the upper right can show Variables, Breakpoints, Inspector, and Display (selectable from the tabs at the bottom of the window). Click on Display, and then type anything you want. You can type an expression (e.g., y * width + x) or a method you'd like to run (e.g., this.print(System.out)). You highlight the expression, right-click, and select "Display" if you want to simply get the answer, and you select "Inspect" if you want that expression copied over to the Inspector tab, where you can then see the value of that expression as it changes over time.

If you have a debugging print method, such as FrameBuffer.print(), you can run this any time using the trick above, and you then get to have a pretty view into the guts of your program.

Another useful technique, particularly when your data structures get more complex, is to build sanity checking methods into your classes. For example, if you're implementing a heap, you might write a sanity checker that verifies the heap property and either returns nothing or has an assertion failure. If you thought your program was misbehaving because the heap was broken, you could call the heap sanity checker from the debugger, using the above "Display" trick, to see if your heap is, in fact, a heap. You could also use this same method as part of a unit test that does a number of heap operations and calls the sanity checker between each of them.

Finally, while we didn't spend any time in class on it, you will find your programs easier to debug when you stick to standard practices for how you format your code. Even if you claim that you have some degerate coding style "burned in" to your fingers, it's often worth the effort to program in a "standard" fashion. This makes your code reusable by other people later on. Please go read Sun's Java Coding Conventions and do your best to follow them.

One last note: the Unix world has standardized that a "tab" character is eight spaces long, and that code should be indented by four spaces for each step. In the PC world, they often let a tab be four spaces long, which tends to make code look ugly if it was formatted for standard Unix conventions. In Comp314, we plan to follow the Unix conventions. This means you may need to reconfigure Eclipse. Under Workbench -> Preferences, open the Java -> Code Formatter option panel. Click the "Style" tab, then set the number of spaces representing a tab to "4" and uncheck the box for "Indentation is represented by a tab." Next, go to Java -> Editor option panel, select the "General" tab, and set the "Displayed tab width" to "8". This will get you standard Unix indentation and display that will let your files move back and forth with Emacs and to your graders.

Yes, Eclipse is complex. But, it's free, it's fast, and it does seem to work, which is more than I can say for NetBeans.

Debugging Guidelines

As programs become large, finding bugs and debugging become a time consuming job. Debugging is an art that can be learned and developed. However, it requires plenty of experience in writing and debugging programs. The structured, top down approach to writing programs we discussed in class is one valuable tool for producing quality, working programs. However, there is no substitute for extensive programming experience and the best way to gain programming experience is to write, test, and debug programs; write, test, and debug programs; write, test, and debug programs; etc. etc.

Certain debugging guidelines;

The first step cannot be emphasized enough. Spend plenty of time in preparing the algorithm. A logically clear algorithm is much easier to debug than an ad hoc algorithm with many fixes for previously found bugs. Trial and error programming may never be bug free.
Use top down development for your algorithms, and use modular programming for your implementation. Top down development makes logic transparent at each stage and hides unnecessary details by relegating them to later stages. Modular programming localizes errors in small functions, which can be easily debugged.
Document your program using comments as you write it. It is a poor habit to delay documenting a program until it is done. Frequently, the very process of documenting a program makes the logic clearer and may well eliminate sources of errors.
Trace your program flow manually. This means: examine what happens to values of key variables at key points in the program. Use judicious starting values for these variables. Particularly, check values of variables at critical points, such as loop beginnings and ends, function calls, and other key points in the program.
Use a symbolic debugger. Eclipse has a great one. The time spent to learn the use of a debugger makes debugging of most programs an easier task.
Otherwise, use helper classes, such as comp314.util.Debug. That is, use it to print out values of key variables at key positions in the program to help pin-point the program segment where the bug may be located. The program segment containing a bug can be narrowed until the exact one or two lines of code are pin-pointed. It is then easier to spot the error and correct it.
Pin-point the methods which generate errors. Rewrite the methods and classes if they are overly complex or long. Many times, it is easier to rewrite a method than to rectify poor logic.
In program development, initially we need debug statements. Later, once a program is debugged, the debug statements must be removed. Aspect-oriented programming, which we will discuss in the next lecture, provides a modular, object-oriented way of introducing such statements in the development program and removing them later.