Debugger limitations

Preface

Source-level debuggers [1] (hereafter also referred to simply as debuggers) are very useful and powerful tools, but just like any other tool, they can also be misused, overused and overrated. According to my experience, some software developers (mostly the novice ones) tend to occasionally overestimate the capabilities of the modern debuggers and neglect to consider other alternatives, usually because they forget or even ignore the limitations of these powerful tools. Hence, I have found it a good idea to present here an indicative collection of debugger limitations, which I have discovered one-by-one during my own debugging experiences. Although I mostly work in C/C++, chances are that many of these limitations will probably apply on most source-level debuggers, regardless of the programming language.

Limitations

Modern debuggers [1] have proved to be extremely helpful tools and invaluable time savers for most of the developers nowadays. However, the modern debuggers still have several limitations, which make them ineffective in some rare but difficult debugging situations. [2] Most of these limitations are in fact preconditions, which are required in order to use the debuggers successfully, like the ones included in the following non-exhaustive, but nevertheless indicative list:

1. The problem should be reproducible in a fairly short amount of time.
2. The problem should not affect the functionality of the debugger itself.
3. The use of the debugger should not affect the behavior of the target problem.
4. The problem should be reproducible in a build configuration, which allows the debugger to work effectively.
5. The problem should be reproducible in a software environment, which allows the debugger to work effectively.
6. The hardware, in which the problem is reproducible, should be accessible by a local or remote debugger.
7. In some cases, the user of the defective program should be willing to help reproducing the problem, sometimes by investing a lot of time and effort.
8. In some cases, the user of the defective program should be willing to provide all the means and rights required for the remote debugging to function.

Since I am having the feeling that the above list of preconditions might seam a bit too theoretical, I am also adding bellow some concrete examples of how exactly these preconditions might be violated:

• Some years ago, I was hunting a very difficult bug, so rare that clearly violated the precondition-1. Furthermore this problem manifested itself only when the program was wrapped inside a security shell of a well-known copy protection system. This shell was implementing a variety of security measures and among them a strong anti-debugging defence mechanism, hence I also had to confront with a violation of the precondition-5. Last but not least, the result of the bug was a blue-screen situation in Windows-XP, which obviously violated the precondition-2. Needless to say, that any attempt to use a debugger for investigating a bug like this, is undoubtedly a pointless waste of time.
• In general, when the problem is causing hangs, crashes or freezes to the operating system itself, the debugger is unlikely to be very helpful. (Violation of the precondition-2.) Fortunately, modern operating systems are very stable and hence these cases are rather rare nowadays.
• When hunting bugs in timer events, other sensitive event handling code, or frequently called drawing code, the interactive features of the source-level debuggers (breakpoints, watchpoints, etc.) tend to become inconvenient and hard to use. Although the use of a debugger is often feasible in most of these cases, using logging instead is usually much more practical and effective. (Conflict with the precondition-2.)
• When the defective program, is using the computer display exclusively by itself (some direct-X applications do that), the use of a debugger can be often impractical and even impossible in some rare situations. (Violation of the precondition-2.) The situation can be improved a lot with the use of logging or remote debugging.
• Developers who are hunting transient performance bugs, or thread/process synchronization bugs, frequently find the presence of a source-level debugger and the use of breakpoints too intrusive. Ironically, when using a debugger in order to reveal such problems, we often end up hiding them even more and making them completely unreachable! (Violation of the precondition-3.)

Apart from the above preconditions, the source-level debuggers also have, due to their interactive nature, some significant shortcomings in their information gathering capabilities. Although debuggers are very good at representing the current sequence of nested function-calls, also known as call stack, they are not particularly good at representing the sequence of actions or states during the execution time. Consequently, when hunting bugs that depend on a previous sequence of actions or states, a simple logging facility can often be more effective than the most powerful debugger. However, some modern debuggers have been recently equipped with the tracepoint [2,3] feature, providing logging capabilities, along with their excellent interactive facilities.

Conclusion

The list of limitations presented in this text indicates that, in spite of the indisputable fact that the modern debuggers are extremely powerful and versatile, it is also true that in some circumstances their effectiveness might be considerably constrained. More practically, if you notice a conflict with the above list, during the debugging, then it might be a good idea to abandon your source-level debugger and consider using alternative debugging techniques. This also proves that, the knowledge of alternative or complementary debugging techniques is still very useful.

References

1. Debugger.
   http://en.wikipedia.org/wiki/Debugger
2. Unusual software bugs.
   http://en.wikipedia.org/wiki/Unusual_software_bug
3. VSD tracepoints.
   http://msdn.microsoft.com/en-us/library/ktf38f66.aspx
4. GDB tracepoints.
   http://developer.apple.com/documentation/developertools/gdb/gdb/gdb_11.html