provided by: 
Originally published at Internet.comFor a native C++ developer, VSTS Developer Edition provides support throughout the development cycle. From VSTS source annotations that enable you to annotate methods with the valid pre- and post-conditions of method parameters (and have the compiler warn you when the annotations are violated) through static code analysis and unit testing, VSTS provides a huge range of functionality that helps the native developer. This article looks at the VSTS Verifier, an analysis tool that monitors unmanaged code as it executes, and detects errors related to incorrect memory, lock, and handle usage.
My previous static code analysis article showed how some types of application errors can be detected by parsing the source code. However, the effectiveness of static code analysis is limited because interaction with black-box systems (such as third-party components and Windows DLLs) makes determining all the possible invalid states that an application can get into impossible.
In contrast, dynamic analysis inspects the values of key variables passed to various system functions to detect problems. While the method being called will detect some invalid parameters and report them via error conditions, some problems can go undetected when the invalid parameter is used. In that case, the symptoms of the problem will show up only much later as corrupt memory and unexpected application terminations.
Getting Started with the Application Verifier
Starting the Application Verifier is a simple matter. Just choose Debug | Start With Application Verifier, as shown in Figure 1.
Figure 1. Starting the Application Verifier The first time you use the Application Verifier, Visual Studio will prompt you to download an additional DLL that allows handle and heap inspection through documented interfaces. For users in controlled environments that may need operations staff approval for patch installation, this Microsoft Knowledge Base article describes the files included in the Windows update.
For most users of the Application Verifier, the most difficult requirement is remembering to start the Verifier rather than launching a standard debugging session. Given that standard debugging functionality is available when using the Verifier, it is worth swapping the default shortcut keys for starting a debug session (F5) with the keys for starting a Verifier session (Shift+Alt+F5). This way, the power of the Verifier will be automatically utilized every time a debugging session occurs.
When the Application Verifier detects a problem, it halts execution and displays a dialog like that shown in Figure 2.
Figure 2. Verifier Stop Message
Diagnosing the problem hopefully will be a simple matter of looking at the history of the handle, lock, or heap pointer that caused the Verifier to stop execution, and working back to a point in time when the corruption occurred. While this isn't always the easiest debugging task, it beats starting the debugging process further downstream where heap or file corruption is the only symptom of a problem.
When the Verifier halts execution, a detailed description of the problem is also added to the task view, as shown in Figure 3. The information in the task view can be very handy, as the dialog shown in Figure 2 needs to be dismissed before the developer can jump into thee debugger and start inspecting variable values.
Figure 3. Verifier Tasks List View
In addition to the Task List information, a separate Application Verifier window provides the most detailed information on the stop event (see Figure 4).
Figure 4. Verifier Window
The Application Verifier Checks
The VSTS Application Verifier focuses on three core areas: operating system handles, locks, and heap memory. Of these three types of check, handles are the simplest-Application Verifier checks to ensure that the handle is not null; the handle points to the correct operating system object type; and the handle is not used in a wait operation in the DllMain function, which is a sure recipe for deadlocks.
Lock checks include the following: * Checks for under and over initialization * Freeing of memory or unloading of modules that are holding a lock * Locks with invalid owners * Corrupt locks
The heap checks follow a similar pattern to the other two verifier targets, with checks on heap corruptions, invalid heap deletions, invalid heap handles, and heap overruns.
Configuring the Application Verifier
The configuration options for the Application Verifier are fairly minimal, as Figure 5 shows.
Figure 5. Verifier Configuration
Apart from being able to turn the three main verification features on or off, you can use the heap verification with or without guard pages (guard pages provide the earliest detection of corruption at the expense of taking up more memory) and the location of the heap guard page. The default location of the heap guard page is at the end of an allocation, which will detect buffer overruns. However, you can move this to the beginning of the allocation block if you need to detect the much-rarer buffer under-runs.
What Are You Waiting For?
The Application Verifier is one of the easiest VSTS tools to gets into-simply choose a different menu option to begin your debugging session, and let the Verifier share the hard work of tracking down some of those nasty native code problems.
About the Author
Nick Wienholt is an independent Windows and .NET consultant based in Sydney, Australia. He is the author of Maximizing .NET Performance from Apress, and specializes in system-level software architecture and development with a particular focus on performance, security, interoperability, and debugging. Nick can be reached at NickW@dotnetperformance.com.
Author: Nick Wienholt
Read article at Internet.com site