Maarten's blog

Application Verifier Locks 0x201 Active Critical Section

Suppose you are trying to get your application Vista Certified or OEM Ready. You test your application with Application Verifier and you see this error in the log:

"<avrf:logEntry Time="2008-10-15 : 14:18:33" LayerName="Locks" StopCode="0x201" Severity="Error">

  <avrf:message>Unloading DLL containing an active critical section.</avrf:message>

Now what? From the log, this is difficult to debug.

Little history how you might have arrived here. When you checked the Basics errors in Application Verifier, you got a warning mentioning something along the lines of "For the basic tests you need a debugger". This is because Application Verifier sets break points. If you don't have a default debugger, those breakpoints are ignored hence you need a debugger. Best thing you can do is run your application from the debugger: preferably Windbg since that has some useful extensions that help you navigate the process' memory. You can download it here. After installing and running you can under File | Open Executable navigate to your application. Hit F5, reproduce the error and break into the debugger.

Next best thing is to set Windbg as the interactive debugger (Windbg –I from elevated command prompt). When the application sets a break point (the application verifier layer in this case), the system will see if there is an interactive debugger. If it finds one, it launches it with your application as the target.

Good. So now both ways have taken you to this point:

(11e4.1604): Break instruction exception - code 80000003 (!!! second chance !!!)
eax=000001ff ebx=75988844 ecx=77dde7c4 edx=00000000 esi=00000000 edi=000001ff
eip=77db0004 esp=003ee8fc ebp=003eeafc iopl=0 nv up ei pl nz na po nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00000202
ntdll!DbgBreakPoint:
77db0004 cc int 3

You of course need symbols. I explained how here.

Now comes the interesting part. Learn and appreciate the power of "!analyze –v":

0:000> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
*** WARNING: Unable to verify checksum for e:\Playground\AVRF\x201\Debug\x201b.exe
APPLICATION_VERIFIER_LOCKS_LOCK_IN_UNLOADED_DLL (201)
Unloading DLL containing an active critical section.
This stop is generated if a DLL has a global variable containing a critical section
and the DLL is unloaded but the critical section has not been deleted. To debug
this stop use the following debugger commands:
$ du parameter3 - to dump the name of the culprit DLL.
$ .reload dllname or .reload dllname = parameter4 - to reload the symbols for that DLL.
$ !cs -s parameter1 - dump information about this critical section.
$ ln parameter1 - to show symbols near the address of the critical section.
This should help identify the leaked critical section.
$ dps parameter2 - to dump the stack trace for this critical section initialization.
Arguments:
Arg1: 718933cc, Critical section address.
Arg2: 00f8a104, Critical section initialization stack trace.
Arg3: 05be6fe4, DLL name address.
Arg4: 71870000, DLL base address.

The information is fairly self explanatory. In the process a dll is loaded. This dll initializes a critical section and then before deleting the critical section, the dll is unloaded. There are four parameters which are useful mentioned above. And then some complaining about symbols; followed by more useful information. (I think it complains about symbols because it wants private symbols for the OS. I don't have those, but the results are good anyway.)

Somewhere halfway, you'll find this:

CRITICAL_SECTION: 718933cc -- (!cs -s 718933cc)

So this is the actual critical section we're leaking. Clicking on the hyperlink gives us:

0:000> !cs -s 718933cc
-----------------------------------------
Critical section = 0x718933cc (<Unloaded_TheLib.dll>+0x233CC)
DebugInfo = 0x05c2afe0
NOT LOCKED
LockSemaphore = 0x0
SpinCount = 0x00000000


Stack trace for DebugInfo = 0x05c2afe0:

0x77df99b6: ntdll!RtlInitializeCriticalSectionAndSpinCount+0x19
0x75976936: vfbasics!AVrfpInitializeCriticalSectionCommon+0x136
0x75976ab2: vfbasics!AVrfpRtlInitializeCriticalSection+0x12
0x71881f37: <Unloaded_TheLib.dll>+0x11F37
0x7188bc28: <Unloaded_TheLib.dll>+0x1BC28
0x5e82c02c: MSVCR90D!_initterm+0x1C
0x71883131: <Unloaded_TheLib.dll>+0x13131
0x7188351e: <Unloaded_TheLib.dll>+0x1351E
0x71883451: <Unloaded_TheLib.dll>+0x13451
0x75a059c9: verifier!AVrfpStandardDllEntryPointRoutine+0x109
0x75a7859d: vrfcore!VfCoreStandardDllEntryPointRoutine+0x127
0x7598105e: vfbasics!AVrfpStandardDllEntryPointRoutine+0x10E
0x77dcfcc0: ntdll!LdrpCallInitRoutine+0x14
0x77dd9b28: ntdll!LdrpRunInitializeRoutines+0x270
0x77dd95ae: ntdll!LdrpLoadDll+0x4D5
0x77df29db: ntdll!LdrLoadDll+0x22A
0x7598164c: vfbasics!AVrfpLdrLoadDll+0x5C
0x75fc4d50: kernel32!LoadLibraryExW+0x231
0x75fc4dca: kernel32!LoadLibraryW+0x11
0x00f42d50: x201b!Cx201bDlg::OnBnClickedButton1+0x30

Hmm. Of course. The dll is gone. Which is kind of the crux of this problem. The bold line indicates where the critical section was initialized. If you have multiple critical sections, you might scratch your head which one you're leaking here. So you can try to reload the dll. There is a little snippet how you should be able to do this with .reload etc. I have never been able to do this. If you have, let me know.

What I do is spin up an instance of the application in the debugger and make sure that the dll is still loaded. (You can set breakpoints on LoadLibrary to make sure that you are just after the load, and before the Free. Subject for another blog potentially.) Then, I do a ln (as in list near) with the address from the bold line in the stack trace.

0:001> ln 71881f37
*** WARNING: Unable to verify checksum for E:\Playground\AVRF\x201\Debug\TheLib.dll
e:\playground\avrf\x201\thelib\thelib.cpp(49)+0x11
(71881ef0) TheLib!CTheLibApp::CTheLibApp+0x47 | (71881f70) TheLib!CTheLibApp::`scalar deleting destructor'

There we go. Thelib.cpp(49). Looking at that source line, takes me to the InitializeCriticalSection call of the problematic critical section.

Maarten

Debugging and Symbols

Anytime you want to do anything in a debugger, you need symbols. Best thing you can do is set up an environment variable, so you're done. Here are public Microsoft symbols. Yves has created a little cmd that you can use which I politely copy here since it is handy:

SetX.exe /M _NT_SYMBOL_PATH "SRV*C:\Symbols\Private*\\Symbols\Symbols;SRV*C:\Symbols\Public*http://msdl.microsoft.com/download/symbols"

And for more information there is a KB article.

DllGetClassObject already defined

When you upgrade your project from VC2005 to VC2008 you might get these errors:

mfcs90u.lib(oleexp.obj) : error LNK2005: _DllGetClassObject@12 already defined in d.obj
mfcs90u.lib(oleexp.obj) : error LNK2005: _DllCanUnloadNow@0 already defined in d.obj

Strictly in release builds though.

You can add "AFX_MANAGE_STATE(AfxGetStaticModuleState());" to the first line of InitInstance() in dllmain.cpp.

Application Verifier Logs for a Service

If you have a service and have told AppVerifier to monitor it, you might wonder where the logs are and how you can see them. (Two separate questions). You can find the logs here "c:\windows\system32\config\systemprofile\appverifierlogs" (or the corresponding directory given the system drive and bitness of your application).

Now if you want to see them in AppVerifier, the easiest way is to just copy them over to the current user profile and then do a view logs from AppVerifier.

OEM Ready Links

While I'm trying to find out how to fix the portals, here are some links to documents we frequently use.

• OEM Ready program: http://msdn.microsoft.com/en-us/windowsvista/cc315067.aspx
• There is a Manual for the tool, but it is not linked from anywhere as far as I can tell: http://go.microsoft.com/?linkid=9093231
• Here is Jay's video on Channel 9: http://channel9.msdn.com/posts/philpenn/The-Microsoft-OEM-Ready-Certification-Tool/

Shared Components

If you need to install components that are shared between multiple applications, you want them to go to Common Files. I found a decent description in this doc: "Windows Vista Application Development Requirements for User Account Control Compatibility".You should stay out of system32.

How Do I know if Windbg I actually took effect?

So you're ready to test your application with Application Verifier. The OEM Ready and Certified for Windows Vista (CFWV) test documents specify you type Windbg –I from an elevated command prompt. So you add your application to Application Verifier, add the specific tests under basic and miscellaneous and start your app. You run through your "normal operations" and exit the application. Nothing happened.

Now you have a lingering doubt: did my app not trigger any AppVerif breakpoints? Or did I not set it up right? Hmm.

Here is a little test app. Copy this in a C command line app and build; add the application.exe to AppVerif as in the test steps and it should break in with a NULL handle stop code 303.

#include "stdafx.h"

#include "windows.h"

int _tmain(int argc, _TCHAR* argv[])

{

    HANDLE h = NULL;

    WaitForSingleObject( h, 0 );

    wprintf(L"Okiedokie\n\r");

    return 0;

}

How to Roll Back WinDbg –I

For OEM Ready tests (and for Certified for Windows Vista) one of the requirements is to set up an interactive debugger. The documentation specifies Windbg since it allows you to do use extensions such as "!analyze –v" which will give you a ton of information. One question that then comes up is "How do get back my machine back to its original state before I ran WinDbg –I?"

Those are the registry keys in question that "WinDbg –I" changes:

• \\HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\AeDebug
• \\HKEY_LOCAL_MACHINE\Software\Wow6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug

When you set WinDbg –I, Windbg will add a new string value "Debugger" with a path to windbg.exe.     There is also an Auto string value added.

The recommendation is to back up the registry key before running WinDbg –I. On one of my clean systems the key was existent but the Debugger and Auto values were missing. Removing those effectively rolled back my system to the state it was in before. But don't take my observation as official guidance.

On a final note, if you absolutely don't like WinDbg, you can use Visual Studio as your interactive debugger. Under Tools/Options/Debugging you will find a Just-In-Time section. If you click the "Native" checkbox, your application will break in to Visual Studio when a breakpoint is encountered.

Where is my Session 0 notification?

Today I was working on an OEM Ready application. It is basically a subset of the Certified for Windows Vista program. Application Verifier is used to test reliability (heap overruns, handle usage, etc.). The test steps outline that you need to set up an interactive debugger. You will pass the test if you never see your app break in the debugger.

I had Application Verifier monitor services as well. One of the services seemed to be hanging. When something is not functioning with services in Vista or Windows Server 2008, first thing you need to think about is session 0 isolation. So I opened taskmgr, clicked "Show Processes for All Users" in the process tab and looked for Windbg. Yes it was there. (If you select Session and PID from the options column, you can even see that WinDbg is running in Session 0). But why did I not get my Session 0 notification?

There is a service called UI0Detect (for Interactive Service Detection) and it is set to start manually. As soon as I started the service, the Interactive Service notification popped up and now I could debug the AppVerifier break in.

HighestAvailable and requiresAdministrator when you’re neither Standard User nor Administrator

There is something between standard users and administrators. In UAC discussions we limit ourselves often times to members of the administrators group who get the filtered token and the full token when they log on. But what if you're a member of – say – Network Configuration operators. In that case you're not a standard user (you can do a little more) and you're certainly not an administrator.

Let's test. I have two applications: one manifested with highestAvailable and another with requiresAdministrator. Now I log in as a member of the Network Configuration Operators and I try to run both applications. Here is what happens when I run the first application:

Ah. The credentials dialog. So I need to retype my credentials (remember that if I were a member of administrators group, I would get the UAC prompt). This will allow me to give admin credentials as well. So I can see why this was chosen instead of the UAC prompt. At least it defaults to the logged in account.

When I launch the requireAdministrator application I get this one.

No surprise, just the regular over-the-shoulder credential dialog. Here is what happens if I type the Network Configuration Operators password:

As expected it won't elevate: the requested operation requires elevation. Elevation to a full administrator that is since it is manifested with requiresElevation.

Memory mapped files from XP to Vista

So you have an NT service and a user application that share date through memory mapped files. It has worked for years. It even survived XP SP2. Now Vista comes along and boom. No more. Let's see what's up.

You create your memory mapped file from your service. Something like this:

    HANDLE mmfile = CreateFileMapping(

INVALID_HANDLE_VALUE,

NULL,

PAGE_READWRITE,

upperSize,

lowerSize,

L"MaartensMap" );

And then from your user application you connect to it like so:

HANDLE mmfile = OpenFileMapping (

FILE_MAP_ALL_ACCESS,

FALSE,

L"MaartensMap" );

In Vista GetLastError() will return 2 (ERROR_FILE_NOT_FOUND). Hmm. Where did it go?

Let's look at where the file gets created on XP. Process Explorer shows this for XP:

And this for Vista:

That gives us no clue whatsoever. Identical as far as I can tell. Let's run the server code on both XP and Vista in the user session and see what Process Explorer makes of that. This might give us a clue how the client side user application will look for the memory mapped file. Here is the XP version:

And here is the one on Vista:

Oopsie. That looks different. Suppose that the client application is actually looking under this same name. No wonder that you would get an File not Found error. The names are different. Online search for "sessions\1\basenamedobject" gives hits that point to Terminal Services. There appear to be Local Global and Session namespaces. So prior to Vista and Server 2008 all services shared the same namespace as the first user logging in to the machine. This was session 0. With Vista this is no longer the case. Just like all compatibility issues it is explained in the cookbook.

Apparently if I want to keep this working I wil need to prefix my name with Global. Let's give that a try.

You will see in Process Explorer that the named object created from the service still looks the same. When you try opening it from the user mode application, you will get a different error: 5 this time. That means Access Denied, but at least we were able to find the file. Progress. The reason we get Access Denied is because we set a standard ACL on the memory mapped file and then from the user application we try to open it with All Access. So there is still some work to be done. But that is outside the scope of this blog. J A quick fix could be to open the file on the client size with only read only. But that will imply one-way traffic.

So we have sessions you say. And somewhere you read that Fast User Switching makes use of those sessions. But wait, FUS existed in the XP days as well! Why didn't that break things then, hmm? Hmm, dunno. Maybe it did. So back to the XP machine and make sure to log in as a different user while keeping the initial session alive. Running the server and trying to hook up the client gives us the initial Error 2: file not found. And when running the server side code from within a user app shows the familiar: "Sessions\1\BaseNamedObjects\MaartensMap". Conclusion: this is not a new issue but a lot more visible due to Session 0 isolation.

Enabling DCOM in Windows Server 2008

In order to make DCOM applications work in Windows Server 2008 you need to do two things. Installing the Application role is not needed (but it does add the COM+ network access as in 1) below).

1. Allow "COM+ network access" in firewall rules (type firewall.cpl from a command prompt and on the left you'll see the setting). COM+ is somewhat of a misnomer since it is also for DCOM. COM+ Network Access is a preconfigured program in the exception list that you can just check off.
2. Add your DCOM server to the list of program exceptions in the firewall.

A quick test of DCOM access is the DCOM test client/server app.

Vista’s Task Scheduler

Last week I was working with an OEM who had two applications: a main application and a scheduler. The sole purpose of the latter was to start the main application. Now the good thing obviously was that the customer did not want to overburden the CPU with the main application during the logon process. A polite gesture for both end-users and other applications that want to be first. However, one could ask why not use the built-in Task Scheduler?

The answer is that on XP the scheduler did not have enough flexibility for a delay. On Vista it does. You can specify all kinds of events (logon, startup, even your events that are logged in the event log). And from there you can specify the delay.

You can schedule tasks programmatically with a COM object model or you can use schtasks.exe to batch script it. No more need for custom schedulers.

Welcome Yves

My good friend Yves joined our team last week. He will be helping out with all the Windows Application Compatibility efforts we are undertaking. Yves and I always have heated discussions; maybe some of it will spill over J.

I am really delighted to have him with us.

b20 Labs and Ice cream

One of the daily challenges is passing this thing in b20 every day

.

It's refilled every week.

If you want one while pondering how to make your application compatible with WS08 or Vista, check out the MTC labs section here http://devreadiness.org. It is not explicitly mentioned but ISVs are encouraged to drop us a line if they want to come in.