This has turned out to be a lot more interesting issue than I expected. I was thinking about contacting MS PSS and so I wrote an as-simple-as-possible test application to reproduce the issue. Surprisingly, the test app worked flawlessly on Vista, so started investigating the differences.

First, my test application was using the HTML Help API (hhctrl.ocx) directly and was statically linked (which means that Windows would not allow the test app to start if hhctrl.ocx is not available on the system path). ORF, on the other hand, uses a third-party library that loads hhctrl.ocx dynamically and uses some registry magic to determine the location of hhctrl.ocx, regardless the system path.

My initial thought was that the ORF Log Viewer and the test app was using different versions of hhctrl.ocx and the Log Viewer found the broken one. To test my theory, I started both applications and checked the loaded DLLs by Process Explorer just to find that Log Viewer doesn’t even have hhctrl.ocx loaded. Wow, that makes sense! Calls to hhctrl.ocx will hardly work if it’s not loaded. Actually, our calls to HtmlHelp() ended up in the middle of invalid memory.

I checked the library code again, now digging deeper in the loader code and found that it tries to determine the location of hhctrl.ocx by checking its COM registration under HKCR\CLSID\{adb880a6-d8ff-11cf-9377-00aa003b7a11}\InprocServer32. Under pre-Vista operating systems, the default value for this key used to be simple String Value (REG_SZ), containing e.g. “C:\WINDOWS\system32\hhctrl.ocx”. On Vista, however, this is an Expandable String Value, pointing to e.g. “%SystemRoot%\System32\hhctrl.ocx”. The library fails to check for this and to expand the the value into a real file system path, so it also fails to load hhctrl.ocx. Even worse, the library does not offer proper error handling, so the caller application is not aware of the failed load and calls the trusted HtmlHelp() call, resulting in some accidental endless loop.

End of story, library fixed.

If you can’t wait for ORF 4.0 to run the Log Viewer and the Reporting Tool on Vista, you can apply a manual fix as described below:

• Run regedt32.exe as Administrator (%SystemRoot%\System32\regedt32.exe)
• Open the HKEY_CLASSES_ROOT\CLSID\{adb880a6-d8ff-11cf-9377-00aa003b7a11}\InprocServer32 key
• Change the %SystemRoot% variable in the value to C:\Windows (or what is your local system root path)

Expect a little fun with permissions. Ah, and don’t forget to change it back to %SystemRoot% when ORF 4.0 ships—I am not aware of any problems with hardcoding the system path, but who knows.

No big deal because there is no point installing ORF on Windows Vista: the IIS SMTP component has been removed from IIS 7 of Vista (rant below). Yet even if you do not install ORF on Vista, you may try running the ORF Log Viewer on your brand new OS, but what you get is a program that eats 100% of the processor time and has to be shut down from Task Manager.

I’ve done some debugging and found that the ORF Log Viewer (and the ORF Reporting Tool) hangs on the HTML Help initialization:

HtmlHelp(0, nil, HH_INITIALIZE, dwCookie);
(see MSDN)

It is a Microsoft API that should just work, but it does not. Any ideas?

And now, the promised ranting (I love ranting! :). Why IIS SMTP had to be removed from IIS 7 on desktop Vistas? According to a Microsoft blog post , the whole point of shipping IIS 7 with Vista before Longhorn Server is released is to make life of developers easier. So why they actually make our life harder? As the blogger points out, developers typically use desktop Windows versions for development and testing. We do not run server operating systems on our desktop computers. Now if we decide to upgrade to Vista for developer desktop, we will have to install Windows 2000/2003 Server or XP into a virtualization solution, testing code in the virtual machines. That is big PITA, so Microsoft, I am not happy for your decision.