JauntePE 0.3.0 workarounds for Vista compatibility issues

[fawkes]

Although JauntePE was not made for Vista it still works with the exception of a couple of issues. Thats why I would like to see all Vista related issues encountered with JauntePE 0.3.0 posted here so we can compile a list of Vista issues and workarounds found for those issues. So far only crownixx has reported issues with JauntePE 0.3.0 and Windows Vista and therefore all credit should go to crownixx. The known Vista issues so far are:

First issue reported by crownixx was that launching an application with JauntePE and having JauntePE hook all APIs would render the application unable to bring up the "Open File" or "Save File" dialog. The workaround suggested by crownixx was to make an optimized configuration for JauntePE hooking only the needed APIs.

Secondly crownixx noticed that JauntePE will not redirect all usage for some applications if the application executable is not UPXed, so the workaround suggested by crownixx is to compress the executable with UPX.

Building upon this I found out that by not hooking "RegOpenKeyExW", JauntePE applications had no problems bringing up the "Open File" or "Save File" dialog so this could also be used as a workaround for the first issue if one does not wish to make an optimized configuration. Hopefully there won't be other issues with Vista, but if so please share them here

[m^(2)]

1. It's likely that you shouldn't be hooking RegOpenKeyExA as well for ANSI programs.
And not hooking this means no portable registry.

2. The UPX thing looks strange, I *guess* that this is a bug and UPX by randomly changing the executables has some chance of making them not trigger the issue...but is not really a solution.

[fawkes]

Good point, I had not thought about that. The application I tested with the workaround of not hooking "RegOpenKeyExW" was an ANSI application. I guess it might be possible that you have to not hook "RegOpenKeyExA" instead if your application is an Unicode application.

[m^(2)]

Good point, I had not thought about that. The application I tested with the workaround of not hooking "RegOpenKeyExW" was an ANSI application. I guess it might be possible that you have to not hook "RegOpenKeyExA" instead if your application is an Unicode application.

Actually I meant it the other way.
A are ANSI functions and W - Unicode ones.
Windows API is implemented the way that ANSI functions usually just translate own arguments and call Unicode counterparts. I'm not sure if it's always this way and wouldn't be surprised if some were actually implemented twice.
Apparently GetOpenFileNameA function, which is responsible for showing the open files dialog in ANSI programs (probably indirectly) calls RegOpenKeyExW.
GetOpenFileNameW surely doesn't call RegOpenKeyExA because translation is never done this way, it will surely call RegOpenKeyExW too.

Hope that's understandable...

[fawkes]

I did not know this, thanks.

[crownixx]

The workaround suggested by crownixx was to make an optimized configuration for JauntePE hooking only the needed APIs

No..no. Dont get my post wrong. If your read carefully, what i write is to make optimized runtime ini configuration and not APIs only. From what i understand, optimize runtime ini can be made by controlling the registry usage, file system usage, module usage, child process usage and APIs usage.

To be honest, optimizing runtime ini using APIs usage is far too advance for me and i'm not dare yet to touch those settings.

Ok, for Open File" or "Save File" dialog problem workaround, to be precise you should optimize the Registry usage.
* If your app is not using registry usage but only use file system usage, then turn off your registry redirection
* If your app use registry, the way you should work is RegistryExclude all, then RegistryInclude the application registry key usage

I try to search which registry keys that cause this failure and the keys are
1=HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID
2=HKEY_CLASSES_ROOT\CLSID
I try to go more detail which CLSID that are cause the failure but i only manage to exclude 50 plus keys then i gave up.

2. The UPX thing looks strange, I *guess* that this is a bug and UPX by randomly changing the executables has some chance of making them not trigger the issue...but is not really a solution

Yes you are right. I learn new english term for this: To make JauntePE work in Vista you have to cross your fingers

[m^(2)]

The workaround suggested by crownixx was to make an optimized configuration for JauntePE hooking only the needed APIs

No..no. Dont get my post wrong. If your read carefully, what i write is to make optimized runtime ini configuration and not APIs only. From what i understand, optimize runtime ini can be made by controlling the registry usage, file system usage, module usage, child process usage and APIs usage.

To be honest, optimizing runtime ini using APIs usage is far too advance for me and i'm not dare yet to touch those settings.

Ok, for Open File" or "Save File" dialog problem workaround, to be precise you should optimize the Registry usage.
* If your app is not using registry usage but only use file system usage, then turn off your registry redirection
* If your app use registry, the way you should work is RegistryExclude all, then RegistryInclude the application registry key usage

I try to search which registry keys that cause this failure and the keys are
1=HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID
2=HKEY_CLASSES_ROOT\CLSID
I try to go more detail which CLSID that are cause the failure but i only manage to exclude 50 plus keys then i gave up.

2. The UPX thing looks strange, I *guess* that this is a bug and UPX by randomly changing the executables has some chance of making them not trigger the issue...but is not really a solution

Yes you are right. I learn new english term for this: To make JauntePE work in Vista you have to cross your fingers

There's a simple, but almost 100% correct way of making such optimization:
Take strings from all application's executables and write it to some file (strings file.exe >log.txt). Then look for strings from the allowable hooks list in the JPE readme. If allowable hook is in the log, then you have to include it, otherwise it's unnecessary.

Warning1 (critical!): Use it only with uncompressed executables that are not installers! To ensure that program isn't packed, use PEID.
Warning2: Doesn't work with some interpreted languages like AutoIT and AutoHotkey - it usually shows more entries than are actually used. NSIS is an installer language, so it doesn't work either.

[fawkes]

Sorry for having misinterpeted what you meant crownixx, but I am happy you cleared it up, if only for nothing else then for my understanding. I will also look into the CLSID keys you were talking about, but m^(2)'s idea looks good. If only we could do it progmatically then the workload would be less. Keep up the great work guys

[crownixx]

If only we could do it progmatically then the workload would be less

It function is already there. Look for the Discovery settings. it will list down the API usage and others information too

[Stone Tablet]

See:

http://www.box.net/shared/4cx4i2k00r

File:JPErt030AlphaNB18.7Z

From Readme file:

This is an experimental, 2K or newer only, version of the default (def) build
of the jauntePE.dll runtime DLL. It's been tested on a number of applications
on XP Home SP2 and has performed fine so far. But use with caution.

This version hooks the low-level ntdll.dll api file system functions instead of
hooking a large number of kernel32 and other higher level api file system
functions. Because of this, apps load and run faster. And because of this, it
might also work better on versions of Windows newer than XP.

[m^(2)]

Good!
That's how it should be done from the start. Should be compatible with more apps.

[crownixx]

Thanks,
I can't wait to put it into the test..

[crownixx]

File:JPErt030AlphaNB18.7Z

.., it
might also work better on versions of Windows newer than XP.

I have test it in vista and the registry usage is still bypass if the application is not compressed upx. The file system redirection however still working fine.