UPX “accidentally” increments LoadCount for DLLs

When I was preparing last dirtyJOE update I’ve noticed that under some circumstances python DLLs are not freed from memory. What was even more interesting, this behaviour was occurring only in ready to release version of application. I’ve tested few scenarios and I figured out that the problem lays in UPX loader.

I’ll try to explain what exactly happens.


Prerequisites:

  • there are two DLLs (preinstalled.dll, mylib.dll) and main executable (test.exe)
  • import table of mylib.dll contains references to preinstalled.dll
  • test.exe dynamically loads both libs

Scenario:

  1. test.exe tries to load preinstalled.dll (to check if it was installed in the system)
  2. if it succeed LoadCount field in _LDR_DATA_TABLE_ENTRY will be incremented (LoadCount = 1)
  3. test.exe now can try to load mylib.dll (without successful step 1, system will show ugly MessageBox about lacking of preinstalled.dll)
  4. if everything is ok LoadCount of preinstalled.dll should be incremented (LoadCount = 2)
  5. (…)
  6. free mylib.dll, LoadCount of preinstalled.dll should be decremented (LoadCount = 1)
  7. free preinstalled.dll, LoadCount decrementation (LoadCount = 0)
  8. system unmaps preinstalled.dll from application memory

Pseudocode:

	HMODULE hPreInst = LoadLibrary("preinstalled.dll");
	if (0 == hPreInst)
	{
		return;
	}
	//LoadCount = 1

	HMODULE hMyLib = LoadLibrary("mylib.dll");
	if (0 == hMyLib)
	{
		FreeLibrary(hPreInst);
		return;
	}
	//LoadCount = 2

	//do some stuff here, it doesn't matter what ;)

	FreeLibrary(hMyLib);		//LoadCount = 1
	FreeLibrary(hPreInst);		//LoadCount = 0, library is unmaped from the memory

In above situation everything is clear and works perfectly until someone will not pack mylib.dll with UPX (or any other packer with similar imports handling). For stability(?) and compatibility(?) reasons UPX keeps one imported function from every referenced library (except kernel32):

Original imports table UPX imports table
Kernel32.dll  

  • func1
  • func2
  • func3
Kernel32.dll  

  • LoadLibraryA
  • GetProcAddress
  • VirtualProtect
  • VirtualAlloc
  • VirtualFree
mylib.dll  

  • func1
  • func2
  • func3
mylib.dll  

  • func1
xxxx.dll  

  • func1
  • func2
  • func3
xxxx.dll  

  • func1

UPX loader is responsible for filling proper addresses in IAT:

	lea     edi, [esi+10000h]

_next_library:
	mov     eax, [edi]
	or      eax, eax
	jz      short _imports_end
	mov     ebx, [edi+4]
	lea     eax, [eax+esi+121B8h]
	add     ebx, esi
	push    eax
	add     edi, 8
	call    dword ptr [esi+121F4h] ; LoadLibraryA
	xchg    eax, ebp

_next_function:
	mov     al, [edi]
	inc     edi
	or      al, al
	jz      short _next_library
	mov     ecx, edi
	push    edi
	dec     eax
	repne scasb
	push    ebp
	call    dword ptr [esi+121F8h] ; GetProcAddress
	or      eax, eax
	jz      short _gpa_error
	mov     [ebx], eax
	add     ebx, 4
	jmp     short _next_function

_gpa_error:
	popa
	xor     eax, eax
	retn    0Ch

_imports_end:

Let’s back to previous pseudocode:

	HMODULE hPreInst = LoadLibrary("preinstalled.dll");
	if (0 == hPreInst)
	{
		return;
	}
	//LoadCount = 1

	HMODULE hMyLib = LoadLibrary("mylib.dll");    //mylib.dll is packed by UPX now !!!
	/*
		Actions taken behind our back:
		- loading UPX imports table from preinstalled.dll by windows loader,
		  LoadCount = 2
		- loading original imports table from preinstalled.dll by UPX loader,
		  LoadCount = 3
	*/
	if (0 == hMyLib)
	{
		FreeLibrary(hPreInst);
		return;
	}
	//LoadCount = 3 !!!

	//do some stuff here, it doesn't matter what ;)

	FreeLibrary(hMyLib);		//LoadCount = 2
	FreeLibrary(hPreInst);		//LoadCount = 1, library will stay in memory !!!

Now, there is one simple question (or maybe not that simple ?): why don’t use GetModuleHandleA instead of LoadLibraryA ?

Comments (6)

  1. 00:07, June 9, 2011Bartosz Wójcik 

    Because nobody cares :), UPX is like this for years, btw. the similar thing is with the UXTHEME.dll system library, load it once and call a few of its APIs and suddenly you’re going to end up with reference counter set to 3 or smth so you have to call FreeLibrary in the loop to free its memory.

    PS. And UPX (and other packers) keeps static imports because of the TLS handling bug in Windows, TLS index isn’t allocated for DLL libraries that aren’t statically linked to the executable file.

  2. 13:01, June 9, 2011Mario Vilas 

    This is probably NOT the case with UPX, but some crappy packers do the same thing out of sheer ignorance of their authors. Apparently GetModuleHandle is a rather obscure API compared to LoadLibrary, and most people don’t care about keeping DLLs in memory anyway (99% of the time the DLL is supposed to be in memory for the whole run of a program, loading and unloading dynamic libraries in Windows is quite slow).

  3. 18:57, June 9, 2011Peter Ferrie 

    The TLS ‘bug’ is a side-effect of forcing kernel32.dll to be loaded during process startup. Previously, the dynamic loading of kernel32.dll would allow any app to use TLS if it loaded something only from kernel32.dll. Now with kernel32.dll loaded already, TLS callbacks aren’t called in that case. You have to import from another DLL instead (doesn’t matter which one).
    TLS is called for dynamically-loaded DLLs, too (even kernel32.dll or ntdll.dll), on Vista and later, despite what the docs say.
    It’s all in my Anti-Unpacker Tricks papers. :-)

  4. 23:12, June 9, 2011Bartosz Wójcik 

    Dear Peter, I wrote about TLS *index*, NOT about issues with TLS Callbacks! Please read carefully what someone writes next time… It’s all in MS KB http://support.microsoft.com/kb/118816/ :)

  5. 20:30, June 10, 2011Peter Ferrie 

    Bartosz, problems with TLS callbacks or TLS index have the same cause.
    It’s simply not supported for dynamic loading prior to Vista.
    Please read carefully what I wrote – TLS is called for dynamically-loaded DLLs on Vista.
    So on Vista, you’ll see that the index is allocated.

  6. 00:33, June 11, 2011Peter Ferrie 

    I asked Laszlo about it. It’s nothing to do with TLS (and we should have guessed that because UPX didn’t even support TLS for a long time). It’s because they had trouble with import tables that were mostly or completely empty.

Pingbacks (1)