by ReWolf

Resolving VMware Workstation 10.0.6 crash

Two days ago VMware published new update for Workstation version 10 (release notes). According to the changelog it should fix some security issues reported in OpenSSL. That’s nice, however there is a small problem with this update. VMware.exe (the GUI part of VMmare) started crashing immediately after update. This was quite a learning experience, do not update critical software if you have something important to do, as the new version can be worse than the one you are using. Since I didn’t have the previous installer at hand, I had to somehow resolve this issue differently (yeah sure, I just wanted to debug it and see, why it is crashing).

The crash on my machine is 100% repeatable, and it crashes on security cookie check inside libcurl.dll. As most of you know libcurl is an open source library, so the bug should be hidden somewhere inside libcurl source code, or in the additional/changed code that guys from VMware possibly touched. In the latter case, there should be available modified source code of the libcurl on the VMware page. It is not the case this time, so (in theory) it is fault of the libcurl itself. Crash happens inside the function that seems to enumerate DNS servers. Libcurl can be compiled with many different options and can use many different third party libraries. VMware’s libcurl version is compiled with static version of zlib and c-ares, it is also dynamically linked with OpenSSL. It is worth to note which versions of mentioned libraries are used. Most of open source libraries includes version number inside compiled binaries so it is rather easy to gather those information:

VMWare 10.0.5 VMWare 10.0.6 Current version
libcurl 7.19.5 7.24.0 7.42.1
zlib 1.2.3 1.2.3 1.2.8
c-ares 1.5.1 1.7.5 1.10.0

Particulary interesting for me was libcurl and c-ares, as zlib probably doesn’t enumerate DNSes :) I’ve tried to localize the crashing function inside the source code of mentioned libs, but I was missing one detail (obviously!). In the newest version of c-ares, the function similar to the one that is crashing VMware is named get_DNS_NetworkParams(). This function doesn’t crash and it looks fine. After a few minutes of looking at the code and wondering what’s wrong I realized that I’ve freshly cloned repository with the latest changes and I should probably go back in time and check some older version. Fortunately c-ares repository is properly tagged with release version numbers (at least since version 1.7.0). After checking out proper revision I’ve easily identified crashing function: get_iphlpapi_dns_info(). Simplified code, just to show the problem:

#define INET_ADDRSTRLEN  22
 
int get_iphlpapi_dns_info(char *ret_buf, size_t ret_size)
{
	const size_t  ipv4_size = INET_ADDRSTRLEN + 1;  /* +1 for ',' at end */
	size_t        left = ret_size;
	char         *ret = ret_buf;
	//...
	for (/*...*/)
	{
		//...
		if (left > ipv4_size)
		{
			some_function_that_fills_given_buffer(ret, ipv4_size - 1);
			size_t stringlen = strlen(ret);
			ret[stringlen] = ',';
			ret[stringlen + 1] = '\0';
			ret += stringlen + 1;
			left -= ret - ret_buf;	// <- !!! BUG BUG !!!
		}
		//...
	}
	//...
}
 
//...
	// function call
	char buf[512];
	get_iphlpapi_dns_info(buf,sizeof(buf));
//...

Variable left should be updated on each iteration with the length of the written data, so it will always contain number of bytes left in the ret_buf. As you may already noticed, ret_buf doesn’t change throughout the function, so left is updated with the length of all strings written since the first iteration. It goes below 0 very fast and since it is declared as size_t which is unsigned it will be interpreted as such in the if (left > ipv4_size) comparison and at the end it will overrun ret_buf (only if there is enough records to process by the for loop).

Described bug was present in c-ares library since the very begining (2004-06-10). There were some changes regarding IPv6 (2011-05-17) that were modifying this function, but bug persisted probably due to copy&paste from IPv4 handler. Someone fixed this bug almost a year later (2012-02-25) and the other person rewrote whole code few months later. Unfortunately VMware chose to use version 1.7.5 which was released before this bug was fixed (2011-08-16). Blast from the past.

If you are experiencing crashes in VMware Workstation 10.0.6 just use libcurl.dll from the previous build, at least it will stop crashing.

UPDATE: Someone at VMware forum pointed out that it is sufficient to get libcurl.dll from the \OVFTool\ directory and it will solve the issue. Indeed it is true, as libcurl from mentioned directory uses c-ares version 1.9.1 (libcurl itself is also newer as it has version 7.30.0).

UPDATE 2 (2015-07-10): This issue was fixed with v10.0.7, libcurl was updated to v7.32.0. I’ve also received free copy of VMware Workstation v11. Thanks!:vmware_gift

32 Comments

  1. Hi,
    I’m from the VMware WS team. We’d be happy to take a look at this issue. Could you please provide us more info such as the Windows host version? Could you possibly be able to provide us the Workstation logs, crash dumps etc?
    Thank you.

    Reply

  6. So for windows users, this translates as:
    Copy libcurl.dll from C:\Program Files (x86)\VMware\VMware Workstation\OVFTool\ into C:\Program Files (x86)\VMware\VMware Workstation\

    Thanks for the post!

    Reply

  7. Thanks a lot for solving this problem for me!
    Also thanks Google for pointing me to your blog post, as this issue as of today (of course?) is unknown in the VMware KB!

    Reply

  9. Great article, corrected my issue that started 2 days ago where VMWare Workstation will crash after computer reboot. DLL copy corrected it.
    Thanks.

    Reply

  10. Thanks for the article. I’m literally onsite and had this issue. Copied the file. Fixed. Awesome work.

    Reply

  11. Thanks for your nice article! I’m troubled with VMware workstation 10.0.6 for a few days.
    I Copied libcurl.dll from \OVFTool\ to \VMware\, then this crash was not occurred.

    My event viewer noticed these bellow log when vmware workstation crashed.

    vmware.exe
    10.0.6.60484
    553febfb
    libcurl.dll
    0.0.0.0
    4fb54a82
    c0000409
    0004243b
    1804
    01d095d98cc4ec7c
    G:\vmware\vmware.exe
    G:\vmware\libcurl.dll

    Reply

  17. Thank you for this post. Would be nice if VMWare were have this fixed by now. May 8 they reply saying they will fix. June 16 still not fixed and pushing out the update.

    Reply

  24. Thanks for this, it did help me a few months back.

    Today, I experienced an extended version of this due to an extended power outage which dropped the machine. After going through all the motions of re-re-re-replacing libcurl even though it worked in the past, I was getting the same error.

    Turns out I hadn’t noticed that apparently CMOS battery had died and my clock had reset to some default 2099 date of the bios, and I was getting the exact same dialog indicating the BEX error. I’m assuming the underlying crypto requires accurate time, this was choking something up, but once I fixed the time the BEX fault went away.

    I just wanted to make sure this was logged out in the ether somewhere in case anyone else encounters this, comes here and finds your great analysis, and remains “stuck” after trying it like I was.

    Reply

  27. Having an issue with WS 11, before reboot when moved out of a VM it was no longer possible to focus the window of workstation, had to close it via taskmanager, then restart vmware workstation but then that happend again.

    After a reboot, the vmware.exe is starting, but there’s no window opening at all, the taskmanger shows vmware.exe is running.

    Using v11 on Windows 10 Pro (release).

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *