Some of you probably noticed that HexRays translates rep movsb opcode to memcpy() function from standard C library. In most cases this is perfectly correct behaviour, but there is at least one example when it will not work as it should.
Case 1 (correct):
Two different buffers (not overlapping), the simplest situation, rep movsb can be translated to memcpy() without any troubles.
Case 2 (incorrect):
Behaviour of memcpy() for overlapping regions is undefined (according to C standard library specification), documentaion suggests to use memmove() function which will check for overlapping buffers. Neither memcpy() nor memmove() can be used as a replacement for rep movsb. Below example will show exactly why:
char tab = "qwertyuiopasdfghjklzxcvbnm"; char* ptr_src = tab; char* ptr_dst = tab + 1; size_t sz = 5;
Now we want to execute rep movsb:
mov esi, ptr_src mov edi, ptr_dst mov ecx, sz rep movsb
after execution of this snippet, tab will contain “qqqqqquiopasdfghjklzxcvbnm”. Above situation is very common in various decompression algorithms.
Let’s check what we will achieve with memcpy() function:
memcpy(ptr_dst, ptr_src, sz);
After execution under GCC 4.1.2, tab contains “qqwerruiopasdfghjklzxcvbnm”, which means that compiler optimized memory operation, and at first it copied 4 bytes with movsd, and then 5th byte was copied with movsb. Different compiler may produce different results, because (as it was mentioned earlier) behaviour of memcpy() is undefined for overlapping buffers.
Last check will use memmove() function:
memmove(ptr_dst, ptr_src, sz);
After execution under GCC 4.1.2, tab contains “qqwertuiopasdfghjklzxcvbnm”, which was expected.
Summing up, if you’re trying to decompile some code with HexRays and it still doesn’t work correctly, better check all occurences of memcpy() and if there will be some overlapping buffers, change memcpy() to plain loop (for, while) that will just copy bytes. Of course similar problems might appear for rep movsw and rep movsd instructions.