64-bit windows VMware detection - visual-c++

Question

I am trying to develop an application which detects if program is running inside a virtual machine.
For 32-bit Windows, there are already methods explained in the following link:
http://www.codeproject.com/Articles/9823/Detect-if-your-program-is-running-inside-a-Virtual
I am trying to adapt the code regarding Virtual PC and VMware detection in an 64-bit Windows operating system. For VMware, the code can detect successfully in an Windows XP 64-bit OS. But the program crashes when I run it in a native system (Windows 7 64-bit OS).
I put the code in an .asm file and define custom build step with ml64.exe file. The asm code for 64-bit Windows is:
IsInsideVM proc
push rdx
push rcx
push rbx
mov rax, 'VMXh'
mov rbx, 0 ; any value but not the MAGIC VALUE
mov rcx, 10 ; get VMWare version
mov rdx, 'VX' ; port number
in rax, dx ; read port
; on return EAX returns the VERSION
cmp rbx, 'VMXh'; is it a reply from VMWare?
setz al ; set return value
movzx rax,al
pop rbx
pop rcx
pop rdx
ret
IsInsideVM endp
I call this part in a cpp file like:
__try
{
returnValue = IsInsideVM();
}
__except(1)
{
returnValue = false;
}
Thanks in advance.

Answer 1

The old red pill from Joanna may work: random backup page of invisiblethings.org blog:
Swallowing the Red Pill is more or less equivalent to the following code (returns non zero when in Matrix):
int swallow_redpill () {
unsigned char m[2+4], rpill[] = "\x0f\x01\x0d\x00\x00\x00\x00\xc3";
*((unsigned*)&rpill[3]) = (unsigned)m;
((void(*)())&rpill)();
return (m[5]>0xd0) ? 1 : 0;
}
The heart of this code is actually the SIDT instruction (encoded as 0F010D[addr]), which stores the contents of the interrupt descriptor table register (IDTR) in the destination operand, which is actually a memory location. What is special and interesting about SIDT instruction is that, it can be executed in non privileged mode (ring3) but it returns the contents of the sensitive register, used internally by operating system.
Because there is only one IDTR register, but there are at least two OS running concurrently (i.e. the host and the guest OS), VMM needs to relocate the guest's IDTR in a safe place, so that it will not conflict with a host's one. Unfortunately, VMM cannot know if (and when) the process running in guest OS executes SIDT instruction, since it is not privileged (and it doesn't generate exception). Thus the process gets the relocated address of IDT table. It was observed that on VMWare, the relocated address of IDT is at address 0xffXXXXXX, whereas on Virtual PC it is 0xe8XXXXXX. This was tested on VMWare Workstation 4 and Virtual PC 2004, both running on Windows XP host OS.
Note: I haven't tested it myself but look that it uses an unprivileged approach. If it does not work at first for x64, some tweaking may help.
Also, just found out a question with content that may help you: Detecting VMM on linux

Answer 2

My guess is that your function corrups registers.
Running on real hardware (non-VM) should probably trigger exception at "in rax, dx". If this happens then control is passed to your exception handler, which sets result, but does not restore registers. This behaviour will be fully unexpected by caller. For example, it can save something into EBX/RBX register, then call your asm code, your asm code does "mov RBX, 0", it executes, catches exception, sets result, returns - and then caller suddently realizes that his saved data isn't in EBX/RBX anymore! If there was some pointer stored in EBX/RBX - you're going to crash hard. Anything can happen.
Surely, your asm code saves/restores registers, but this happens only when no exception is raised. I.e. if your code is running on VM. Then your code does its normal execution path, no exceptions are raised, registers will be restored normally. But if there is the exception - your POPs will be skipped, because execution will be passed to exception handler.
The correct code should probably do PUSH/POPs outside of try/except block, not inside.