Calling windows functions
Closed this issue · 12 comments
Hey,
I figure since the hypervisor is loaded using the elf loader and only using the windows driver as an entry to run the loader in the kernel it won't be possible to call to windows functions (since we don't have their addresses).
Is there a way to overcome this issue? since using the os itself is in a lot of use cases (especially in research).
You are correct that at the moment, you cannot execute Windows functions from within the hypervisor. Now, that being said, I think @ionescu007 would agree that most Windows functions really should not be executed in Ring -1 since interrupts are disabled. When we wrote the MoRE hypervisor, we used Windows only as a means to launch the system and tried to keep Windows API calls in Ring -1 to a minimum (if there are any at all).
Now that being said, there are two ways to approach this:
- For a lot of research, you simply need to re-think the approach. For example, DdiMon needs accesses to Windows APIs so that it can hook them, but the reality is, a kernel driver could just as easily tell the hypervisor the address of the function to hook with the address of the hook itself, and Bareflank could be used to perform the hook just the same, without Windows APIs being needed in the hypervisor itself.
- Worst case you could write a "kallsyms_lookup_name" like function that gets the address (or is given by the a kernel driver) to a set of functions that you need, and then executes them from within the hypervisor. All you would need to do is execute the symbol using a function similar to this but in reverse (this function is used to call symbols from within the kernel to our C++ code in system V, so the opposite would be needed).
Yeah... don't ever call Windows functions, even if you figure out a way to. I documented many of these reasons in HyperPlatform's Issue Tracker, but the short story is:
-- You're with interrupts disabled, but CR8 is not modified by the hypervisor. So Windows will still think you're a PASSIVE_LEVEL. Windows APIs will not work the way you expect them to, and will crash
-- You can update CR8/IRQL value to make Windows happy, and now you can't call 90% of APIs (which is a good thing, since the others will crash). The 10% you can call will likely also hang/crash, for the following 3 reasons:
- They might issue IPIs to other processor cores because they have to flush the TLB. You'll hang because you're in VMX root mode in the hypervisor, or hit weird VMExit paths.
- They will check the stack -- a lot of functions do this. They will see that your stack is not a REAL kernel stack (you're on the hypervisor stack). This is seen as a security fault/vulnerability/attack, and you'll crash.
- They will take some time to complete. Time you're spending with all interrupts disabled in VMX root mode.... clocks will drift, if you VMEXIT'ed on an interrupt or DPC you'll hit the watchdog timer (which itself might be running on CPU 0 and you blocked CPU n) and crash the system.
Don't call Windows functions from the Hypervisor. Implement your own or statically link ReactOS if you need shit like Rtl* APIs.
@ionescu007 Ha... never thought to statically link ReactOS code. That should actually work if your looking for some of those functions.
But yeah, at any rate I think it's really a matter of just re-thinking what your trying to do. @mor619dx If you can give us some examples of what you would like to do, we can probably help you out with more details.
So we're looking to do hooking with the hypervisor onto some stuff like processes, dlls memory and we need other windows functions related to processes.
In the hypervisor we are basically only needs addresses, so from the hypervisor work I don't need to run windows functions only the outputs of them. I might need to do driver side which performs the calling to the functions and send the output to the hypervisor.
@mor619dx So there are two options, and it just depends on your use case:
- In most cases, it should be sufficient to write your own driver that performs a VMCall (see here) to tell the hypervisor to hook a function, memory, etc... (the EPT API RFC still needs to be created, which I should get started once we are done with the VMCall code). This is how I would do it.
- If you are worried about the driver, and don't want to protect that separately, you could provide the hypervisor with introspection code to find the symbols that you are looking for. This is a lot more work, but it should be possible. I would take a look at LibVMI as a lot of the work to do this should be there.
Once we have guest support (end of this year, early next year), you could also put LibVMI code into a guest that can introspect Windows (even in the type 2 case if you want).
I agree that driver is the most easy to write and easy to extend way.
My only worry is about how to protect the driver since it's not only making sure it's running and its memory is not corrupted but also about the return values from the windows function that he calls.
For example I need the list of running processes and another driver can easilly edit the process list structure to make it "dissapper".
Although thinking about it, it can happen also with introspection. Which lead me to think that I might just need to protect the parts of windows itself that we using (which is quit a lot).
@mor619dx If your trying to prevent a rootkit from hooking Windows... then yeah this is a different problem and really is the reason Patch Guard exists. Certainly would be an interesting use case for Bareflank as it would be a better way to accomplish the same goals as Patch Guard, and might even be a bit more cross platform.
If your just trying to preserve the integrity of your driver, you can either protect it using the hypervisor, or you can move that logic to its own VM (which is how everyone else does it). Basically, you have an introspection VM that looks into Windows, and gets info on what it's doing.
Short term we would love to have some basic support for LibVMI (@tklengyel) in the host VM which is neat.. but has little advantage over just using a driver. Longer term, we would like to provide support (using the soon to be Hyperkernel) to run something like LibVMI in it's own VM so that it can introspect on the host VM, which is similar to other hypervisors. Probably one of the key differences (besides the availability of C++, and the modularity of Bareflank itself) is that we hope to execute libraries and apps like these using a simple App as a VM model, instead of a full blown OS in a VM like everyone else does (hence the name Hyperkernel). There is a lot of work to get there, but that's our long term goal for the external repo as this would provide the smallest possible footprint for a lot of things, introspection included. We also have plans in the mid-term to support IncludeOS guests (@alfred-bratterud, @RicoAntonioFelix), which could also be used for introspection and provides a nice middle ground between a basic app VM and the usual full OS VM.
Calling Windows functions is actually doable from the hypervisor without staying in ring-1 if you deconstruct how a functioncall is done by the compiler. You have to push your arguments on the kernel stack (including the return address) and just update the stack pointer and instruction pointer to the function you want to call when the target vcpu is in ring0 (like after a mov-to-cr3). If you want to also get the return of the function you just hook the return adress to trap to the hypervisor and and get the results there. You can use the same method for calling userspace functions too, it's just a bit harder to catch the vcpu in usermode. Getting LibVMI to work with Bareflank would certainly be very helpful in this regard ;)
@tklengyel It's not that you can't, it's that you shouldn't. Like @ionescu007 said, if you call a kernel function from the hypervisor, Windows will asssume your at PASSIVE_LEVEL which will likely cause a hidden explosive in your Microsoft certified computer to go off.
It does sound more logical to use driver inside a seperate vm to introspect on the host.
I'm looking for integrity of the driver itself to restrict access to our code and data if the os compromises (similar to windows credential guard) rather than protecting from hooking.
Thanks for the very needed information about this subject!
@mor619dx No problem
If your looking to run your software in the guest to introspect, stay tuned as gust support is on it's way. Getting something as complete as Linux to run in a guest will not be complete until around June next year, but getting basic application support in a guest is something we are looking to do ASAP.
We first have to complete version 1.1, and then we have to complete some of the Extended API work which should not take long, and then it's on to the Hyperkernel repo.
@rianquinn: nope, you setup the stack and vcpu registers while in ring -1 but then do a vmenter. Afterwards you continue in normal kernel context as if a normal functioncall happened. I already do this for functioncall injection for ntdll functions and it is all seemless. See https://github.com/tklengyel/drakvuf/blob/master/src/libinjector/injector.c