EPT Entries
frostiest opened this issue · 6 comments
According to this tutorial https://rayanfam.com/topics/hypervisor-from-scratch-part-8/
Scroll to "An Important Note When Modifying EPT Entries"
One interesting thing that I encountered during the test of my driver on the multi-core system was the fact that EPT entries should be modified in one instruction.
For example, if you change the access bits of an EPT entry, bit by bit, then you probably get the error (EPT Misconfiguration) that one access bits changed and before the next access bit applies another core tries to access page table and it sometimes leads to an EPT Misconfiguration and sometimes you might not get the desired behavior.
so good thing to change that.
I had a question about EPT exits. Does anyone know why something would be reading the page of a hooked kernel function multiple times? I'd assume maybe patchguard but seems way too frequent. Wouldn't it just need to execute it since its function code? Also why would it read it back to back after I already set the page, example
ept violation address:0000000002805000, type:read/write
ept violation address:0000000002805000, type:read/write
ept violation address:0000000002805350, type:exec
shouldn't after the first read/write it should only trigger a EPT exit if something tries to exec(since the page is already set from first read/write)?
You appear to be testing with a customized code so I can only speculate the cause including bugs in code, but I would be curious what is happening.
On the EPT entry modification, this project uses per-core EPT and does not have the issue. Sharing the single EPT for multi-core is not a good design for use cases I ever had. Concurrent access must have been handled without race, EPT cache shoot down (IPI) must have been issued when EPT entry is changed, etc, and those are very hard to test correctness.
You appear to be testing with a customized code so I can only speculate the cause including bugs in code, but I would be curious what is happening.
On the EPT entry modification, this project uses per-core EPT and does not have the issue. Sharing the single EPT for multi-core is not a good design for use cases I ever had. Concurrent access must have been handled without race, EPT cache shoot down (IPI) must have been issued when EPT entry is changed, etc, and those are very hard to test correctness.
Quite right in your assertion. I'm using basically this #26 (comment) to do the lazy exit handling like hvpp as you mentioned in that post and to avoid the extra exit from MTF. I also use Physical address as described at bottom of that thread instead of virtual to locate the page and avoid possible bluescreens from that.
So based on the above is there any obvious issue I need to address or is it still an issue/bug with my code? I'd like to add this wouldn't be a problem however i'm inline hooking quite a number of kernel functions and ones like NtDeviceIoControlFile are spamming EPT exits even for same address and duplicate read/writes as mentioned above so is causing performance issues.
I'm still testing but here's what i've discovered so far. I set VMWARE to 1 CPU and 1 Core. I hook only NtDeviceIoControlFile and unfortunately every time the operating system calls NtSetInformationThread it triggers a read/write fault if the hook page is set to execute only. Since both functions reside in the same 4096 page/byte range and inside NtSetInformationThread it appears to read a variable in the same area
fffff807`49e04338 4c8d05c1bc9fff lea r8,[nt!SeConvertSecurityDescriptorToStringSecurityDescriptor <PERF> (nt+0x0) (fffff807`49800000)] fffff807`49e0433f 410fb6840048526000 movzx eax,byte ptr [r8+rax+605248h] <---triggers read/write exit fffff807`49e04348 418b8c8038526000 mov ecx,dword ptr [r8+rax*4+605238h]
so that kind of shatters my hopes of NtDeviceIoControlFile not triggering so many vm exits by mostly keeping the page on execute only. As far as why its logging duplicate execute or read/write exits i'm not sure yet but i'm
studying https://rayanfam.com/topics/hypervisor-from-scratch-part-8/ "Virtual Processor ID (VPID) & TLB" which you(and subsequently I) already do with UtilInveptGlobal(); but i'll keep testing ^^
so for some reason using UtilInveptGlobal in both the exec and the read/write ept handlers that set the pages would crash me eventually. But after I modified it to have AsmInvept use kSingleContextInvalidation instead of global for ept refresh appears to fix both the crash and duplication issues I had. So thank you for your help like always. ^^
That’s odd and does not make sense to me. But thank you for sharing what you found with us!
That’s odd and does not make sense to me. But thank you for sharing what you found with us!
I feel it's prudent to share my final thoughts in case others have similar issues with similar changes. The only guess I have is perhaps it's a speed issue. I hook a great deal of kernel functions that trigger read/write+execute exits non stop. And that tutorial i've linked mentioned that EPT is slow after each use of clearing the cache globally since it has to refill it. Maybe me changing it to single context made things fast enough to prevent a system freeze and somehow fix the duplicate entries. Which by the way it was always a system freeze never a bluescreen.