Addressing race-condition with multiple CPUs
tklengyel opened this issue · 5 comments
Hi there,
great work! I really like to see more open source implementations that make use of virtualization like this. I've used essentially the same technique in the DRAKVUF analysis system on top of the Xen hypervisor. One issue though that you should keep in mind is that when you are changing the mapping in the EPT to allow the violating CPU to progress while you enable MTF on it, is that the other CPUs are also free to fetch the instruction that should have been breakpointed. In other words, you have a race-condition on the EPT with multiple CPUs. A solution to this that I can recommend is creating multiple EPTs (one for X and one for R/W) so that you can swap the EPT pointer in the VMCS specific to each vCPU instead of changing the mapping in the shared EPT on-the-fly. In Xen this is implemented with the altp2m subsystem.
Cheers,
Tamas
Hi Tamas,
Thank you for reporting the issue and providing your suggestion. As this issue is currently being the top priority, your insight is greatly helpful.
What you pointed out is exactly correct and even commented around the DriverpIsSuppoetedOS() function. To address this issue, I was going to update HyperPlatform with the same way as you suggested (#9).
I was worried about amount of physical memory required to build extra EPT structures as i would take 32+k to cover 1GB of PA for each processor. However, that solution sounds like the best way since you already faced the same issue and solved it with that approach.
Thank you,
Great to see that it was already on the TODO ;) Space could be an issue for the extra EPTs but you could also share portions of the EPTs that are the same. It may be a bit more complex that way but you could save space.
Hi Tamas,
I have implemented per-processor EPT on HyperPlatform about with below commit and updated DdiMon to make use of it (6030ca0 is a stable version after this update).
tandasat/HyperPlatform@c19bdc3
It is different from having two EPTs and swiching them through an EPT pointer, but there should not longer be a race condition since an EPT structure is not shared between processors anymore. I close this issue but feel free to re-open it if you think still there is the issue.
By the way, I happened to find an article mentioning DRAKVUF today. Very nice work!
https://blog.xenproject.org/2016/04/13/stealthy-monitoring-with-xen-altp2m/
Good to hear! Keep up the good work. And thanks for the link but I wrote that post ;)
Aha! I thought it must have been written by someone knew DRAKVUF very well ;)