What is Mhook?
If you dabble in this area then you’ll already know that Microsoft Research's Detours pretty much sets the benchmark when it comes to API hooking. Why don't we get a comparison out of the way quickly then?
DETOURS VS. MHOOK
Detours is available for free with a noncommercial license but it only supports the x86 platform. Detours can also be licensed for commercial use which also gives you full x64 support, but you only get to see the licensing conditions after signing an NDA.
Mhook is freely distributed under an MIT license with support for x86 and x64.
Detours shies away from officially supporting the attachment of hooks to a running application. Of course, you are free to do it - but if you end up causing a random crash here or there, you can only blame yourself.
Mhook was meant to be able to set and remove hooks in running applications – after all, that’s what you need it for in the real world. It does its best to avoid overwriting code that might be under execution by another thread.
Detours supports transactional hooking and unhooking; that is, setting a bunch of hooks at the same time with an all-or-nothing approach. Hooks will only be set if all of them can be set, otherwise the library will roll back any changes made. Mhook does not do this.
Finally, Mhook is pretty lazy when it comes to managing memory for the trampolines it uses. Detours allocates blocks of memory as needed, and uses the resulting data area to store as many trampolines within as will fit. Mhook, on the other hand, uses one call to VirtualAlloc per hook being set. Every hook needs less than 100 bytes of storage so this is very wasteful, since VirtualAlloc ends up grabbing 64K from the process' virtual address space every time Mhook calls it. (Actual allocated memory will be a single page which is also quite wasteful.) In the end though, this probably does not really matter, unless you are setting a very large number of hooks in an application. Also, this is very easy to fix.