Method of Preventing Buffer Overflow Attacks by Intercepting DLL Functions

The way of intercepting Windows DLL functions against buffer overflow attacks is evaluated. It's produced at the expense of hooking vulnerable DLL functions by addition of check code. If the return address in the stack belongs to a heap or stack page, the call is from illicit code and the program is terminated. The signature of malicious code is recorded, so it is possible for the next attack to be filtered out. The return-into-libc attacks are detected by comparing the entry address of DLL functions with the overwritten return address in the stack. The presented method interrupts the execution of malicious code and prevents the system from being hijacked when these intercepted DLL functions are invoked in the context of buffer overflow.