Quick Tips

Enable DEP in Windows® XP SP2 to prevent buffer overflow attack

In Windows® XP SP2, Data Execution Prevention(DEP) is a new Memory Protection Technology implemented to mitigate the buffer overflow exploit in which a virus or other attack tool has injected executable code into a process and then attempted to execute the injected code. DEP is enforced in both hardware and software. Hardware DEP relies on hardware CPU to mark memory with an attribute that indicates that code should not be executed from that memory. Software DEP adds a set of data execution prevention checks to forbid the code execution in heap and buffer segment.

Both hardware DEP and software DEP are configured in same manner. DEP configuration is controlled through Boot.ini switches. The Boot.ini settings related to DEP are as following.

/noexecute=policy_level

where policy_level is defined as AlwaysOn, AlwaysOff, OptIn, or OptOut.

OptIn - This is the default option. With this option, only Windows system binaries are protected by DEP.
OptOut - DEP is enabled by default for all processes. Users can manually create a list of specific applications which do not have DEP applied using System in Control Panel.
AlwaysOn - This provides full DEP coverage for the entire system. All processes always run with DEP applied.
AlwaysOff - This does not provide any DEP coverage for any part of the system, regardless of hardware DEP support.

By default, OptIn is enabled during SP2 installation. This provides protection minimum protection to Windows system binaries. If you want more programs being protected by DEP from buffer overflow attack, then OptOut and AlwaysOn settings need to be considered. OptOut protects all processes running except the processes in exception list. AlwaysOn will protect all processes. One question which most people ask is why we do not always turn on AlwaysOn settings to have all programs being protected. The reason is some vendor's program are incompatible with DEP especially the program generate the dynamic code. If you set AlwaysOn setting, those program will not run successfully.

Both Advanced Micro Devices(TM) (AMD) and Intel® Corporation have defined and shipped Windows-compatible processors that are compatible with DEP. You can check with http://www.amd.com and http://www.intel.com to find out if your computer's CPU has hardware DEP feature.