Writing Security Tools and Exploits

Writing shellcode requires an in-depth understanding of the Assembly language for the target architecture in question. Different shellcode is required for each version of each type of operating system in each type of hardware architecture. This is why public exploits tend to exploit vulnerabilities on highly specific target systems, and also why a long list of target versions, operating systems, and hardware is included in the exploit. System calls are used to perform actions within shellcode; therefore, most shellcode is operating system-dependent, because most operating systems use different system calls. Reusing the program code in which the shellcode is injected is possible but difficult. It is recommended that you first write the shellcode in C using only system calls, and then write it in Assembly. This will force you to think about the system calls used, and also facilitates translating the C program.
After an overview of the Assembly programming language, this chapter looks at two common shellcode problems: addressing and Null-byte. It concludes with examples of writing both remote and local shellcode for the 32-bit Intel Architecture (IA32) platform (also referred to as x86). When shellcode is used to take control of a program, it has to be put into the program's memory and then executed, which requires creative thinking (e.g., a single-threaded Web server may have old request data in memory while starting to process a new request. Thus, the shellcode might be embedded with the rest of the payload in the first request, while triggering...