program

Detect c/c++ memory overflow

Understand the process address space

  • 0 to 0x08048000, keep for OS, not used.
  • 0x08048000 to 0x40000000
    • read only of code and data
    • read and write of global variables
    • heap (malloc and free)
  • 0x40000000
    • mmap allocate memory start from here, grow up
  • 0xC0000000
    • the stack allocate start from here, grow down
  • 0xC0000000 to 0xFFFFFFFF, use for kernel space

Stack and heap

For multiple threads program, OS will allocate a stack for every threads.

  • Stack
    • ESP register point to the top of stack
    • EBP register point to the function action record
    • Parameters be pushed to stack from right to left
    • Return big data structure
    • Stack [from high to low]
      • Parameter N
      • Parameter …
      • Parameter 2
      • Parameter 1
      • EIP (Next instruction address)
      • EBP
      • local variable 1
      • local variable 2
      • local variable …
  • Heap
    • malloc and mmap
      • malloc for small block memory, address bellow 0x40000000, and extend by brk/sbrk
      • When big block memory using mmap, address upper 0x40000000.
    • malloc: put the memory size before the real memory space to user

Failures

  • stack overflow
    • failures
      • changed global variable
      • task not work, function not work
      • some local variable be changed
    • reason
      • thread stack too small
      • defined too large local variable
      • function called too deep
  • global or heap overflow
    • failures
      • global variable be changed
      • memory leak
      • thread-A delete object but thread-B modify object again

Detect tools

Reference