public
Expert-Level GDB Mastery Guide
linux/advanced_gdb.md
Concept Flow
Expert GDB Core Dump Analysis Without Symbols
I'll guide you through mastering GDB, building from core concepts to advanced techniques that distinguish expert-level debugging.
Foundation and Compilation
To debug effectively with GDB, compile with debug symbols using -g and consider disabling optimization during development with -O0. For production debugging, -Og provides a reasonable compromise. You can also use -ggdb for GDB-specific enhancements or -g3 to include macro definitions in debug info.
Core Debugging Workflow
Launch GDB with gdb ./program or attach to running processes with gdb -p <pid>. The fundamental cycle involves setting breakpoints, running the program, examining state when execution stops, and continuing or stepping through code.
Breakpoint Mastery
Beyond basic breakpoints with break function or break file.cpp:42, expert usage involves conditional breakpoints: break main if argc > 2 stops only when the condition is true. Temporary breakpoints with tbreak automatically delete after being hit once.
Watchpoints monitor memory changes. Use watch variable to break when a variable changes, rwatch for read access, and awatch for any access. For watching specific memory addresses: watch *(int*)0x7fffffffe000. This is invaluable for tracking down corruption bugs.
Catchpoints trap events like catch throw for C++ exceptions, catch syscall write for system calls, or catch fork for process creation. You can disable, enable, and delete breakpoints by number, and use save breakpoints file.gdb to persist complex breakpoint configurations.
Examining Program State
The print command evaluates expressions: print variable, print *pointer, print array[5]. Use format specifiers like print/x for hex, print/t for binary, print/c for characters. The convenience variable $ holds the last printed value, and $$ the one before that.
The x command examines raw memory with format and size specifiers: x/10x $rsp displays 10 hex words at the stack pointer, x/s 0x400500 shows a string, x/10i $rip disassembles 10 instructions at the program counter.
Use display to automatically show expressions after each stop: display/i $pc shows the next instruction, display argc keeps argc visible. Remove with undisplay.
The info command family is essential: info locals shows local variables, info args displays function arguments, info registers dumps all registers, info threads lists threads, info frame shows the current stack frame details.
Navigation and Execution Control
Step through code with step (into functions), next (over functions), stepi and nexti for instruction-level stepping. The finish command runs until the current function returns, while until continues to a specific line.
The continue command resumes execution until the next breakpoint. You can continue and skip a certain number of hits: continue 5 ignores the next 5 breakpoint hits.
Advanced Stack Frame Manipulation
Use backtrace or bt to see the call stack, bt full to include local variables. Navigate frames with frame N to select a specific frame, up and down to move through the stack. Once in a frame, examine its locals and arguments as if you were at that point in execution.
The return command can force a premature return from a function, optionally with a specified value: return 42. This is powerful for testing error paths or skipping problematic code sections.
Memory and Data Structure Analysis
For complex structures, enable pretty-printing with set print pretty on. Cast and dereference to navigate data structures: print *(struct my_struct*)0x7ffff000. Call functions directly: call malloc(100) or call my_debug_function().
Examine types with ptype variable to see the full type definition including struct members. Use whatis for a quick type summary.
Multi-threaded Debugging
List threads with info threads, switch contexts with thread N. The thread apply all command runs a command on all threads, like thread apply all bt for all backtraces. Control whether other threads run during stepping with set scheduler-locking on (only current thread), off (all threads), or step (only during stepping).
Core Dump Analysis
Generate core dumps with ulimit -c unlimited then reproduce the crash. Load with gdb ./program core. Examine the state at crash time using all the standard inspection commands. This is post-mortem debugging and works identically to live debugging for inspection purposes.
For generating cores programmatically, call gcore <pid> while the process runs, or use generate-core-file from within GDB.
Reverse Execution and Record/Replay
Enable with record or record full. Once recording, use reverse-continue, reverse-step, reverse-next, and reverse-finish to execute backwards. This lets you wind back to see what led to a bug. Use record stop to end recording. Note that recording has performance overhead and memory limits.
Scripting and Automation
GDB uses a Python API for powerful automation. Access with python for single commands or python ... end for blocks:
python
class MyBreakpoint(gdb.Breakpoint):
def stop(self):
frame = gdb.selected_frame()
if frame.read_var('x') > 100:
print("x exceeded threshold!")
return True
return False
MyBreakpoint("function_name")
end
Write GDB command scripts in files with sequences of GDB commands, then load with source script.gdb. Use command-line arguments: gdb -x script.gdb ./program.
Define custom commands with define:
define print_locals_hex
info locals
print/x $arg0
end
Advanced Techniques
Convenience Variables: Create with set $myvar = expression. These persist across stops and can store complex values.
Checkpoints: Use checkpoint to save process state, then restart N to return to that checkpoint. This creates a fork and is useful for exploring different execution paths.
Debugging Optimized Code: Use set substitute-path to map source paths. Debug info may be incomplete, so use disassembly with disassemble and instruction-level stepping.
Remote Debugging: Use target remote host:port to debug processes on remote machines or embedded systems via gdbserver.
TUI Mode: Launch with gdb -tui or toggle with Ctrl-x Ctrl-a for split-screen source and assembly views. Navigate with layout src, layout asm, layout split, layout regs.
Signal Handling: Control with handle SIGNAL action where action is stop, nostop, print, noprint, pass, or nopass. Example: handle SIGPIPE nostop noprint pass.
Expert Workflow Tips
Create a .gdbinit file in your home directory or project directory for automatic initialization. Enable history with set history save on and set history filename ~/.gdb_history. Add set print pretty on, set pagination off for scripting, and custom commands.
Use command abbreviations: b for break, r for run, c for continue, n for next, s for step, p for print, bt for backtrace.
Combine with other tools: valgrind --vgdb=yes --vgdb-error=0 launches your program under Valgrind with GDB control. This combines GDB's inspection with Valgrind's memory checking.
For debugging without source, become proficient with disassembly, register examination (info registers), and understanding calling conventions. Know your architecture's ABI—where arguments go, return value locations, caller vs callee-saved registers.
Common Expert Patterns
Finding segfaults: Run until crash, bt to see the call stack, examine the faulting address and surrounding pointers.
Memory corruption: Set watchpoints on the corrupted memory, run until the watchpoint triggers to catch the culprit.
Race conditions: Use set scheduler-locking step to control thread execution, set thread-specific breakpoints with break file.cpp:42 thread 3.
Heisenbug investigation: Use reverse execution to replay exactly what happened, or use core dumps for deterministic analysis.
This should establish a comprehensive foundation for expert GDB usage. Would you like me to dive deeper into any specific area, such as Python scripting, debugging techniques for particular bug classes, or integration with specific development workflows?
Expert GDB Core Dump Analysis Without Symbols
Debugging stripped binaries and core dumps without symbols is a critical skill. Here's the complete toolkit.
Loading and Initial Analysis
# Load core dump
gdb /path/to/binary core
# Or if binary path is unknown
gdb -c core
# Then: file /path/to/binary
# First commands to run
info registers # See register state at crash
info frame # Current frame (may be garbage without symbols)
info threads # List all threads
x/20i $pc # Disassemble around program counter
Understanding the Crash
Examine Registers
Registers hold critical information:
info registers
# or
info all-registers # Includes floating point/SIMD
# Key registers (x86-64):
# rip/pc - instruction pointer (where crash occurred)
# rsp - stack pointer
# rbp - base pointer (frame pointer)
# rdi, rsi, rdx, rcx, r8, r9 - first 6 function arguments
# rax - return value
Look for anomalies:
- Is
rippointing to invalid memory? (segfault in executable code) - Is
rspaligned properly? (should be 16-byte aligned on x86-64) - Are pointer registers (
rdi,rsi, etc.) valid addresses?
Identify the Faulting Instruction
x/i $pc # Current instruction
x/10i $pc-20 # Instructions before crash
x/20i $pc # Instructions around crash
# Check what memory was being accessed
info registers $rsi # If instruction was using rsi
x/x $rsi # Try to read that address
Common crash patterns:
mov (%rax), %rdxwithrax = 0x0→ null pointer dereferencecall *%raxwith invalidrax→ function pointer corruptionretwith corrupted stack → stack overflow/corruption
Manual Stack Walking
Without symbols, you must manually walk the stack:
# Examine stack memory
x/40gx $rsp # 40 8-byte values in hex
x/100gx $rsp # More context
# Look for return addresses (typically in text segment)
info proc mappings # See memory layout
# or
info files # Show loaded sections
Return addresses on the stack point to locations just after call instructions. They typically fall within the executable's text segment (often 0x400000-0x600000 range for static, 0x555555554000+ for PIE).
Identifying Stack Frames
# Pattern for x86-64 stack frame:
# saved rbp
# return address
# local variables
# ...
# Walk manually
x/2gx $rbp # Saved rbp and return address
x/2gx 0x<saved_rbp_value> # Next frame
# Continue until you reach main or garbage
# Automated attempt (may fail without symbols)
backtrace
# or
bt 50 # Try deeper
Create a manual backtrace:
# At each potential return address
x/5i <return_addr - 10> # Disassemble before return
# Look for 'call' instruction - that's the function that was called
Disassembly Analysis
Understanding Assembly Without Symbols
# Disassemble around PC
disassemble $pc,$pc+100
# Find function boundaries (look for function prologue/epilogue)
x/50i $pc-100
# Prologue typically: push %rbp; mov %rsp,%rbp; sub $X,%rsp
# Epilogue typically: leave; ret or pop %rbp; ret
# Disassemble potential function starts
disassemble <address>
Recognizing Function Signatures
Without symbols, recognize patterns:
Function call setup (x86-64 System V ABI):
mov $0x..., %rdi # 1st argument
mov $0x..., %rsi # 2nd argument
mov $0x..., %rdx # 3rd argument
mov $0x..., %rcx # 4th argument
mov $0x..., %r8 # 5th argument
mov $0x..., %r9 # 6th argument
# Stack arguments for 7+
call <address>
Common library functions have characteristic patterns:
printf/fprintf: format string inrdi/rsimalloc: size inrdi, returns pointer inraxmemcpy: dest inrdi, src inrsi, size inrdxstrlen: string inrdi
Finding Function Boundaries
# Scan backwards from crash point for function start
x/i $pc-4
x/i $pc-8
# ... continue until you see function prologue
# Or use disassemble with estimated range
disassemble $pc-200,$pc+50
# Look for:
push %rbp # Common function start
mov %rsp,%rbp
sub $0x..,%rsp # Allocate stack space
Memory Examination Techniques
Finding Strings and Data
# Search for ASCII strings around crash
x/s <address> # Try to interpret as string
# Search memory for patterns
find <start>, <end>, "string"
find 0x400000, 0x500000, "error"
# Examine data around pointers
x/20wx $rdi # 20 words in hex
x/20s $rsi # Try as strings
# Look for C++ vtables (start with pointer to type_info)
x/10gx <object_pointer> # First value is vtable pointer
x/10gx <vtable_pointer> # Vtable contains function pointers
Interpreting Memory Layout
# Get memory map
info proc mappings
# Typical layout (ASLR enabled):
# 0x555555554000-0x555555556000 .text (code)
# 0x555555756000-0x555555757000 .data (initialized data)
# 0x555555757000-0x555555758000 .bss (uninitialized data)
# 0x7ffff7a0e000-0x7ffff7bce000 libc.so
# 0x7ffffffde000-0x7ffffffff000 [stack]
# Identify regions
x/10gx 0x555555554000 # Code (will show instructions)
x/10gx 0x7fffffffe000 # Stack (will show addresses/data)
Heap Analysis
# Find heap allocations
info proc mappings # Look for [heap]
# Examine heap chunks (glibc malloc)
# Chunk header: prev_size, size (with flags in low 3 bits)
x/4gx <heap_address-16> # Show chunk metadata
# Look for heap corruption patterns
x/100gx <heap_start> # Scan heap
Reconstructing the Call Chain
Manual Stack Trace Construction
# Python script to help extract call chain
gdb -batch -ex "set pagination off" \
-ex "thread apply all bt" \
-ex "quit" \
/path/to/binary core > bt.txt
From within GDB:
# Semi-automated approach
python
import gdb
sp = int(gdb.parse_and_eval("$rsp"))
# Read stack looking for code pointers
for offset in range(0, 400, 8):
addr = sp + offset
try:
val = gdb.parse_and_eval(f"*(unsigned long*){addr}")
val_int = int(val)
# Check if value looks like a code address
if 0x400000 < val_int < 0x800000 or \
0x555555554000 < val_int < 0x555555600000:
# Try to see if it's preceded by a call
gdb.execute(f"x/2i {val_int-7}")
except:
pass
end
Using External Tools
# Get additional info from core
eu-unstrip -n --core=core # Find which libraries were loaded
# Extract executable from core if needed
eu-readelf -n core | grep "NT_FILE"
# Examine with readelf/objdump
readelf -a /path/to/binary
objdump -d /path/to/binary # Full disassembly
objdump -d --start-address=0x<addr> --stop-address=0x<addr> /path/to/binary
# Try to get symbols from separate debug info
apt-get install <package>-dbgsym # Debian/Ubuntu
debuginfo-install <package> # RedHat/Fedora
# Search for debug symbols
find /usr/lib/debug -name "*.debug"
Advanced Pattern Recognition
Identify C++ Crashes
# C++ often crashes in:
# - Virtual function calls (look for indirect calls through vtable)
# - Exception handling (look for _Unwind_* or __cxa_throw)
# - Destructor calls (often called in loops or during unwind)
# Check for C++ runtime functions
x/i $pc
# If you see calls to addresses in libstdc++, check:
info proc mappings | grep libstdc
# C++ object layout
x/20gx <this_pointer> # First is usually vtable pointer
x/10gx *(void**)0x<obj> # Examine vtable
Recognize Common Crash Causes
Null pointer dereference:
# Look for register containing 0 being dereferenced
info registers
x/i $pc
# If instruction is "mov (%rax),%rdx" and rax=0, that's the bug
Use after free:
# Freed memory often filled with patterns
x/20gx <accessed_address>
# Look for: 0xfefefefe (Windows), 0x5a5a5a5a, repeating patterns
Stack overflow:
info proc mappings # Check stack size
# Calculate stack usage
print $rsp - <stack_start>
# If near stack limit, likely overflow
# Look for recursive calls
x/100gx $rsp # Repeating patterns of return addresses
Buffer overflow:
# Look for corrupted return address on stack
x/20gx $rbp
# Return address should be valid code address
# If it's garbage or pattern (0x4141414141...), that's overflow
Signal Information
info signal # What signal caused crash
info program # Why program stopped
# Common signals:
# SIGSEGV (11) - Segmentation fault (bad memory access)
# SIGABRT (6) - Abort (often from assert/abort())
# SIGBUS (7) - Bus error (alignment issue)
# SIGFPE (8) - Floating point exception
# SIGILL (4) - Illegal instruction
Practical Debugging Workflow
Complete Analysis Checklist
# 1. Load and get orientation
file /path/to/binary
core core
info program
info signal
# 2. Examine crash context
info registers
x/20i $pc-20
x/40gx $rsp
# 3. Identify faulting instruction
x/i $pc
# Determine what memory was accessed
# 4. Check register values
info registers
# Are pointers valid?
x/x $rdi
x/x $rsi
x/x $rdx
# 5. Examine memory map
info proc mappings
# Is PC in valid code region?
# Is SP in stack region?
# 6. Attempt backtrace
bt
thread apply all bt
# 7. Manual stack walk if needed
x/100gx $rsp
# Pick out return addresses
# 8. Examine each potential caller
x/10i <return_addr-20>
# Find the call instruction
# 9. Look for data/strings
find 0x400000, 0x800000, "error"
find 0x400000, 0x800000, "assert"
# 10. Check for common patterns
# - Null pointers
# - Stack corruption
# - Heap corruption
# - Use after free
Creating Helper Scripts
Save this as ~/.gdbinit:
# Helper commands for stripped debugging
define xxd
dump binary memory dump.bin $arg0 $arg0+$arg1
shell xxd dump.bin
shell rm dump.bin
end
define find_calls
python
import gdb
frame = gdb.selected_frame()
pc = int(frame.pc())
for i in range(-200, 50, 1):
try:
ins = gdb.execute(f"x/i {pc+i}", to_string=True)
if 'call' in ins:
print(ins)
except:
pass
end
end
define search_stack_returns
python
import gdb
sp = int(gdb.parse_and_eval("$rsp"))
print("Potential return addresses on stack:")
for offset in range(0, 800, 8):
try:
addr = sp + offset
val = int(gdb.parse_and_eval(f"*(unsigned long*){addr}"))
# Check if looks like code address
if (0x400000 < val < 0x800000) or (0x555555554000 < val < 0x600000000):
print(f"[{hex(addr)}] = {hex(val)}")
gdb.execute(f"x/i {val}")
except:
pass
end
end
set pagination off
set print pretty on
Multi-threaded Core Analysis
info threads # List all threads
thread apply all bt # Backtrace of all threads
# Switch to each thread
thread 1
info registers
x/20i $pc
thread 2
info registers
x/20i $pc
# Find thread that crashed
thread apply all info registers
# Look for unusual PC values
Reverse Engineering Techniques
Finding Function Arguments
Since you know the calling convention:
# For x86-64, before function call:
# Arg 1: rdi
# Arg 2: rsi
# Arg 3: rdx
# Arg 4: rcx
# Arg 5: r8
# Arg 6: r9
# Arg 7+: on stack
# Inside function, arguments may be in:
# - Original registers (if not clobbered)
# - Stack at known offsets
# - Local stack slots
# Check function prologue to see where args are saved
x/20i <function_start>
Identifying Variables
# Local variables are at [rbp-offset]
# Find function prologue
x/10i <function_start>
# Note: sub $0x40,%rsp <- 0x40 bytes of locals
# Examine local space
x/16gx $rbp-0x40
This toolkit should enable you to debug virtually any core dump, even completely stripped binaries. The key is understanding assembly, calling conventions, and memory layout. Practice on your own programs first - compile with and without symbols to verify your analysis techniques.
Expert GDB Core Dump Analysis Without Symbols - AArch64
Debugging stripped AArch64 binaries requires understanding ARM64 architecture specifics. Here's the complete guide.
Loading and Initial Analysis
# Load core dump
gdb /path/to/binary core
# First commands for AArch64
info registers # See register state at crash
info all-registers # Include NEON/FP registers
info frame
info threads
x/20i $pc # Disassemble around program counter
AArch64 Register Architecture
Understanding AArch64 Registers
info registers
# General Purpose Registers (64-bit):
# x0-x7 - Function arguments and return values
# x8 - Indirect result location (structs)
# x9-x15 - Temporary registers (caller-saved)
# x16-x17 - Intra-procedure call scratch (IP0, IP1)
# x18 - Platform register (varies by OS)
# x19-x28 - Callee-saved registers
# x29 - Frame pointer (FP)
# x30 - Link register (LR) - stores return address
# sp - Stack pointer
# pc - Program counter
# Also have 32-bit versions: w0-w30 (lower 32 bits of x0-x30)
# NEON/FP registers:
# v0-v31 - 128-bit SIMD/FP registers
# Can be accessed as:
# - q0-q31 (128-bit)
# - d0-d31 (64-bit double)
# - s0-s31 (32-bit float)
# - h0-h31 (16-bit half)
# - b0-b31 (8-bit byte)
Key differences from x86-64:
- Return address stored in
x30(LR), not on stack - Frame pointer is
x29, notrbp - Stack grows downward, but
spmust be 16-byte aligned - No dedicated flags register - conditions stored in PSTATE
Examining Crash State
info registers
# Check critical registers:
# pc - where crash occurred
# sp - stack pointer (must be 16-byte aligned)
# x29 - frame pointer
# x30 - return address (where we came from)
# x0-x7 - function arguments
# Look for anomalies:
print/x $pc # Is this valid code address?
print/x $sp # Is stack aligned? (low 4 bits should be 0)
print/x $x30 # Valid return address?
x/i $pc # What instruction faulted?
x/i $x30 # Where would we return to?
AArch64 Calling Convention (AAPCS64)
Function Arguments
# Arguments 1-8: x0-x7
# Argument 9+: on stack
# Return value: x0 (or x0+x1 for 128-bit)
# Floating point args: v0-v7
# FP return: v0
# Before a function call, examine:
info registers x0 x1 x2 x3 x4 x5 x6 x7
# Common function patterns:
# strlen(char *s) - s in x0
# memcpy(void *d, void *s, size_t n) - d in x0, s in x1, n in x2
# printf(const char *fmt, ...) - fmt in x0, args in x1-x7, then stack
Register Preservation
# Caller-saved (may be clobbered by function):
# x0-x18 (except x16-x17 are special)
# Callee-saved (must be preserved):
# x19-x28, x29 (FP), x30 (LR), sp
# If debugging in middle of function, callee-saved registers
# from caller should still have original values
Understanding the Crash
Identify the Faulting Instruction
x/i $pc # Current instruction
x/10i $pc-40 # Instructions before crash (ARM is 4-bytes/instruction)
x/20i $pc # Instructions around crash
# Common crash patterns on AArch64:
# 1. Null pointer dereference
ldr x1, [x0] # Load from x0, if x0=0 → crash
str x2, [x1, #16] # Store to x1+16, if x1=0 → crash
# 2. Invalid memory access
ldr x0, [x1, #0x1000] # If x1+0x1000 is unmapped
ldrb w0, [x2] # Byte load from invalid x2
# 3. Function pointer call
blr x3 # Branch to address in x3, if invalid → crash
br x4 # Branch without link
# 4. Alignment fault (if strict alignment enabled)
ldr x0, [x1] # If x1 not 8-byte aligned
ldp x0, x1, [x2] # Load pair, needs 8-byte alignment
# Check what memory was being accessed
info registers x0 x1 x2 x3
x/x $x0 # Try to read addresses
x/x $x1
AArch64 Instruction Patterns
# Memory access patterns:
ldr x0, [x1] # Load 64-bit: x0 = *x1
ldr w0, [x1] # Load 32-bit: w0 = *(uint32_t*)x1
ldrb w0, [x1] # Load byte: w0 = *(uint8_t*)x1
ldrh w0, [x1] # Load half: w0 = *(uint16_t*)x1
ldp x0, x1, [x2] # Load pair: x0=*x2, x1=*(x2+8)
ldr x0, [x1, #16] # Load with offset: x0 = *(x1+16)
ldr x0, [x1, x2] # Load with register offset: x0 = *(x1+x2)
str x0, [x1] # Store 64-bit: *x1 = x0
str w0, [x1] # Store 32-bit
strb w0, [x1] # Store byte
strh w0, [x1] # Store half
stp x0, x1, [x2] # Store pair
# Branching:
bl <offset> # Branch with link (call), LR=PC+4
blr x0 # Branch with link to register
b <offset> # Unconditional branch
br x0 # Branch to register
ret # Return (br x30)
ret x5 # Return to x5 instead of x30
# Conditional branches:
cbz x0, <offset> # Compare and branch if zero
cbnz x0, <offset> # Compare and branch if non-zero
b.eq <offset> # Branch if equal (after cmp)
b.ne, b.gt, b.lt, etc.
Manual Stack Walking
AArch64 Stack Frame Layout
# Typical stack frame (with frame pointer):
#
# High addresses
# +------------------+
# | Previous FP | <- x29 points here in previous frame
# | Return Addr (LR) | <- x30 saved here
# +------------------+
# | Saved x19-x28 | <- Callee-saved registers
# | (if used) |
# +------------------+
# | Local variables |
# | and spills |
# +------------------+
# | Outgoing args | <- For calls to other functions
# | (if needed) |
# +------------------+ <- sp (16-byte aligned)
# Low addresses
# Walk the stack using frame pointer
x/2gx $x29 # Show saved FP and LR
# [x29] = previous FP
# [x29+8] = saved LR (return address)
x/2gx *(unsigned long*)$x29 # Next frame
# Continue until you hit garbage or known address
Manual Stack Trace
# Method 1: Using saved frame pointers
set $fp = $x29
while ($fp != 0)
x/2gx $fp
set $fp = *(unsigned long*)$fp
end
# Method 2: Using saved link registers
x/40gx $sp # Look for return addresses
# Return addresses should:
# 1. Be in code segment (check with info proc mappings)
# 2. Point just after a 'bl' or 'blr' instruction
info proc mappings # Find code regions
# Look for addresses in executable regions
Identifying Stack Frames Without Frame Pointer
If compiled with -fomit-frame-pointer:
# Look for saved LR on stack
x/100gx $sp
# Return addresses are typically:
# - In executable memory ranges
# - 4-byte aligned
# - Preceded by 'bl' instruction when you examine them
# For each potential return address:
x/3i <addr-12> # Look for 'bl' before it
Function Recognition
AArch64 Function Prologue/Epilogue
# Standard function prologue (with frame pointer):
stp x29, x30, [sp, #-16]! # Save FP and LR, pre-decrement sp
mov x29, sp # Set up frame pointer
sub sp, sp, #<size> # Allocate stack space
# May also see:
stp x19, x20, [sp, #-16]! # Save callee-saved registers
stp x21, x22, [sp, #-16]!
# Without frame pointer (leaf functions often):
sub sp, sp, #<size> # Just allocate stack
str x30, [sp, #<offset>] # Save LR if needed
# Function epilogue:
ldp x29, x30, [sp], #16 # Restore FP and LR, post-increment sp
ret # Return (br x30)
# Or:
add sp, sp, #<size> # Deallocate stack
ldp x29, x30, [sp], #16
ret
Finding Function Boundaries
# Scan backwards from crash point
x/i $pc-4
x/i $pc-8
# Continue until you see function prologue
# Or disassemble a range
disassemble $pc-200,$pc+50
# Look for prologue pattern:
# stp x29, x30, [sp, #-XX]!
# This is almost always function start
# Find function extent
x/100i <function_start>
# Look for 'ret' instruction for function end
Memory Examination Techniques
Finding Strings and Data
# AArch64 string loading patterns:
adrp x0, <page> # Load page address
add x0, x0, #<offset> # Add page offset
# Result: x0 points to string/data
# Or:
ldr x0, [pc, #<offset>] # PC-relative load
# Examine potential string pointers
x/s $x0
x/s $x1
x/s $x2
# Search for strings
find 0x400000, 0x500000, "error"
find <start>, <end>, "assert"
# Check all argument registers for strings
x/s $x0
x/s $x1
x/s $x2
x/s $x3
x/s $x4
x/s $x5
x/s $x6
x/s $x7
Memory Layout (AArch64 Linux)
info proc mappings
# Typical layout (with ASLR):
# 0x0000005500000000-0x0000005500001000 .text (code) - PIE
# 0x0000005500010000-0x0000005500011000 .data
# 0x0000005500011000-0x0000005500012000 .bss
# 0x0000007f00000000-0x0000007f00200000 libc.so
# 0x0000007ffffff000-0x0000008000000000 [stack]
# Stack grows downward from high addresses
# On AArch64, stack is typically at very high addresses
# Check if address is valid
x/x <address>
# or
find /b <address>, <address>+1, 0xff
Examining Data Structures
# C++ objects (vtable pointer is first)
x/10gx <this_pointer> # First is vtable
x/10gx *(void**)$x0 # If x0 is 'this'
# Arrays
x/20gx $x1 # If x1 points to array
# Structures - manually decode offsets
# struct example {
# uint64_t field1; // offset 0
# uint32_t field2; // offset 8
# char *field3; // offset 16
# };
x/3gx $x0 # Show all fields
print *(uint64_t*)($x0)
print *(uint32_t*)($x0+8)
x/s *(char**)($x0+16)
Disassembly Analysis
Common AArch64 Patterns
# Function calls
bl <function> # Direct call
blr x0 # Indirect call (function pointer)
# Check what was called
x/i $pc
# If: blr x3
info registers x3 # What function?
x/10i $x3 # Disassemble target
# Parameter setup before call
# mov x0, x1 # Set arg1
# mov x1, #0 # Set arg2 = 0
# bl <function>
# Null checks
cbz x0, <error_label> # if (x0 == 0) goto error
cbnz x0, <ok_label> # if (x0 != 0) goto ok
# Comparisons
cmp x0, #0 # Compare x0 with 0
b.eq <label> # Branch if equal
cmp x1, x2 # Compare registers
b.ne <label> # Branch if not equal
# Loop patterns
.loop:
ldr x1, [x0], #8 # Load and post-increment
cbz x1, .done
bl <process>
b .loop
.done:
Virtual Function Calls (C++)
# Pattern for virtual call:
ldr x0, [x19] # Load 'this' pointer
ldr x8, [x0] # Load vtable pointer
ldr x8, [x8, #24] # Load function pointer from vtable
blr x8 # Call virtual function
# Examine vtable
x/10gx *(void**)$x19 # Show vtable entries
x/10i <vtable_entry> # Disassemble virtual function
Advanced Debugging Techniques
Reconstructing the Call Chain
# Method 1: Follow saved LR values
set $current_lr = $x30
x/10i $current_lr-4 # Should show 'bl' instruction
# Walk frames
set $fp = $x29
printf "Frame 0: LR=%p\n", $x30
set $i = 1
while ($fp != 0 && $i < 20)
set $saved_lr = *(unsigned long*)($fp + 8)
printf "Frame %d: FP=%p LR=%p\n", $i, $fp, $saved_lr
x/2i $saved_lr-8
set $fp = *(unsigned long*)$fp
set $i = $i + 1
end
# Method 2: Search stack for return addresses
python
import gdb
sp = int(gdb.parse_and_eval("$sp"))
print("Potential return addresses on stack:")
for offset in range(0, 800, 8):
try:
addr = sp + offset
val = int(gdb.parse_and_eval(f"*(unsigned long*){addr}"))
# Check if looks like code address (adjust ranges for your binary)
if (0x5500000000 < val < 0x5600000000) or \
(0x7f00000000 < val < 0x8000000000):
# Check if preceded by bl/blr
try:
insn = gdb.execute(f"x/i {val-4}", to_string=True)
if 'bl' in insn:
print(f"[{hex(addr)}] = {hex(val)}")
print(insn)
except:
pass
except:
pass
end
Signal Information
info signal
info program
# Common signals on AArch64:
# SIGSEGV (11) - Invalid memory reference
# - si_addr shows the faulting address
# SIGBUS (7) - Bus error (alignment on strict-align systems)
# SIGILL (4) - Illegal instruction
# SIGFPE (8) - Arithmetic exception
# SIGTRAP (5) - Trace/breakpoint trap
# Get more signal details
$_siginfo
# or
print $_siginfo
Crash Pattern Recognition
Null Pointer Dereference:
# Look for x0 (or other register) being 0
info registers
x/i $pc
# Example:
# ldr x1, [x0]
# If x0 = 0x0000000000000000 → crash here
# Check all registers that might be dereferenced
info registers | grep "0x0"
Stack Overflow:
info proc mappings # Find stack region
# Stack is typically: 0x007ffffff000-0x008000000000
# Calculate stack usage
print/x $sp
# If very close to stack start → overflow
# Look for deep recursion
x/200gx $sp
# Repeating patterns of same return addresses indicate recursion
Use After Free:
# Check for poison patterns in freed memory
x/20gx <accessed_address>
# Common patterns:
# 0xfefefefefefefefe (debug malloc)
# 0xdeadbeefdeadbeef
# 0x5a5a5a5a5a5a5a5a
# Also check if address is in heap region
info proc mappings | grep heap
Buffer Overflow:
# Check saved LR for corruption
x/2gx $x29
# Saved LR at [x29+8] should be valid code address
# If it's garbage or pattern (0x41414141...) → overflow
# Check stack for patterns
x/100gx $sp
Alignment Fault:
# AArch64 usually allows unaligned access, but:
# - Can be disabled by kernel
# - Atomic operations require alignment
# - LDP/STP require 8-byte alignment in some cases
info registers
x/i $pc
# Check if address is misaligned
# ldp x0, x1, [x2]
# If x2 = 0x12345 (not 8-byte aligned) → may crash
print/x $x2 & 7
# Should be 0 for 8-byte operations
Practical Debugging Workflow
Complete Analysis Checklist
# 1. Load core dump
file /path/to/aarch64-binary
core-file core
# 2. Basic orientation
info program
info signal
info threads
# 3. Examine crash state
info registers
info all-registers
# 4. Analyze the crash
x/i $pc # Faulting instruction
x/10i $pc-40 # Context before
x/10i $pc # Context after
# 5. Check register contents
# Especially argument registers and pointers
x/x $x0
x/x $x1
x/x $x2
# Check for null, invalid addresses
# 6. Examine return address
x/i $x30 # Where we came from
x/5i $x30-20 # Show the call site
# 7. Stack examination
x/2gx $x29 # Saved FP and LR
x/40gx $sp # Stack contents
# 8. Memory mapping
info proc mappings
# Verify PC, SP, LR are in valid regions
# 9. Attempt backtrace
bt
bt full
thread apply all bt
# 10. Manual stack walk if needed
# (use scripts from earlier section)
# 11. Look for strings/data
x/s $x0
x/s $x1
find 0x5500000000, 0x5600000000, "error"
# 12. Multi-threaded analysis
info threads
thread apply all bt
thread apply all info registers
Helper GDB Script for AArch64
Save as ~/.gdbinit-aarch64:
# AArch64 debugging helpers
define arm64-regs
printf "x0: %016lx x1: %016lx\n", $x0, $x1
printf "x2: %016lx x3: %016lx\n", $x2, $x3
printf "x4: %016lx x5: %016lx\n", $x4, $x5
printf "x6: %016lx x7: %016lx\n", $x6, $x7
printf "x8: %016lx x29: %016lx (FP)\n", $x8, $x29
printf "x30: %016lx (LR) sp: %016lx\n", $x30, $sp
printf "pc: %016lx\n", $pc
end
define arm64-args
printf "arg0 (x0): "
x/gx $x0
printf "arg1 (x1): "
x/gx $x1
printf "arg2 (x2): "
x/gx $x2
printf "arg3 (x3): "
x/gx $x3
printf "arg4 (x4): "
x/gx $x4
printf "arg5 (x5): "
x/gx $x5
printf "arg6 (x6): "
x/gx $x6
printf "arg7 (x7): "
x/gx $x7
end
define arm64-frame-walk
python
import gdb
fp = int(gdb.parse_and_eval("$x29"))
frame_num = 0
print("Manual stack frame walk:")
print("=" * 60)
while fp != 0 and frame_num < 30:
try:
saved_fp = int(gdb.parse_and_eval(f"*(unsigned long*){fp}"))
saved_lr = int(gdb.parse_and_eval(f"*(unsigned long*)({fp}+8)"))
print(f"\nFrame {frame_num}:")
print(f" FP: {hex(fp)}")
print(f" Saved LR: {hex(saved_lr)}")
# Try to show the call site
try:
insn = gdb.execute(f"x/2i {saved_lr-8}", to_string=True)
print(f" Call site:\n{insn}", end='')
except:
pass
# Sanity check
if saved_fp < fp or saved_fp > fp + 0x100000:
print(" (Frame pointer looks invalid, stopping)")
break
fp = saved_fp
frame_num += 1
except:
print(f" Error reading frame {frame_num}")
break
print("\n" + "=" * 60)
end
end
define arm64-stack-scan
python
import gdb
sp = int(gdb.parse_and_eval("$sp"))
print("Scanning stack for potential return addresses:")
print("=" * 60)
for offset in range(0, 1024, 8):
try:
addr = sp + offset
val = int(gdb.parse_and_eval(f"*(unsigned long*){addr}"))
# Check if looks like code (adjust for your binary)
# Typical ranges: 0x5500000000-0x5600000000 for PIE
# 0x7f00000000-0x8000000000 for libs
if ((0x5500000000 < val < 0x5600000000) or
(0x7f00000000 < val < 0x8000000000)):
# Check if 4-byte aligned (all ARM64 instructions are)
if val & 3 == 0:
try:
# Look for 'bl' before this address
insn = gdb.execute(f"x/2i {val-8}", to_string=True)
if 'bl' in insn:
print(f"\n[sp+{hex(offset)}] = {hex(val)}")
print(insn, end='')
except:
pass
except:
pass
print("\n" + "=" * 60)
end
end
define arm64-check-null-deref
python
import gdb
pc = int(gdb.parse_and_eval("$pc"))
# Get the instruction
insn = gdb.execute(f"x/i {pc}", to_string=True)
print("Checking for null pointer dereference:")
print("=" * 60)
print(f"Faulting instruction:\n{insn}")
# Check if it's a memory access instruction
mem_ops = ['ldr', 'str', 'ldp', 'stp', 'ldrb', 'strb', 'ldrh', 'strh']
for op in mem_ops:
if op in insn.lower():
print(f"\nFound memory operation: {op}")
# Check common pointer registers
for reg in ['x0', 'x1', 'x2', 'x3', 'x4', 'x5']:
try:
val = int(gdb.parse_and_eval(f"${reg}"))
status = "NULL" if val == 0 else "INVALID" if val < 0x1000 else "OK"
print(f" {reg}: {hex(val)} [{status}]")
except:
pass
break
print("=" * 60)
end
end
define arm64-crash-summary
printf "\n"
printf "=== AArch64 Crash Summary ===\n"
printf "\n"
printf "Signal: "
info signal
printf "\nCrash Location:\n"
x/i $pc
printf "\nRegister State:\n"
arm64-regs
printf "\nStack Frame:\n"
x/2gx $x29
printf "\nPotential Return Path:\n"
x/3i $x30-8
printf "\nStack Top:\n"
x/8gx $sp
printf "\n"
end
set pagination off
set print pretty on
printf "AArch64 debugging helpers loaded.\n"
printf "Commands: arm64-regs, arm64-args, arm64-frame-walk,\n"
printf " arm64-stack-scan, arm64-check-null-deref,\n"
printf " arm64-crash-summary\n"
Load with:
gdb -x ~/.gdbinit-aarch64 /path/to/binary core
Quick Reference
First Steps:
arm64-crash-summary # Quick overview
info threads # Check all threads
thread apply all bt # All backtraces
Manual Analysis:
arm64-check-null-deref # Check for null pointer
arm64-frame-walk # Walk call stack
arm64-stack-scan # Find return addresses
Memory Investigation:
x/20i $pc-40 # Context around crash
x/s $x0 # Check string args
x/40gx $sp # Stack dump
info proc mappings # Memory layout
AArch64-Specific Considerations
Position Independent Executable (PIE)
# AArch64 heavily uses ADRP for PIC
adrp x0, <page> # Get page address
add x0, x0, #<pageoff> # Add page offset
ldr x1, [x0] # Load from computed address
# To find what data/string this references:
# 1. Execute the adrp+add mentally or in calculator
# 2. Or step through and examine x0 after
NEON/SIMD Crashes
# If crash is in SIMD code
info all-registers # Show v0-v31
info vector # SIMD register state
# SIMD load/store
ld1 {v0.16b}, [x0] # Load 16 bytes to v0
st1 {v1.8h}, [x1] # Store 8 halfwords from v1
# Check alignment for SIMD operations
print $x0 & 15 # Should be 0 for 128-bit ops
Thread Local Storage (TLS)
# AArch64 uses special register for TLS
# tpidr_el0 on Linux (not directly accessible in GDB)
# TLS access pattern:
mrs x0, tpidr_el0 # Read thread pointer
ldr x1, [x0, #<offset>] # Access TLS variable
# In core dump, TLS data may be visible in thread-specific stack
This guide should equip you to debug any AArch64 core dump without symbols. The key differences from x86-64 are the calling convention, register layout, and instruction encoding, but the fundamental principles remain the same: understand the architecture, read assembly, and manually reconstruct program state.
Expert Guide to GDB Scripting: Python & Internal Commands
Part 1: GDB Internal Scripting
Basic Command Scripts
GDB has its own command language for creating custom commands and automating tasks.
Defining Custom Commands
# Basic command definition
define mycommand
# Commands go here
info registers
backtrace
end
# Command with documentation
define mycommand
document mycommand
Custom command that shows registers and backtrace.
Usage: mycommand
end
info registers
backtrace
end
# Command with arguments
define print_array
set $i = 0
while $i < $arg0
print array[$i]
set $i = $i + 1
end
end
# Usage: print_array 10
Arguments in Commands
# $arg0, $arg1, etc. are the arguments
define dump_memory
dump binary memory $arg0 $arg1 $arg2
end
# Usage: dump_memory output.bin 0x400000 0x401000
# With documentation
define dump_memory
document dump_memory
Dump memory region to file.
Usage: dump_memory <filename> <start_addr> <end_addr>
end
if $argc != 3
printf "Usage: dump_memory <filename> <start_addr> <end_addr>\n"
else
dump binary memory $arg0 $arg1 $arg2
printf "Dumped memory from %p to %p into %s\n", $arg1, $arg2, $arg0
end
end
Control Flow
# Conditional execution
define check_null
if $arg0 == 0
printf "Pointer is NULL!\n"
else
printf "Pointer: %p\n", $arg0
x/x $arg0
end
end
# While loops
define print_list
set $node = $arg0
while $node != 0
printf "Node: %p, data: %d\n", $node, $node->data
set $node = $node->next
end
end
# Loop with counter
define hexdump
set $addr = $arg0
set $count = $arg1
set $i = 0
while $i < $count
printf "%p: ", $addr + $i
x/x $addr + $i
set $i = $i + 1
end
end
Convenience Variables
# User-defined convenience variables start with $
define save_context
set $saved_rip = $rip
set $saved_rsp = $rsp
set $saved_rbp = $rbp
printf "Context saved\n"
end
define restore_context
set $rip = $saved_rip
set $rsp = $saved_rsp
set $rbp = $saved_rbp
printf "Context restored\n"
end
# Persistent variables across debugging sessions
set $myvar = 42
show convenience # Show all convenience variables
Hooks
# Hooks run before/after commands
define hook-stop
# Runs every time execution stops
printf "\n=== Stopped ===\n"
x/i $pc
printf "===============\n"
end
define hook-next
# Runs before 'next' command
printf "About to execute next...\n"
end
define hookpost-next
# Runs after 'next' command
printf "Completed next\n"
info registers rax rbx rcx
end
# Hook for specific breakpoints
define hook-break
# Runs before setting breakpoint
printf "Setting breakpoint...\n"
end
Output and Formatting
define show_state
printf "PC: 0x%016lx\n", $pc
printf "SP: 0x%016lx\n", $sp
printf "Frame: "
info frame
# Output redirection
set logging file debug.log
set logging on
backtrace full
set logging off
end
# Formatted output
define print_struct
printf "struct {\n"
printf " field1: %d\n", $arg0->field1
printf " field2: 0x%x\n", $arg0->field2
printf " field3: %s\n", $arg0->field3
printf "}\n"
end
# Conditional output
define verbose_print
if $verbose_mode
printf "Debug: value = %d\n", $arg0
end
end
Advanced Internal Scripting
# Error handling
define safe_deref
# GDB doesn't have try-catch, but we can check
set $valid = 1
# Check if address looks valid
if $arg0 < 0x1000
printf "Invalid pointer: %p\n", $arg0
set $valid = 0
end
if $valid
x/x $arg0
end
end
# Function-like behavior
define calculate
set $result = $arg0 + $arg1 * $arg2
# Return via convenience variable
set $retval = $result
end
# Call it
calculate 10 20 30
printf "Result: %d\n", $retval
# Complex data structure traversal
define walk_binary_tree
set $node = $arg0
if $node != 0
# Pre-order traversal
printf "Value: %d\n", $node->value
walk_binary_tree $node->left
walk_binary_tree $node->right
end
end
# Breakpoint management
define set_conditional_break
break $arg0 if $arg1 > $arg2
printf "Conditional breakpoint set at %s\n", $arg0
end
Command Files
# Save to file: startup.gdb
set pagination off
set print pretty on
set history save on
set history filename ~/.gdb_history
define init
printf "Custom GDB environment loaded\n"
source ~/project/.gdbinit
end
# Auto-run on startup
init
# Load with: gdb -x startup.gdb
Part 2: Python Scripting in GDB
Basic Python Interface
# Single-line Python
(gdb) python print("Hello from Python")
# Multi-line Python
(gdb) python
>import sys
>print(f"Python version: {sys.version}")
>for i in range(5):
> print(f"Count: {i}")
>end
# Or in script files
python
import gdb
class MyCommand(gdb.Command):
"""My custom command"""
def __init__(self):
super().__init__("mypy", gdb.COMMAND_USER)
def invoke(self, args, from_tty):
print("Hello from Python command!")
MyCommand()
end
GDB Python API - Core Concepts
Accessing Inferior (Debugged Program)
python
import gdb
# Get the current inferior
inferior = gdb.selected_inferior()
# Read memory
address = 0x400000
size = 100
memory = inferior.read_memory(address, size)
# Returns memoryview object
data = bytes(memory)
print(f"First byte: {data[0]:02x}")
# Write memory
new_data = b'\x90\x90\x90\x90' # NOP instructions
inferior.write_memory(0x400500, new_data)
# Check if program is running
if inferior.is_valid():
print("Inferior is valid")
# Get all inferiors (for multi-process debugging)
for inf in gdb.inferiors():
print(f"Inferior {inf.num}: PID={inf.pid}")
end
Working with Values
python
import gdb
# Parse and evaluate expressions
value = gdb.parse_and_eval("my_variable")
ptr = gdb.parse_and_eval("my_pointer")
expr = gdb.parse_and_eval("x + y * 2")
# Get value as Python types
int_val = int(value)
str_val = str(value)
float_val = float(value)
# Dereference pointers
dereferenced = ptr.dereference()
# Access struct members
struct_val = gdb.parse_and_eval("my_struct")
field = struct_val['field_name']
# Cast values
casted = value.cast(gdb.lookup_type("int *"))
# Array access
array = gdb.parse_and_eval("my_array")
element = array[5]
# Value attributes
print(f"Type: {value.type}")
print(f"Address: {value.address}")
print(f"Is optimized out: {value.is_optimized_out}")
# Arithmetic operations
result = value + 10
result = ptr + 4 # Pointer arithmetic
end
Type System
python
import gdb
# Lookup types
int_type = gdb.lookup_type("int")
struct_type = gdb.lookup_type("struct my_struct")
ptr_type = gdb.lookup_type("char").pointer()
# Type properties
print(f"Type name: {struct_type.name}")
print(f"Type code: {struct_type.code}") # TYPE_CODE_STRUCT, etc.
print(f"Type size: {struct_type.sizeof}")
# Type codes
# gdb.TYPE_CODE_PTR - pointer
# gdb.TYPE_CODE_ARRAY - array
# gdb.TYPE_CODE_STRUCT - struct
# gdb.TYPE_CODE_UNION - union
# gdb.TYPE_CODE_ENUM - enum
# gdb.TYPE_CODE_FUNC - function
# gdb.TYPE_CODE_INT - integer
# gdb.TYPE_CODE_FLT - float
# Navigate type properties
if struct_type.code == gdb.TYPE_CODE_STRUCT:
for field in struct_type.fields():
print(f"Field: {field.name}, offset: {field.bitpos//8}")
# Pointer type operations
base_type = ptr_type.target() # Get pointed-to type
array_type = int_type.array(10) # Create array type
# Type comparison
if value.type == int_type:
print("Value is int")
# Type casting
value_as_ptr = gdb.parse_and_eval("&my_var")
as_long_ptr = value_as_ptr.cast(gdb.lookup_type("long").pointer())
end
Frames and Backtraces
python
import gdb
# Get current frame
frame = gdb.selected_frame()
# Frame information
print(f"Frame name: {frame.name()}")
print(f"PC: {hex(frame.pc())}")
print(f"Frame level: {frame.level()}")
# Get older/newer frames
older = frame.older()
newer = frame.newer()
# Navigate all frames
frame = gdb.newest_frame()
while frame:
print(f"Level {frame.level()}: {frame.name()} at {hex(frame.pc())}")
frame = frame.older()
# Read variables from frame
try:
var = frame.read_var("local_variable")
print(f"Variable value: {var}")
except ValueError as e:
print(f"Variable not found: {e}")
# Read registers in frame
rip = frame.read_register("rip")
rsp = frame.read_register("rsp")
# Get all registers
arch = frame.architecture()
for reg in arch.registers():
try:
value = frame.read_register(reg)
print(f"{reg.name}: {value}")
except:
pass
# Select frame
gdb.execute("frame 2") # Switch to frame 2
# Block and symbol information
block = frame.block()
if block:
print(f"Block start: {hex(block.start)}")
print(f"Block end: {hex(block.end)}")
# Iterate symbols in block
for symbol in block:
print(f"Symbol: {symbol.name}")
end
Breakpoints
python
import gdb
# Create simple breakpoint
bp = gdb.Breakpoint("main")
bp = gdb.Breakpoint("file.cpp:42")
bp = gdb.Breakpoint("*0x400500") # Address breakpoint
# Breakpoint properties
bp.enabled = True
bp.silent = True # Don't print when hit
bp.ignore_count = 5 # Ignore first 5 hits
bp.condition = "x > 100" # Conditional breakpoint
# Watchpoints
wp = gdb.Breakpoint("my_variable", gdb.BP_WATCHPOINT)
rwp = gdb.Breakpoint("my_variable", gdb.BP_READ_WATCHPOINT)
awp = gdb.Breakpoint("my_variable", gdb.BP_ACCESS_WATCHPOINT)
# Custom breakpoint with callback
class MyBreakpoint(gdb.Breakpoint):
def __init__(self, spec):
super().__init__(spec)
self.hit_count = 0
def stop(self):
self.hit_count += 1
frame = gdb.selected_frame()
# Access variables
try:
x = frame.read_var("x")
print(f"Hit #{self.hit_count}: x = {x}")
# Conditional stop
if int(x) > 100:
print("x exceeded 100, stopping")
return True # Stop execution
else:
return False # Continue execution
except:
return True
# Create custom breakpoint
mybp = MyBreakpoint("function_name")
# Breakpoint with commands
class CommandBreakpoint(gdb.Breakpoint):
def stop(self):
gdb.execute("info locals")
gdb.execute("backtrace 3")
return True
# Temporary breakpoint
temp_bp = gdb.Breakpoint("init", temporary=True)
# Get all breakpoints
for bp in gdb.breakpoints():
print(f"Breakpoint {bp.number}: {bp.location}")
bp.delete() # Delete breakpoint
end
Threads
python
import gdb
# Get all threads
for thread in gdb.selected_inferior().threads():
print(f"Thread {thread.num}: {thread.name}")
print(f" PTID: {thread.ptid}")
print(f" Is stopped: {thread.is_stopped()}")
print(f" Is running: {thread.is_running()}")
# Switch thread
thread = gdb.selected_inferior().threads()[0]
thread.switch()
# Or use thread number
gdb.execute("thread 2")
# Get selected thread
current = gdb.selected_thread()
if current:
print(f"Current thread: {current.num}")
# Get thread-specific inferior
inferior = current.inferior()
# Iterate threads and get backtraces
for thread in gdb.selected_inferior().threads():
thread.switch()
print(f"\n=== Thread {thread.num} ===")
frame = gdb.newest_frame()
while frame:
print(f" {frame.name()}")
frame = frame.older()
end
Advanced Python Scripting
Custom Commands
python
import gdb
class DumpStructCommand(gdb.Command):
"""Dump a C structure with formatting.
Usage: dump-struct <expression>
"""
def __init__(self):
super().__init__("dump-struct", gdb.COMMAND_DATA)
def invoke(self, args, from_tty):
try:
value = gdb.parse_and_eval(args)
self.dump_value(value, 0)
except Exception as e:
print(f"Error: {e}")
def dump_value(self, value, indent):
indent_str = " " * indent
vtype = value.type
# Strip typedefs and references
while vtype.code == gdb.TYPE_CODE_TYPEDEF or \
vtype.code == gdb.TYPE_CODE_REF:
vtype = vtype.target()
if vtype.code == gdb.TYPE_CODE_STRUCT:
print(f"{indent_str}{{")
for field in vtype.fields():
if field.is_base_class:
continue
try:
field_val = value[field.name]
print(f"{indent_str} {field.name}: ", end="")
self.dump_value(field_val, indent + 1)
except:
print(f"{indent_str} {field.name}: <error>")
print(f"{indent_str}}}")
elif vtype.code == gdb.TYPE_CODE_PTR:
try:
addr = int(value)
print(f"0x{addr:x}", end="")
if addr != 0:
deref = value.dereference()
print(" -> ", end="")
self.dump_value(deref, indent)
else:
print(" (NULL)")
except:
print("<invalid pointer>")
elif vtype.code == gdb.TYPE_CODE_ARRAY:
length = vtype.range()[1] - vtype.range()[0] + 1
print(f"[{length}] {{")
for i in range(min(length, 10)): # Limit to 10 elements
print(f"{indent_str} [{i}]: ", end="")
self.dump_value(value[i], indent + 1)
if length > 10:
print(f"{indent_str} ... ({length - 10} more)")
print(f"{indent_str}}}")
else:
print(value)
def complete(self, text, word):
# Tab completion
return gdb.COMPLETE_SYMBOL
DumpStructCommand()
end
Pretty Printers
python
import gdb
import gdb.printing
class VectorPrinter:
"""Pretty printer for std::vector"""
def __init__(self, val):
self.val = val
def to_string(self):
start = self.val['_M_impl']['_M_start']
finish = self.val['_M_impl']['_M_finish']
end_storage = self.val['_M_impl']['_M_end_of_storage']
size = int(finish - start)
capacity = int(end_storage - start)
return f"std::vector of length {size}, capacity {capacity}"
def children(self):
start = self.val['_M_impl']['_M_start']
finish = self.val['_M_impl']['_M_finish']
size = int(finish - start)
for i in range(size):
element = (start + i).dereference()
yield (f"[{i}]", element)
def display_hint(self):
return 'array'
class LinkedListPrinter:
"""Pretty printer for custom linked list"""
def __init__(self, val):
self.val = val
def to_string(self):
count = 0
node = self.val['head']
while node != 0:
count += 1
node = node['next']
if count > 1000: # Prevent infinite loops
break
return f"LinkedList with {count} nodes"
def children(self):
node = self.val['head']
index = 0
while node != 0:
yield (f"[{index}]", node['data'])
node = node['next']
index += 1
if index > 100: # Limit display
break
# Register pretty printers
def build_pretty_printer():
pp = gdb.printing.RegexpCollectionPrettyPrinter("my_printers")
# Register by type name (regex)
pp.add_printer('vector', '^std::vector<.*>$', VectorPrinter)
pp.add_printer('list', '^LinkedList$', LinkedListPrinter)
return pp
gdb.printing.register_pretty_printer(
gdb.current_objfile(),
build_pretty_printer(),
replace=True
)
# Or register globally
# gdb.printing.register_pretty_printer(None, build_pretty_printer())
end
Memory Analysis
python
import gdb
import struct
class MemoryAnalyzer(gdb.Command):
"""Analyze memory regions.
Usage: memanalyze <start> <size>
"""
def __init__(self):
super().__init__("memanalyze", gdb.COMMAND_DATA)
def invoke(self, args, from_tty):
args_list = gdb.string_to_argv(args)
if len(args_list) != 2:
print("Usage: memanalyze <start_address> <size>")
return
start = int(gdb.parse_and_eval(args_list[0]))
size = int(args_list[1])
# Read memory
inferior = gdb.selected_inferior()
try:
memory = inferior.read_memory(start, size)
data = bytes(memory)
except gdb.MemoryError as e:
print(f"Memory read error: {e}")
return
self.analyze(data, start)
def analyze(self, data, start_addr):
print(f"Memory analysis from 0x{start_addr:x}:")
print(f" Size: {len(data)} bytes")
# Check for patterns
null_bytes = data.count(0)
print(f" NULL bytes: {null_bytes} ({null_bytes*100//len(data)}%)")
# Look for ASCII strings
strings = self.find_strings(data, start_addr)
if strings:
print(f" Found {len(strings)} strings:")
for addr, s in strings[:5]: # Show first 5
print(f" 0x{addr:x}: {s}")
# Look for pointers (assuming 64-bit)
pointers = self.find_pointers(data, start_addr)
if pointers:
print(f" Found {len(pointers)} potential pointers")
# Entropy check (simple)
entropy = self.calculate_entropy(data)
print(f" Entropy: {entropy:.2f}")
def find_strings(self, data, base_addr, min_length=4):
strings = []
current = []
current_start = 0
for i, byte in enumerate(data):
if 32 <= byte < 127: # Printable ASCII
if not current:
current_start = i
current.append(chr(byte))
else:
if len(current) >= min_length:
s = ''.join(current)
strings.append((base_addr + current_start, s))
current = []
if len(current) >= min_length:
s = ''.join(current)
strings.append((base_addr + current_start, s))
return strings
def find_pointers(self, data, base_addr):
pointers = []
# Unpack as 64-bit values
for i in range(0, len(data) - 7, 8):
value = struct.unpack('<Q', data[i:i+8])[0]
# Check if looks like a pointer (heuristic)
# Typical userspace: 0x00005500_00000000 - 0x00007fff_ffffffff
if 0x0000550000000000 <= value <= 0x00007fffffffffff:
pointers.append((base_addr + i, value))
return pointers
def calculate_entropy(self, data):
if not data:
return 0
# Simple byte frequency entropy
freq = [0] * 256
for byte in data:
freq[byte] += 1
import math
entropy = 0
for count in freq:
if count > 0:
p = count / len(data)
entropy -= p * math.log2(p)
return entropy
MemoryAnalyzer()
end
Heap Analysis
python
import gdb
import struct
class HeapWalker(gdb.Command):
"""Walk glibc heap and show allocations.
Usage: heapwalk [arena_address]
"""
def __init__(self):
super().__init__("heapwalk", gdb.COMMAND_DATA)
def invoke(self, args, from_tty):
# This is a simplified version for glibc malloc
# Real implementation would need to handle all chunk types
# Find heap region
inferior = gdb.selected_inferior()
# Get process mappings
try:
mappings = gdb.execute("info proc mappings", to_string=True)
except:
print("Cannot get process mappings")
return
# Parse mappings to find [heap]
heap_start = None
heap_end = None
for line in mappings.split('\n'):
if '[heap]' in line:
parts = line.split()
if len(parts) >= 2:
addrs = parts[0].split('-')
if len(addrs) == 2:
heap_start = int(addrs[0], 16)
heap_end = int(addrs[1], 16)
break
if not heap_start:
print("Heap not found")
return
print(f"Heap: 0x{heap_start:x} - 0x{heap_end:x}")
self.walk_heap(heap_start, heap_end)
def walk_heap(self, start, end):
# Simplified glibc chunk header:
# struct malloc_chunk {
# size_t prev_size; // Only if previous chunk is free
# size_t size; // Size with flags in low 3 bits
# // For free chunks:
# struct malloc_chunk *fd;
# struct malloc_chunk *bk;
# };
PREV_INUSE = 1
IS_MMAPPED = 2
NON_MAIN_ARENA = 4
inferior = gdb.selected_inferior()
ptr = start
chunk_num = 0
print("\nChunk Address Size Flags Status")
print("=" * 70)
while ptr < end and chunk_num < 100: # Limit iterations
try:
# Read chunk header (16 bytes)
header = inferior.read_memory(ptr, 16)
prev_size, size = struct.unpack('<QQ', bytes(header))
# Extract flags
flags = size & 7
actual_size = size & ~7
# Determine status
status = "ALLOCATED" if (flags & PREV_INUSE) else "FREE"
flags_str = ""
if flags & PREV_INUSE:
flags_str += "P"
if flags & IS_MMAPPED:
flags_str += "M"
if flags & NON_MAIN_ARENA:
flags_str += "N"
print(f"{chunk_num:5d} 0x{ptr:016x} {actual_size:8d} "
f"{flags_str:4s} {status}")
# Try to show some data for allocated chunks
if status == "ALLOCATED" and actual_size >= 16:
try:
data = inferior.read_memory(ptr + 16, min(32, actual_size - 16))
data_bytes = bytes(data)
# Check if looks like a string
if all(32 <= b < 127 or b == 0 for b in data_bytes[:16]):
try:
s = data_bytes.decode('utf-8', errors='ignore')
s = s.split('\0')[0]
if s:
print(f" Data: {s[:50]}")
except:
pass
except:
pass
# Move to next chunk
if actual_size == 0:
break
ptr += actual_size
chunk_num += 1
except gdb.MemoryError:
print(f" <memory read error at 0x{ptr:x}>")
break
except Exception as e:
print(f" <error: {e}>")
break
print("=" * 70)
print(f"Total chunks analyzed: {chunk_num}")
HeapWalker()
end
Call Stack Analysis
python
import gdb
import re
class SmartBacktrace(gdb.Command):
"""Enhanced backtrace with argument detection.
Usage: sbt
"""
def __init__(self):
super().__init__("sbt", gdb.COMMAND_STACK)
def invoke(self, args, from_tty):
frame = gdb.newest_frame()
level = 0
print("Level Function Arguments")
print("=" * 80)
while frame:
try:
name = frame.name() or "??"
pc = frame.pc()
# Try to get function arguments
args_str = self.get_arguments(frame)
print(f"{level:5d} {name:40s} {args_str}")
level += 1
frame = frame.older()
except Exception as e:
print(f"{level:5d} <error: {e}>")
break
print("=" * 80)
def get_arguments(self, frame):
"""Try to determine function arguments"""
arch = frame.architecture().name()
if 'x86-64' in arch or 'i386:x86-64' in arch:
return self.get_x86_64_args(frame)
elif 'aarch64' in arch:
return self.get_aarch64_args(frame)
else:
return ""
def get_x86_64_args(self, frame):
"""Get arguments for x86-64 calling convention"""
arg_regs = ['rdi', 'rsi', 'rdx', 'rcx', 'r8', 'r9']
args = []
for i, reg in enumerate(arg_regs):
try:
value = frame.read_register(reg)
val_int = int(value)
# Try to determine type
arg_str = self.format_value(val_int, frame)
args.append(f"arg{i}={arg_str}")
if len(args) >= 4: # Limit to 4 args for display
break
except:
break
return ", ".join(args)
def get_aarch64_args(self, frame):
"""Get arguments for AArch64 calling convention"""
arg_regs = ['x0', 'x1', 'x2', 'x3', 'x4', 'x5', 'x6', 'x7']
args = []
for i, reg in enumerate(arg_regs):
try:
value = frame.read_register(reg)
val_int = int(value)
arg_str = self.format_value(val_int, frame)
args.append(f"arg{i}={arg_str}")
if len(args) >= 4:
break
except:
break
return ", ".join(args)
def format_value(self, value, frame):
"""Try to format a value intelligently"""
# Check if NULL
if value == 0:
return "NULL"
# Check if looks like a pointer
if value > 0x1000:
try:
# Try to read as string
inferior = gdb.selected_inferior()
data = inferior.read_memory(value, 32)
# Check if printable ASCII
is_string = True
for i, byte in enumerate(bytes(data)):
if byte == 0:
break
if not (32 <= byte < 127):
is_string = False
break
if is_string:
s = bytes(data).split(b'\0')[0].decode('utf-8', errors='ignore')
if len(s) > 20:
s = s[:20] + "..."
return f'"{s}"'
except:
pass
# Just return as hex
return f"0x{value:x}"
SmartBacktrace()
end
Watchpoint with History
python
import gdb
class WatchHistory:
"""Track watchpoint hits and show history"""
def __init__(self):
self.history = []
self.max_history = 100
def record(self, variable, old_val, new_val, location):
import time
entry = {
'time': time.time(),
'variable': variable,
'old_value': old_val,
'new_value': new_val,
'location': location,
'backtrace': self.get_backtrace()
}
self.history.append(entry)
if len(self.history) > self.max_history:
self.history.pop(0)
def get_backtrace(self):
bt = []
frame = gdb.newest_frame()
while frame and len(bt) < 5:
name = frame.name() or "??"
bt.append(name)
frame = frame.older()
return bt
def show_history(self, count=10):
print(f"\nWatch History (last {count}):")
print("=" * 80)
for entry in self.history[-count:]:
print(f"\nVariable: {entry['variable']}")
print(f" Old value: {entry['old_value']}")
print(f" New value: {entry['new_value']}")
print(f" Location: {entry['location']}")
print(f" Backtrace: {' -> '.join(entry['backtrace'])}")
# Global history object
watch_history = WatchHistory()
class TrackedWatchpoint(gdb.Breakpoint):
"""Watchpoint that tracks history of changes"""
def __init__(self, variable):
super().__init__(variable, gdb.BP_WATCHPOINT)
self.variable = variable
self.last_value = None
def stop(self):
global watch_history
try:
frame = gdb.selected_frame()
current_value = gdb.parse_and_eval(self.variable)
if self.last_value is not None:
location = f"{frame.name()}:{frame.pc():#x}"
watch_history.record(
self.variable,
str(self.last_value),
str(current_value),
location
)
self.last_value = current_value
print(f"\nWatchpoint: {self.variable} changed")
print(f" New value: {current_value}")
watch_history.show_history(5)
return True
except Exception as e:
print(f"Error in watchpoint: {e}")
return True
class WatchHistoryCommand(gdb.Command):
"""Show watchpoint history.
Usage: watch-history [count]
"""
def __init__(self):
super().__init__("watch-history", gdb.COMMAND_DATA)
def invoke(self, args, from_tty):
global watch_history
count = 10
if args:
try:
count = int(args)
except:
print("Usage: watch-history [count]")
return
watch_history.show_history(count)
WatchHistoryCommand()
# Example usage:
# python TrackedWatchpoint("my_variable")
end
Event Handling
python
import gdb
# Stop event handlers
def on_stop(event):
"""Called whenever execution stops"""
if isinstance(event, gdb.SignalEvent):
print(f"Stopped by signal: {event.stop_signal}")
elif isinstance(event, gdb.BreakpointEvent):
print(f"Stopped at breakpoint")
for bp in event.breakpoints:
print(f" Breakpoint {bp.number} at {bp.location}")
# Show current location
frame = gdb.selected_frame()
if frame:
print(f"Stopped at: {frame.name()}")
gdb.events.stop.connect(on_stop)
# Continue event
def on_continue(event):
print("Execution continuing...")
gdb.events.cont.connect(on_continue)
# Exited event
def on_exit(event):
print(f"Program exited with code: {event.exit_code}")
gdb.events.exited.connect(on_exit)
# New thread event
def on_new_thread(event):
thread = event.inferior_thread
print(f"New thread created: {thread.num}")
gdb.events.new_thread.connect(on_new_thread)
# Breakpoint created
def on_breakpoint_created(bp):
print(f"Breakpoint created: #{bp.number} at {bp.location}")
gdb.events.breakpoint_created.connect(on_breakpoint_created)
# Breakpoint deleted
def on_breakpoint_deleted(bp):
print(f"Breakpoint deleted: #{bp.number}")
gdb.events.breakpoint_deleted.connect(on_breakpoint_deleted)
# New objfile (library) loaded
def on_new_objfile(event):
objfile = event.new_objfile
print(f"New object file loaded: {objfile.filename}")
gdb.events.new_objfile.connect(on_new_objfile)
# Clear inferior (when debugging ends)
def on_clear_objfiles(event):
print("Object files cleared")
gdb.events.clear_objfiles.connect(on_clear_objfiles)
# Disconnect event handler
# gdb.events.stop.disconnect(on_stop)
end
Complex Example: Automatic Leak Detection
python
import gdb
class LeakDetector:
"""Track malloc/free calls to detect memory leaks"""
def __init__(self):
self.allocations = {} # address -> (size, backtrace)
self.malloc_bp = None
self.free_bp = None
self.enabled = False
def start(self):
"""Start tracking allocations"""
if self.enabled:
print("Already tracking")
return
# Set breakpoints on malloc and free
try:
self.malloc_bp = MallocBreakpoint(self)
self.free_bp = FreeBreakpoint(self)
self.enabled = True
print("Leak detection started")
except Exception as e:
print(f"Failed to start tracking: {e}")
def stop(self):
"""Stop tracking and report leaks"""
if not self.enabled:
print("Not tracking")
return
if self.malloc_bp:
self.malloc_bp.delete()
if self.free_bp:
self.free_bp.delete()
self.enabled = False
self.report_leaks()
def record_malloc(self, address, size):
"""Record a malloc"""
bt = self.get_backtrace()
self.allocations[address] = (size, bt)
print(f"Malloc: 0x{address:x} size={size}")
def record_free(self, address):
"""Record a free"""
if address in self.allocations:
size, bt = self.allocations[address]
del self.allocations[address]
print(f"Free: 0x{address:x} size={size}")
else:
if address != 0: # free(NULL) is valid
print(f"WARNING: Free of untracked address: 0x{address:x}")
def get_backtrace(self):
"""Get current backtrace"""
bt = []
frame = gdb.newest_frame()
while frame:
name = frame.name() or "??"
bt.append(f"{name}:0x{frame.pc():x}")
frame = frame.older()
if len(bt) >= 10: # Limit depth
break
return bt
def report_leaks(self):
"""Report any leaked memory"""
if not self.allocations:
print("\nNo leaks detected!")
return
print(f"\n{'='*80}")
print(f"MEMORY LEAKS DETECTED: {len(self.allocations)} allocations")
print(f"{'='*80}")
total_leaked = 0
for addr, (size, bt) in self.allocations.items():
total_leaked += size
print(f"\nLeak: 0x{addr:x}, size={size} bytes")
print(" Allocated at:")
for frame in bt:
print(f" {frame}")
print(f"\n{'='*80}")
print(f"Total leaked: {total_leaked} bytes")
print(f"{'='*80}")
class MallocBreakpoint(gdb.FinishBreakpoint):
"""Breakpoint on malloc that captures return value"""
def __init__(self, detector):
# Find malloc
try:
super().__init__(gdb.newest_frame(), internal=True)
self.detector = detector
self.silent = True
except Exception as e:
# Fallback: set regular breakpoint
super().__init__()
self.spec = "malloc"
self.detector = detector
def stop(self):
try:
# Get return value (allocated address)
addr = int(self.return_value)
# Get size argument (first argument, in rdi on x86-64)
frame = gdb.selected_frame()
size = int(frame.read_register("rdi"))
if addr != 0:
self.detector.record_malloc(addr, size)
except Exception as e:
print(f"Error in malloc breakpoint: {e}")
return False # Continue execution
class FreeBreakpoint(gdb.Breakpoint):
"""Breakpoint on free"""
def __init__(self, detector):
super().__init__("free", internal=True)
self.detector = detector
self.silent = True
def stop(self):
try:
# Get address argument (first argument)
frame = gdb.selected_frame()
addr = int(frame.read_register("rdi"))
self.detector.record_free(addr)
except Exception as e:
print(f"Error in free breakpoint: {e}")
return False # Continue execution
# Global leak detector
leak_detector = LeakDetector()
class LeakDetectCommand(gdb.Command):
"""Control leak detection.
Usage: leak-detect [start|stop]
"""
def __init__(self):
super().__init__("leak-detect", gdb.COMMAND_DATA)
def invoke(self, args, from_tty):
global leak_detector
if args == "start":
leak_detector.start()
elif args == "stop":
leak_detector.stop()
else:
print("Usage: leak-detect [start|stop]")
LeakDetectCommand()
end
Useful Utility Functions
python
import gdb
def read_string(addr, max_len=1024):
"""Read a null-terminated string from memory"""
inferior = gdb.selected_inferior()
try:
data = inferior.read_memory(addr, max_len)
# Find null terminator
for i, byte in enumerate(bytes(data)):
if byte == 0:
return bytes(data[:i]).decode('utf-8', errors='ignore')
return bytes(data).decode('utf-8', errors='ignore')
except:
return None
def is_valid_address(addr):
"""Check if address is valid/mapped"""
try:
inferior = gdb.selected_inferior()
inferior.read_memory(addr, 1)
return True
except gdb.MemoryError:
return False
def get_register_size():
"""Get register size (4 for 32-bit, 8 for 64-bit)"""
return gdb.parse_and_eval("sizeof(void*)").type.sizeof
def follow_pointer(ptr, depth=1):
"""Follow pointer chain"""
current = ptr
for i in range(depth):
try:
val = gdb.parse_and_eval(f"*(void**){current}")
current = int(val)
print(f"Level {i}: 0x{current:x}")
except:
print(f"Level {i}: invalid")
break
def find_containing_function(addr):
"""Find which function contains an address"""
try:
block = gdb.block_for_pc(addr)
while block and not block.function:
block = block.superblock
if block and block.function:
return block.function.name
except:
pass
return None
# Helper to convert between types
def cast_to(value, type_name):
"""Cast a value to a different type"""
target_type = gdb.lookup_type(type_name)
return value.cast(target_type)
# Example usage in GDB:
# python print(read_string(0x400500))
# python follow_pointer(0x12345678, 3)
# python print(find_containing_function(0x400123))
end
This comprehensive guide covers both GDB's internal scripting language and the powerful Python API. The Python API is generally more flexible and powerful for complex tasks, while internal scripts are lighter weight for simple automation. In practice, many advanced GDB users combine both approaches as needed.