C++ Segmentation Fault & Core Dump: GDB/LLDB Debugging

C++ Segmentation Fault & Core Dump: GDB/LLDB Debugging

· 18분 읽기 · 수정 Intermediate Troubleshooting
C++ Segmentation Fault & Core Dump: GDB/LLDB Debugging

Introduction: When you see “Segmentation fault (core dumped)“

A segmentation fault means the OS terminated your process after invalid memory access: null dereference, dangling pointers, stack overflow, buffer overrun, etc. This article walks through enabling core dumps, loading them in GDB/LLDB, reading backtraces, and using AddressSanitizer (ASan) to catch issues earlier. Topics covered:

  • Core dumps: ulimit, systemd/kernel settings
  • Post-mortem analysis: load core in GDB/LLDB, bt, frame navigation, variables
  • Common causes: null deref, use-after-free, stack overflow
  • ASan for proactive detection
  • Production: automated core collection and analysis See also: GDB/LLDB, AddressSanitizer.

Experience from real projects

On large C++ codebases, book theory rarely matches production crashes. Profiling and code review helped turn vague failures into concrete fixes. The patterns below are meant to shorten your own trial-and-error loop.

Problem scenarios

Scenario 1: Occasional crashes in production

Logs show only Segmentation fault (core dumped) and no core file. Cause: ulimit -c is 0, core_pattern discards cores, or systemd limits cores. Fix: Enable cores → collect with coredumpctl or core_pattern → gdb ./program core and bt.

Scenario 2: Crash inside a third-party library

Cause: Bad pointer arguments, size/type mismatch, use-after-free across the FFI boundary. Fix: bt full, move to caller frames with frame N, inspect arguments; info sharedlibrary for symbols.

Scenario 3: Recursion or huge stack objects

Cause: Default stack size (~8 MB) exceeded. Fix: Repeated frames in bt → recursion; ulimit -s or heap allocation.

Scenario 4: Buffer overrun only on certain inputs

Fix: Re-run under ASan or Valgrind for exact line and size.

Summary

ScenarioHintApproach
Intermittent prod crashNo coreulimit, coredumpctl
Crash in .soTrace callersbt full, inspect args
Stack overflowDeep btulimit -s or heap
Input-dependentBoundsASan / Valgrind

Segfault debugging flow

flowchart TB
    subgraph Detect[Segfault]
        A[Segmentation fault] --> B{Core file?}
    end
    subgraph NoCore[No core]
        B -->|No| C[ulimit -c unlimited]
        C --> D[Check kernel.core_pattern]
        D --> E[Reproduce]
    end
    subgraph WithCore[Core available]
        B -->|Yes| F[gdb ./program core]
        F --> G[bt backtrace]
        G --> H[frame 0]
        H --> I[print variables]
        I --> J{Root cause?}
    end
    subgraph Prevent[Prevention]
        K[ASan build] --> L[Run tests]
        L --> M[Fix on first report]
    end
    E --> F
    J -->|Reproducible| K

Table of contents

  1. Enabling core dumps
  2. Analyzing cores with GDB/LLDB
  3. Complete debugging examples
  4. Common causes
  5. Debugging strategies
  6. Defensive coding
  7. ASan
  8. Production patterns
  9. Common tool errors
  10. FAQ

1. Enabling core dumps

ulimit

  • ulimit -c: if 0, no core file is written.
  • ulimit -c unlimited: in the shell session that runs the binary.
  • For persistence: /etc/security/limits.conf or shell rc files.
ulimit -c
ulimit -c unlimited
echo "ulimit -c unlimited" >> ~/.zshrc

Location and naming

  • Often core or core. in the process CWD.
  • /proc/sys/kernel/core_pattern: e.g. core.%e.%p.%t.
cat /proc/sys/kernel/core_pattern
sysctl kernel.core_pattern
echo "core.%e.%p.%t" | sudo tee /proc/sys/kernel/core_pattern

systemd

  • Set DefaultLimitCORE=infinity (and service LimitCORE= as needed).
  • coredumpctl list, coredumpctl debug or time range.
# /etc/systemd/system/myapp.service
[Service]
DefaultLimitCORE=infinity

2. GDB/LLDB core analysis

Loading a core

Use the same binary build that produced the crash, preferably with -g.

gdb ./your_program core
lldb -c core ./your_program

Backtrace

  • bt: call stack; #0 is the faulting frame.
  • bt full: locals per frame (GDB); bt -f (LLDB).

Frames and variables

  • frame 0, list, print ptr
  • *print ptr may fail if the address is invalid in the dump. | Task | GDB | LLDB | |------|-----|------| | Backtrace | bt | bt | | Locals | bt full | bt -f | | Select frame | frame N | frame select N | | Print | print x | frame variable x | | Disassembly | disas | disassemble |

3. Complete examples

Example 1: Null pointer dereference

// g++ -g -o segfault_demo segfault_demo.cpp && ./segfault_demo
#include <iostream>
int main() {
    int* p = nullptr;
    std::cout << *p << "\n";  // null deref → segfault
    return 0;
}
gdb ./segfault_demo core
(gdb) bt
(gdb) print p
$1 = (int *) 0x0

Example 2: Use-after-free

int* p = new int(42);
delete p;
std::cout << *p << "\n";  // UAF

Rebuild with -fsanitize=address and run; ASan prints heap-use-after-free with free and use sites.

Example 3: Stack overflow (deep recursion)

bt shows the same function repeated many times → reduce recursion or increase stack / use heap.

Example 4: Buffer overflow

Use ASan: stack-buffer-overflow with file and line.

Example 5: Double free

ASan: attempting double-free with allocation and both frees.

Example 6: LLDB on macOS

lldb -c core ./myapp
(lldb) bt
(lldb) bt -f
(lldb) frame select 0

Enable cores: ulimit -c unlimited; cores may appear under ~/Library/Logs/DiagnosticReports/.

4. Common causes

CauseGDB / cluesASan
Null derefprint ptr → 0x0Crash at site
UAFmappings / ASanheap-use-after-free
Stack overflowRepeated frames—
Buffer overrunHard to seestack/heap-buffer-overflow
Double free—attempting double-free

5. Strategies

  1. Ensure cores or ASan repro.
  2. bt → frame 0 → print.
  3. For library crashes, inspect caller frames and arguments.
  4. CI with ASan tests; Valgrind where ASan is awkward.
flowchart LR
    A[Core?] --> B[GDB / print]
    A --> C[ASan run]
    B --> D[Fix]
    C --> D

6. Defensive patterns

  • Check pointers before use; assert in debug.
  • delete p; p = nullptr;
  • std::unique_ptr / std::shared_ptr
  • Prefer strncpy / snprintf / std::string over unchecked C APIs.
  • RAII for files, locks, memory.

7. AddressSanitizer

g++ -g -fsanitize=address -fno-omit-frame-pointer -o myapp myapp.cpp

CMake: target_compile_options / target_link_options with -fsanitize=address.

8. Production patterns

  • systemd: DefaultLimitCORE=infinity, LimitCORE=infinity
  • Docker: ulimit / core pattern volume
  • Scripts: gdb -batch -ex “bt full” -ex quit on program + core
  • coredumpctl: coredumpctl debug, dump -o
  • CI: ASan build + ctest
  • Split debug info: objcopy —only-keep-debug, debuglink

Common tool errors

  1. Core format mismatch: Same arch, same build machine where possible; file core vs file ./app.
  2. No symbols: Rebuild with -g, matching binary.
  3. Cannot access memory: In dumps, some addresses are unmapped—avoid dereferencing in GDB if it fails.
  4. ulimit cannot change: systemd / limits.conf / Docker —ulimit core=-1

FAQ

Q. No core file?
A. Set ulimit -c unlimited, fix core_pattern, systemd limits, and use coredumpctl on systemd. Q. No source in GDB?
A. Debug or RelWithDebInfo, -g, same build as the crashing binary. Q. ASan in production?
A. Typically no—use ASan in test/CI; production uses cores + symbols. Q. When is this guide useful?
A. Whenever you need a repeatable workflow from segfault → core → bt → fix, plus ASan for earlier detection. Q. What to read next?
A. Follow Previous/Next links at the bottom or the C++ series index. Q. Learn more?
A. cppreference, sanitizer docs, GDB/LLDB manuals.

Keywords

Segmentation fault, core dumped, GDB, LLDB, core dump, debugging, use-after-free, null pointer, AddressSanitizer.

Practical tips

  • Reproduce with minimal tests; fix compiler warnings first.
  • Measure before optimizing unrelated paths.

Summary

  1. ulimit -c unlimited (and systemd/Docker as needed).
  2. gdb ./program core or coredumpctl debug.
  3. bt → frame 0 → inspect pointers and logic.
  4. Suspect null, UAF, stack, overrun in order.
  5. -fsanitize=address when you can reproduce.
  6. Automate core handling and keep debug symbols. Next: CMake link errors (#49-2) Next article: CMake link errors LNK2019 / undefined reference Previous article: Custom memory pool (#48-3)

Frequently Asked Questions (FAQ)

Q. When would I use this in practice?

A. Enable core dumps, analyze crashes with GDB or LLDB, and catch use-after-free with AddressSanitizer.

Q. What should I read before this?

A. Follow the previous article or related articles links at the bottom of each post to learn in sequence. See the C++ series index for the full picture.

Q. Where can I study this more deeply?

A. Check cppreference and the relevant library’s official documentation. The reference links at the end of the article are also worth using.

Other articles related to this topic.

Keywords Covered in This Article (Related Search Terms)

This article covers C++, Segmentation fault, core dump, GDB, LLDB, debugging, AddressSanitizer.