Linux Kernel Debugging: Leverage open source tools and advanced techniques to debug Linux kernel or module issues
- Length: 571 pages
- Edition: 1
- Language: English
- Publisher: Packt Publishing
- Publication Date: 2022-08-09
- ISBN-10: 1801075034
- ISBN-13: 9781801075039
- Sales Rank: #0 (See Top 100 Books)
Debug Linux kernel modules and the kernel itself by mastering powerful open source tools and advanced kernel debugging techniques
Key Features
- Learn how to use a variety of kernel and module debug tools and techniques with the help of examples
- Understand how to expertly interpret a kernel oops and identify the underlying defects
- Includes easy-to-look up tables and simplified explanations of complex kernel-level defects
Book Description
The Linux kernel is at the very core of arguably the world’s best production-quality OS. Debugging it, though, can be a complex endeavor. Linux Kernel Debugging is a thorough guide to advanced kernel debugging.
This book covers a range of debugging techniques in detail, starting with instrumentation-based debugging techniques, the kernel’s powerful Kprobes framework, printk, and friends. The book features two whole chapters devoted to the tools and techniques for debugging memory-related bugs. It also prepares you to interpret the underlying issue when the Linux kernel throws an oops. As you advance through the chapters, you’ll be able to demystify concurrency by learning what data races are and how to handle them, including the use of modern features like Kernel Concurrency Sanitizer (KCSAN). You’ll also discover how to overcome some thorny issues related to debugging and performance through detailed kernel-level tracing and learn to wield the power of Ftrace and its frontends. What’s more? The book shows you how to deal with kernel hangs and panic, leverage the venerable GDB tool within the kernel, and much more.
By the time you complete reading this debugging kernel book, you’ll have at your disposal, a whole range of powerful debug tools and techniques, along with a keen sense as to when to use which.
What you will learn
- Instrumentation-based printk and the powerful dynamic debug framework
- Use Kprobes statically and dynamically to trap into kernel or module functions
- Catch kernel memory defects with KASAN, UBSAN, SLUB debug, and kmemleak
- Precisely identify the source location of kernel and module bugs
- Understand data races and use KCSAN to catch evasive concurrency defects
- Leverage Ftrace and trace-cmd to trace the kernel flow in great detail
- Write a custom kernel panic handler using the WD and hangcheck timer
- Learn to use KGDB to single-step and debug kernel or module source code
Who This Book Is For
This book is for Linux kernel developers, module or driver authors, and testers interested in debugging and enhancing their Linux systems at the level of the kernel. System administrators who want to understand and debug the internal infrastructure of their Linux kernels will also find this book useful. A good hold of C programming and the Linux command line is necessary. Some experience with module or kernel development will be beneficial too.
Linux Kernel Debugging Linux Kernel Debugging 1 A General Introduction to Debugging Software Technical requirements Cloning this book’s code repository Software debugging – what it is, origins, and myths Software bugs – a few actual cases Patriot missile failure ESA’s unmanned Ariane 5 rocket Mars Pathfinder reset issue The Boeing 737 MAX aircraft – the MCAS and lack of training to the flight crew Other cases Setting up the workspace Running Linux as a native or guest OS? Running Linux as a guest OS Installing the Oracle VirtualBox guest additions Installing required software packages A tale of two kernels A production and a debug kernel Setting up our custom production kernel Setting up our custom debug kernel Seeing the difference – production and debug kernel config Debugging – a few quick tips A programmer’s checklist – seven rules Summary Further reading 2 Approaches to Kernel Debugging Technical requirements Classifying bug types Types of bugs – the classic view Types of bugs – the memory view Types of bugs – the CVE/CWE security-related view Types of bugs – the Linux kernel Kernel debugging – why there are different approaches to it Summarizing the different approaches to kernel debugging Development phase Unit testing and/or QA phases Categorizing into different scenarios Summary Further reading 3 Debug via Instrumentation – printk and friends Technical requirements The ubiquitous kernel printk Using the printk API’s logging levels Leveraging the pr_<foo> convenience macros Understanding where the printk output goes Practically using the printk format specifiers – a few quick tips Leveraging the printk for debug purposes Writing debug messages to the kernel log Debug printing – quick and useful tips Device drivers – use the dev_dbg() Trying our kernel module on the custom production kernel Rate limiting the printk Using the kernel’s powerful dynamic debug feature Dynamic debug via module parameters Specifying what and how to print debug messages Exercising dynamic debug on a kernel module on a production kernel Remaining printk miscellany Printing before console init – the early printk Designating the printk to some known presets Printing exactly once Emitting a printk from userspace Easily dumping buffer content Remaining points – bootloader log peeking, LED flashing and more Summary Further reading 4 Debug via Instrumentation – Kprobes Understanding kprobes basics What we intend to do Using static kprobes – traditional approaches to probing Demo 1 – static kprobe – trapping into the file open the traditional static kprobes way – simplest case Demo 2 – Static kprobe – specifying the function to probe via a module parameter Understanding the basics of the Application Binary Interface (ABI) Using static kprobes – demo 3 and demo 4 Demo 3 – static kprobe – probing the file open and retrieving the filename parameter Demo 4 – Semi-automated static kprobe via our helper script Getting started with kretprobes Kprobes miscellany Kprobes – limitations and downsides Interface stability The easier way – dynamic kprobes or kprobe-based event tracing Kprobe-based event tracing – minimal internal details Setting up a dynamic kprobe (via kprobe events) on any function Using dynamic kprobe event tracing on a kernel module Setting up a return probe (kretprobe) with kprobe-perf Trapping into the execve() – via perf and eBPF tooling System calls and where they land in the kernel Observability with eBPF tools – an introduction Summary Further reading 5 Debugging Kernel Memory Issues – Part 1 Technical requirements What’s the problem with memory anyway? Tools to catch kernel memory issues – a quick summary Using KASAN and UBSAN to find memory bugs Understanding KASAN – the basics Requirements to use KASAN Configuring the kernel for generic KASAN mode Bug hunting with KASAN Using the UBSAN kernel checker to find UB Building your kernel and modules with Clang Using Clang 13 on Ubuntu 21.10 Catching memory defects in the kernel – comparisons and notes (Part 1) Miscellaneous notes Summary Further reading 6 Debugging Kernel Memory Issues – Part 2 Technical requirements Detecting slab memory corruption via SLUB debug Configuring the kernel for SLUB debug Leveraging SLUB debug features via the slub_debug kernel parameter Running and tabulating the SLUB debug test cases Interpreting the kernel’s SLUB debug error report Learning to use the slabinfo and related utilities Finding memory leakage issues with kmemleak Configuring the kernel for kmemleak Using kmemleak A few tips for developers regarding dynamic kernel memory allocation Catching memory defects in the kernel – comparisons and notes (Part 2) Miscellaneous notes Summary Further reading 7 Oops! Interpreting the kernel bug diagnostic Technical requirements Generating a simple kernel bug and Oops The procmap utility What’s this NULL trap page anyway A simple Oops v1 – dereferencing the NULL pointer Doing a bit more of an Oops – our buggy module v2 A kernel Oops and what it signifies Devil in the details – decoding the Oops Line-by-line interpretation of an Oops Tools and techniques to help determine the location of the Oops Using objdump to help pinpoint the Oops code location Using GDB to help debug the Oops Using addr2line to help pinpoint the Oops code location Taking advantage of kernel scripts to help debug kernel issues Leveraging the console device to get the kernel log after Oops’ing in IRQ context An Oops on an ARM Linux system and using netconsole Figuring out the actual buggy code location (on ARM) A few actual Oops’es Summary Further reading
Donate to keep this site alive
How to download source code?
1. Go to: https://github.com/PacktPublishing
2. In the Find a repository… box, search the book title: Linux Kernel Debugging: Leverage open source tools and advanced techniques to debug Linux kernel or module issues
, sometime you may not get the results, please search the main title.
3. Click the book title in the search results.
3. Click Code to download.
1. Disable the AdBlock plugin. Otherwise, you may not get any links.
2. Solve the CAPTCHA.
3. Click download link.
4. Lead to download server to download.