Kernel marker
Encyclopedia
Kernel markers were a static kernel instrumentation support mechanism for Linux kernel
source code, allowing special tools such as LTTng
or SystemTap
to trace
information exposed by these probe points. Kernel markers were declared in the kernel code by one-liners of the form:
Where name is the marker's unique name, and format_string describes the remaining arguments' types.
A marker can be on or off depending on whether a probe is connected to it or not. Code which wants to hook into a trace point first calls:
to register its probe callback with the marker point (pdata is a private data value that the code wants to pass to the probe). Later, the probe is turned on and off using:
Using markers has a negligible overhead thanks in part to Immediate Values, another support mechanism that embeds switches in the code that can be dynamically turned on and off, without using a memory reference and thus saving cache lines.
The initial motivation to create this static instrumentation infrastructure was the large performance overhead induced by the predating dynamic instrumentation mechanism Kprobe mechanism, which depends on breakpoint
s. Static instrumentation can also more easily survive source code changes because the markers are in the source code.
Kernel Markers consisted essentially of a C
preprocessing macro which added, in the instrumented function, a branch over a function call. By doing so, neither the stack
setup nor the function call are executed when instrumentation is not enabled. By identifying the branch executing stack setup and
function call as unlikely (using the gcc
built-in expect), a hint is given to the compiler
to position the tracing instructions away from cache lines involved in standard kernel execution.
Two Kernel Markers drawbacks were identified which led to its replacement by Tracepoints:
A patch-set implementing them was merged into version 2.6.24, which was released on January 24, 2008. To address issues regarding kernel markers, Mathieu Desnoyers, their original author, implemented a simpler and more type-safe version of static probe points named Tracepoints. A patch-set implementing Tracepoints was merged into version 2.6.28, which was released on December 25, 2008. Starting then, kernel markers were slowly removed from kernel sources and eventually fully removed in Linux kernel 2.6.32, which was released on December 3, 2009.
Linux kernel
The Linux kernel is an operating system kernel used by the Linux family of Unix-like operating systems. It is one of the most prominent examples of free and open source software....
source code, allowing special tools such as LTTng
LTTng
LTTng is a system software package for tracing the Linux kernel. LTTng consists of a kernel patch and a kernel module package. It is used together with*ltt-control, a toolchain to control tracing, and...
or SystemTap
SystemTap
In computing, SystemTap is the primary scripting language and tool for dynamically instrumenting running production Linux operating systems...
to trace
Tracing (software)
In software engineering, tracing is a specialized use of logging to record information about a program's execution. This information is typically used by programmers for debugging purposes, and additionally, depending on the type and detail of information contained in a trace log, by experienced...
information exposed by these probe points. Kernel markers were declared in the kernel code by one-liners of the form:
Where name is the marker's unique name, and format_string describes the remaining arguments' types.
A marker can be on or off depending on whether a probe is connected to it or not. Code which wants to hook into a trace point first calls:
to register its probe callback with the marker point (pdata is a private data value that the code wants to pass to the probe). Later, the probe is turned on and off using:
Using markers has a negligible overhead thanks in part to Immediate Values, another support mechanism that embeds switches in the code that can be dynamically turned on and off, without using a memory reference and thus saving cache lines.
The initial motivation to create this static instrumentation infrastructure was the large performance overhead induced by the predating dynamic instrumentation mechanism Kprobe mechanism, which depends on breakpoint
Breakpoint
In software development, a breakpoint is an intentional stopping or pausing place in a program, put in place for debugging purposes. It is also sometimes simply referred to as a pause....
s. Static instrumentation can also more easily survive source code changes because the markers are in the source code.
Kernel Markers consisted essentially of a C
C (programming language)
C is a general-purpose computer programming language developed between 1969 and 1973 by Dennis Ritchie at the Bell Telephone Laboratories for use with the Unix operating system....
preprocessing macro which added, in the instrumented function, a branch over a function call. By doing so, neither the stack
Call stack
In computer science, a call stack is a stack data structure that stores information about the active subroutines of a computer program. This kind of stack is also known as an execution stack, control stack, run-time stack, or machine stack, and is often shortened to just "the stack"...
setup nor the function call are executed when instrumentation is not enabled. By identifying the branch executing stack setup and
function call as unlikely (using the gcc
GNU Compiler Collection
The GNU Compiler Collection is a compiler system produced by the GNU Project supporting various programming languages. GCC is a key component of the GNU toolchain...
built-in expect), a hint is given to the compiler
Compiler
A compiler is a computer program that transforms source code written in a programming language into another computer language...
to position the tracing instructions away from cache lines involved in standard kernel execution.
Two Kernel Markers drawbacks were identified which led to its replacement by Tracepoints:
- Type verification was limited to scalar types because the API is based on format strings. This could be problematic if pointers must be dereferenced by the tracer code.
- The Markers "hide" the instrumentation in the source code, keeping no global registry of the instrumentation. This makes namespace conventions and tracking of instrumentation modification difficult unless the whole kernel tree is monitored.
A patch-set implementing them was merged into version 2.6.24, which was released on January 24, 2008. To address issues regarding kernel markers, Mathieu Desnoyers, their original author, implemented a simpler and more type-safe version of static probe points named Tracepoints. A patch-set implementing Tracepoints was merged into version 2.6.28, which was released on December 25, 2008. Starting then, kernel markers were slowly removed from kernel sources and eventually fully removed in Linux kernel 2.6.32, which was released on December 3, 2009.
External links
- Jonathan Corbet, Kernel markers, LWN.net, 2007
- Mathieu Desnoyers, Using the Linux Kernel Markers, Linux kernelLinux kernelThe Linux kernel is an operating system kernel used by the Linux family of Unix-like operating systems. It is one of the most prominent examples of free and open source software....
documentation, 2008 - Jonathan Corbet, Tracing: no shortage of options, LWN.net, 2008