perf (Linux)

perf (sometimes called perf_events^[1] or perf tools, originally Performance Counters for Linux, PCL)^[2] is a performance analyzing tool in Linux, available from Linux kernel version 2.6.31 in 2009.^[3] Userspace controlling utility, named perf, is accessed from the command line and provides a number of subcommands; it is capable of statistical profiling of the entire system (both kernel and userland code).

It supports hardware performance counters, tracepoints, software performance counters (e.g. hrtimer), and dynamic probes (for example, kprobes or uprobes).^[4] In 2012, two IBM engineers recognized perf (along with OProfile) as one of the two most commonly used performance counter profiling tools on Linux.^[5]

The interface between the perf utility and the kernel consists of only one syscall and is done via a file descriptor and a mapped memory region.^[6] Unlike LTTng or older versions of oprofile, no service daemons are needed, as most functionality is integrated into the kernel. The perf utility dumps raw data from the mapped buffer to disk when the buffer becomes filled up. According to R. Vitillo (LBNL), profiling performed by perf involves a very low overhead.^[6]

As of 2010^[update], architectures that provide support for hardware counters include x86, PowerPC64, UltraSPARC (III and IV), ARM (v5, v6, v7, Cortex-A8 and -A9), Alpha EV56 and SuperH.^[4] Usage of Last Branch Records,^[7] a branch tracing implementation available in Intel CPUs since Pentium 4, is available as a patch.^[6] Since version 3.14 of the Linux kernel mainline, released on March 31, 2014, perf also supports running average power limit (RAPL) for power consumption measurements, which is available as a feature of certain Intel CPUs.^[8][9][10]

Perf is natively supported in many popular Linux distributions, including Red Hat Enterprise Linux (since its version 6 released in 2010)^[11] and Debian in the linux-tools-common package (since Debian 6.0 (Squeeze) released in 2011).^[12]

perf is used with several subcommands:

• stat: measure total event count for single program or for system for some time
• top: top-like dynamic view of hottest functions
• record: measure and save sampling data for single program^[13]
• report: analyze file generated by perf record; can generate flat, or graph profile.^[13]
• annotate: annotate sources or assembly
• sched: tracing/measuring of scheduler actions and latencies^[14]
• list: list available events

The documentation of perf is not very detailed (as of 2014); for example, it does not document most events or explain their aliases (often external tools are used to get names and codes of events^[15]).^[16] Perf tools also cannot profile based on true wall-clock time.,^[16] something that has been addressed by the addition of off-CPU profiling.

The perf subsystem of Linux kernels from 2.6.37 up to 3.8.8 and RHEL6 kernel 2.6.32 contained a security vulnerability (CVE-2013-2094), which was exploited to gain root privileges by a local user.^[17][18] The problem was due to an incorrect type being used (32-bit int instead of 64-bit) in the event_id verification code path.^[19]

• List of performance analysis tools
• OProfile
• Performance Application Programming Interface
• Profiling (computer programming)

• perf's wiki on kernel.org
• Arnaldo Carvalho de Melo, The New Linux ’perf’ tools, presentation from Linux Kongress, September, 2010
• Hardware PMU support charts - check perf_event column
• perf Examples by Brendan Gregg