# pprof

pprof is a tool for visualization and analysis of profiling data.

pprof reads a collection of profiling samples in profile.proto format and
generates reports to visualize and help analyze the data. It can generate both
text and graphical reports (through the use of the dot visualization package).

profile.proto is a protocol buffer that describes a set of callstacks
and symbolization information. A common usage is to represent a set of
sampled callstacks from statistical profiling. The format is
described on the src/proto/profile.proto file. For details on protocol
buffers, see https://developers.google.com/protocol-buffers

Profiles can be read from a local file, or over http. Multiple
profiles of the same type can be aggregated or compared.

If the profile samples contain machine addresses, pprof can symbolize
them through the use of the native binutils tools (addr2line and nm).

# pprof profiles

pprof operates on data in the profile.proto format. Each profile is a collection
of samples, where each sample is associated to a point in a location hierarchy,
one or more numeric values, and a set of labels. Often these profiles represents
data collected through statistical sampling of a program, so each sample
describes a program call stack and a number or weight of samples collected at a
location. pprof is agnostic to the profile semantics, so other uses are
possible. The interpretation of the reports generated by pprof depends on the
semantics defined by the source of the profile.

# General usage

The objective of pprof is to generate a report for a profile. The report is
generated from a location hierarchy, which is reconstructed from the profile
samples. Each location contains two values: *flat* is the value of the location
itself, while *cum* is the value of the location plus all its
descendants. Samples that include a location multiple times (eg for recursive
functions) are counted only once per location.

The basic usage of pprof is

    pprof <format> [options] source

Where *format* selects the nature of the report, and *options* configure the
contents of the report. Each option has a value, which can be boolean, numeric,
or strings. While only one format can be specified, most options can be selected
independently of each other.

Some common pprof options are:

* **-flat [default]:** Sort entries based on their flat weight, on text reports.
* **-cum:** Sort entries based on cumulative weight, on text reports.
* **-functions [default]:** Accumulate samples at the function level; profile
  locations that describe the same function will be merged into a report entry.
* **-lines:** Accumulate samples at the source line level; profile locations that
  describe the same function will be merged into a report entry.
* **-addresses:** Accumulate samples at the instruction address; profile locations
  that describe the same function address will be merged into a report entry.
* **-nodecount= _int_:** Maximum number of entries in the report. pprof will only print
  this many entries and will use heuristics to select which entries to trim.
* **-focus= _regex_:** Only include samples that include a report entry matching
  *regex*.
* **-ignore= _regex_:** Do not include samples that include a report entry matching
  *regex*.
* **-show= _regex_:** Only show entries that match *regex*.
* **-hide= _regex_:** Do not show entries that match *regex*.

Each sample in a profile may include multiple values, representing different
entities associated to the sample. pprof reports include a single sample value,
which by convention is the last one specified in the report. The `sample_index=`
option selects which value to use, and can be set to a number (from 0 to the
number of values - 1) or the name of the sample value.

Sample values are numeric values associated to a unit. If pprof can recognize
these units, it will attempt to scale the values to a suitable unit for
visualization. The `unite=` option will force the use of a specific unit. For
example, `sample_index=sec` will force any time values to be reported in
seconds. pprof recognizes most common time and memory size units.

## Text reports

pprof text reports show the location hierarchy in text format.

* **-text:** Prints the location entries, one per line, including the flat and cum
  values.
* **-tree:** Prints each location entry with its predecessors and successors.
* **-peek= _regex_:** Print the location entry with all its predecessors and
  successors, without trimming any entries.
* **-traces:** Prints each sample with a location per line.

## Graphical reports

pprof can generate graphical reports on the DOT format, and convert them to
multiple formats using the graphviz package.

These reports represent the location hierarchy as a graph, with a report entry
represented as a node. Solid edges represent a direct connection between
entries, while dotted edges represent a connection where some intermediate nodes
have been removed. Nodes are removed using heuristics to limit the size of
the graph, controlled by the *nodecount* option.

The size of each node represents the flat weight of the node, and the width of
each edge represents the cumulative weight of all samples going through
it. Nodes are colored according to their cumulative weight, highlighting the
paths with the highest cum weight.

* **-dot:** Generates a report in .dot format. All other formats are generated from
  this one.
* **-svg:** Generates a report in SVG format.
* **-web:** Generates a report in SVG format on a temp file, and starts a web
  browser to view it.
* **-png, -jpg, -gif, -pdf:** Generates a report in these formats,

## Annotated code

pprof can also generate reports of annotated source with samples associated to
them. For these, the source or binaries must be locally available, and the
profile must contain data with the appropriate level of detail.

pprof will look for source files on its current working directory and all its
ancestors. pprof will look for binaries on the directories specified in the
`$PPROF_BINARY_PATH` environment variable, by default `$HOME/pprof/binaries`
(`%USERPROFILE%\pprof\binaries` on Windows). It will look binaries up by name,
and if the profile includes linker build ids, it will also search for them in
a directory named as the build id.

pprof uses the binutils tools to examine and disassemble the binaries. By
default it will search for those tools in the current path, but it can also
search for them in a directory pointed to by the environment variable
`$PPROF_TOOLS`.

* **-disasm= _regex_:** Generates an annotated source listing for functions matching
  regex, with flat/cum weights for each source line.
* **-list= _regex_:** Generates an annotated disassembly listing for functions
  matching *regex*.
* **-weblist= _regex_:** Generates a source/assembly combined annotated listing for
  functions matching *regex*, and starts a web browser to display it.

# Fetching profiles

pprof can read profiles from a file or directly from a URL over http. Its native
format is a gzipped profile.proto file, but it can also accept some legacy
formats generated by [gperftools](https://github.com/gperftools/gperftools).

When fetching from a URL handler, pprof accepts options to indicate how much to
wait for the profile.

* **-seconds= _int_:** Makes pprof request for a profile with the specified duration
  in seconds. Only makes sense for profiles based on elapsed time, such as CPU
  profiles.
* **-timeout= _int_:** Makes pprof wait for the specified timeout when retrieving a
  profile over http. If not specified, pprof will use heuristics to determine a
  reasonable timeout.

If multiple profiles are specified, pprof will fetch them all and merge
them. This is useful to combine profiles from multiple processes of a
distributed job. The profiles may be from different programs but must be
compatible (for example, CPU profiles cannot be combined with heap profiles).

pprof can subtract a profile from another in order to compare them. For that,
use the **-base= _profile_** option, where *profile* is the filename or URL for the
profile to be subtracted. This may result on some report entries having negative
values.

## Symbolization

pprof can add symbol information to a profile that was collected only with
address information. This is useful for profiles for compiled languages, where
it may not be easy or even possible for the profile source to include function
names or source coordinates.

pprof can extract the symbol information locally by examining the binaries using
the binutils tools, or it can ask running jobs that provide a symbolization
interface.

pprof will attempt symbolizing profiles by default, and its `-symbolize` option
provides some control over symbolization:

* **-symbolize=none:** Disables any symbolization from pprof.

* **-symbolize=local:** Only attempts symbolizing the profile from local
  binaries using the binutils tools.

* **-symbolize=remote:** Only attempts to symbolize running jobs by contacting
  their symbolization handler.

For local symbolization, pprof will look for the binaries on the paths specified
by the profile, and then it will search for them on the path specified by the
environment variable `$PPROF_BINARY_PATH`. Also, the name of the main binary can
be passed directly to pprof as its first parameter, to override the name or
location of the main binary of the profile, like this:

    pprof /path/to/binary profile.pb.gz

By default pprof will attempt to demangle and simplify C++ names, to provide
readable names for C++ symbols. It will aggressively discard template and
function parameters. This can be controlled with the `-symbolize=demangle`
option. Note that for remote symbolization mangled names may not be provided by
the symbolization handler.

* **--symbolize=demangle=none:** Do not perform any demangling. Show mangled
  names if available.

* **-symbolize=demangle=full:** Demangle, but do not perform any
  simplification. Show full demangled names if available.

* **-symbolize=demangle=templates:** Demangle, and trim function parameters, but
  not template parameters.

