7 Speculative Tracing
WARNING:
Oracle Linux 7 is now in Extended Support. See Oracle Linux Extended Support and Oracle Open Source Support Policies for more information.
Migrate applications and data to Oracle Linux 8 or Oracle Linux 9 as soon as possible.
For more information about DTrace, see Oracle Linux: DTrace Release Notes and Oracle Linux: Using DTrace for System Tracing.
This chapter describes how to use the DTrace facility for speculative tracing, which includes the ability to tentatively trace data and then later decide whether to commit the data to a tracing buffer or discard it.
About Speculative Tracing
In DTrace, the primary mechanism for filtering out uninteresting events is the predicate mechanism, which is described in more detail in D Program Structure. Predicates are useful when you know whether a probe event is of interest at the time that it fires. For example, if you are only interested in activity that is associated with a certain process or a certain file descriptor, you know when the probe fires if it is associated with the process or file descriptor of interest. Note that in other situations, you might not know whether a given probe event is of interest until some time after the probe fires.
Take the example of a system call that is occasionally failing
with a common error code such as EIO
or
EINVAL
. In this instance, you might want to
examine the code path leading to the error condition. To capture
the code path, you could enable every probe, but only if the
failing call can be isolated in such a way that a meaningful
predicate can be constructed. If the failures are sporadic or
non-deterministic, you would be forced to trace all of the events
that might be interesting, then later post-process the data to
filter out the events that were not associated with the failing
code path. In this case, even though the number of interesting
events might be reasonably small, the number of events that must
be traced is very large, making post-processing difficult.
In such situations, you can use speculative tracing facility to tentatively trace data at one or more probe locations. You can then decide to commit the data to the principal buffer at another probe location. The result is that your trace data only contains the output that is of interest; no post-processing is required and the DTrace overhead is minimized.
Speculation Interfaces
The following table describes DTrace speculation functions.
Table 7-1 DTrace Speculation Functions
Function | Args | Description |
---|---|---|
|
None |
Returns an identifier for a new speculative buffer. |
|
ID |
Denotes that the remainder of the clause should be traced to the speculative buffer specified by ID. |
|
ID |
Commits the speculative buffer that is associated with ID. |
|
ID |
Discards the speculative buffer that is associated with ID. |
Creating a Speculation
The speculation
function allocates a
speculative buffer and returns a speculation identifier. The
speculation identifier should be used in subsequent calls to the
speculate
function. Speculative buffers are a
finite resource. If no speculative buffer is available when
speculation
is called, an ID of zero is
returned and a corresponding DTrace error counter is incremented.
An ID of zero is always invalid, but it can be passed to the
speculate
, commit
and
discard
functions. If a call to
speculation
fails, dtrace
generates a message similar to the following:
dtrace: 2 failed speculations (no speculative buffer space available)
The number of speculative buffers defaults to one but can be optionally tuned higher. See Speculation Options and Tuning.
Using a Speculation
To use a speculation, an identifier that is returned from
speculation
must be passed to the
speculate
function in a clause prior to any
data-recording actions. All subsequent data-recording actions in a
clause containing a speculate
are speculatively
traced. The D compiler generates a compile-time error if a call to
speculate
follows data-recording actions in a D
probe clause. Therefore, clauses might contain speculative tracing
or non-speculative tracing requests, but not both.
Aggregating actions, destructive actions, and the
exit
action may never be speculative. Any
attempt to take one of these actions in a clause containing a
speculate
results in a compile-time error.
Also, a speculate
may not follow a
speculate
. Only one speculation is permitted
per clause. A clause that contains only a
speculate
speculatively traces the default
action, which is defined to trace only the enabled probe ID. See
Actions and Subroutines for a description of the default
action.
Typically, you assign the result of speculation
to a thread-local variable and then use that variable as a
subsequent predicate to other probes, as well as an argument to
speculate
, as shown in the following example:
syscall::openat:entry { self->spec = speculation(); } syscall::: /self->spec/ { speculate(self->spec); printf("this is speculative"); }
Committing a Speculation
You commit speculations by using the commit
function. When a speculative buffer is committed, its data is
copied into the principal buffer. If there is more data in the
specified speculative buffer than there is available space in the
principal buffer, no data is copied and the drop count for the
buffer is incremented. If the buffer has been speculatively traced
on more than one CPU, the speculative data on the committing CPU
is copied immediately, while speculative data on other CPUs is
copied some time after the commit
. Thus, some
time might elapse between a commit
that begins
on one CPU, while the data is being copied from speculative
buffers to principal buffers on all CPUs. This length of time is
guaranteed to be no longer than the time dictated by the cleaning
rate. See Speculation Options and Tuning.
A committing speculative buffer is not made available to
subsequent speculation
calls until each per-CPU
speculative buffer has been completely copied into its
corresponding per-CPU principal buffer. Similarly, subsequent
calls to speculate
to the committing buffer are
silently discarded, and subsequent calls to
commit
or discard
silently
fail. Finally, a clause containing a commit
cannot contain a data recording action. However, a clause can
contain multiple commit
calls to commit
disjoint buffers.
Discarding a Speculation
You discard speculations by using the discard
function. When a speculative buffer is discarded, its contents are
also discarded. If the speculation has only been active on the CPU
calling discard
, the buffer is immediately
available for subsequent calls to speculation
.
If the speculation has been active on more than one CPU, the
discarded buffer will be available for subsequent
speculation
some time after the call to
discard
. The length of time between a
discard
on one CPU and the buffer being made
available for subsequent speculations is guaranteed to be no
longer than the time that is dictated by the cleaning rate. If, at
the time speculation
is called, no buffer is
available because all speculative buffers are currently being
discarded or committed, dtrace generates a
message similar to the following:
dtrace: 905 failed speculations (available buffer(s) still busy)
You can reduce the likelihood of all buffers being unavailable by tuning the number of speculation buffers or the cleaning rate. See Speculation Options and Tuning.
Example of a Speculation
One potential use for speculations is to highlight a particular
code path. The following example shows the entire code path under
the open()
system call when the call fails.
Type the following source code and save it in a file named
specopen.d
:
#!/usr/sbin/dtrace -Fs syscall::open:entry { /* * The call to speculation() creates a new speculation. If this fails, * dtrace will generate an error message indicating the reason for * the failed speculation(), but subsequent speculative tracing will be * silently discarded. */ self->spec = speculation(); speculate(self->spec); /* * Because this printf() follows the speculate(), it is being * speculatively traced; it will only appear in the data buffer if the * speculation is subsequently committed. */ printf("%s", copyinstr(arg0)); } syscall::open:return /self->spec/ { /* * To balance the output with the -F option, we want to be sure that * every entry has a matching return. Because we speculated the * open entry above, we want to also speculate the open return. * This is also a convenient time to trace the errno value. */ speculate(self->spec); trace(errno); } syscall::open:return /self->spec && errno != 0/ { /* * If errno is non-zero, we want to commit the speculation. */ commit(self->spec); self->spec = 0; } syscall::open:return /self->spec && errno == 0/ { /* * If errno is not set, we discard the speculation. */ discard(self->spec); self->spec = 0; }
Running the previous script produces output similar to the following:
# ./specopen.d dtrace: script ā./specopen.dā matched 4 probes CPU FUNCTION 1 => open /var/ld/ld.config 1 <= open 2 1 => open /images/UnorderedList16.gif 1 <= open 4 ...
Speculation Options and Tuning
If a speculative buffer is full when a speculative tracing action is attempted, no data is stored in the buffer and a drop count is incremented. In this situation, dtrace generates a message similar to the following:
dtrace: 38 speculative drops
Speculative drops do not prevent the full speculative buffer from
being copied into the principal buffer when it is committed.
Similarly, speculative drops can occur even if drops were
experienced on a speculative buffer that were ultimately
discarded. Speculative drops can be reduced by increasing the
speculative buffer size, which is tuned by using the
specsize
option. The
specsize
option can be specified with any size
suffix. The resizing policy of this buffer is dictated by the
bufresize
option.
Speculative buffers might be unavailable when
speculation
is called. If buffers that have not
yet been committed or discards exist, dtrace
generates a message similar to the following:
dtrace: 1 failed speculation (no speculative buffer available)
You can reduce the likelihood of failed speculations of this
nature by increasing the number of speculative buffers by
specifying the nspec
option. The value of
nspec
defaults to 1
.
Also, speculation
can fail if all speculative
buffers are busy. In this case, an error message similar to the
following is displayed:
dtrace: 1 failed speculation (available buffer(s) still busy)
This error message indicates that speculation
was called after commit
was called for a
speculative buffer, but before that buffer was actually committed
on all CPUs. You can reduce the likelihood of failed speculations
of this nature by increasing the rate at which CPUs are cleaned by
using the cleanrate
option. The value of
cleanrate
defaults to 101
.