OpenTelemetry

Modern computing has seen more distributed systems scaled and more services added, such as with microservices and serverless applications. With this growth, service ownership across the system is allocated to different individuals, or even organizations. It is increasingly difficult to observe how these services depend on each and affect other services without a unified observability framework. This becomes clear after a deployment or during an outage when issue identification requires speed and accuracy.

OpenTelemetry is such a unified observability framework. It is a popular open-source observability framework for instrumenting, generating, collecting, and exporting telemetry data. It provides a common specification and protocol so that multiple services can furnish a unified version of traces, metrics, and logs. OpenTelemetry relies on components that participate in the observability model to be instrumented, which means the code emits traces, metrics, and logs.

Numerous managed ODP.NET and ODP.NET Core APIs have been instrumented to support OpenTelemetry observability and standards. Developers and operators can then customize the ODP.NET metrics collected.

ODP.NET OpenTelemetry Traces

Traces record the request paths taken either by an application or end user as they propagate through a multi-service architecture. Traces in OpenTelemetry are defined implicitly by their spans. A trace can be thought of as a directed acyclic graph of spans where the edges between spans are defined as parent/child relationships.

A span, or activity, represents an operation within a transaction. Each span encapsulates the following state:

Display name
ODP.NET NuGet version
Start timestamp and duration
Attributes – list of key-value pairs
Events – a tuple (timestamp, name, attributes) and can number from zero or more
The parent span identifier
Links to causally related spans through SpanContext, which can number from zero or more

ODP.NET spans become child spans if the application creates a (parent) span before calling ODP.NET instrumented APIs.

ODP.NET OpenTelemetry publishes the following activity tags, or attributes.

Table 3-43 ODP.NET OpenTelemetry activity tags

Attribute Name	Description
`Activity.DisplayName`	Activity display name or the name of the operation being instrumented.
`db.name`	Database being accessed by the command. The attribute value is the same as the `OracleConnection` `DatabaseName` property.
`db.odp.connection.id`	Gives each connection a unique identifier for diagnostic purposes. This identifier can be prefixed with a custom application generated string by setting the `ConnectionIdPrefix` property in `OracleConnection` or `OracleCommand`.
`db.odp.rows_affected`	Row count affected by the SQL DML (`INSERT`, `UPDATE`, and `DELETE` only) when using `ExecuteNonQuery`, `ExecuteReader`, or `ExecuteScalar*` methods. Does not return a value for stored procedures and anonymous PL/SQL.
`db.odp.sql_id`	Oracle database SQL statement identifier value, equivalent to `SQL_ID`
`db.odp.user.statement`	Redacted ODP.NET user-supplied command text which can differ from the database statement being executed, such as the case with stored procedures and XML commands
`db.statement`	Redacted database statement being executed
`db.system`	Database name. `Oracle` is the attribute value for Oracle databases.
`db.user`	Database user
`exception.message`	Exception message
`exception.stacktrace`	Exception stack trace
`exception.type`	Exception type
`otel.status_code`	Status code of the trace
`otel.status_description`	Status description of the trace
`server.address`	Name of the database host
`server.port`	Server port number

ODP.NET OpenTelemetry Trace Sample Exporter Visualization:


Activity.TraceId:            29399ee7598dea88defd7c5421bc41d4
Activity.SpanId:             e1878b5fc4435f23
Activity.TraceFlags:         Recorded
Activity.ParentSpanId:       8487a50b9ace7808
Activity.ActivitySourceName: Oracle.ManagedDataAccess.Core
Activity.ActivitySourceVersion: 23.6.0
Activity.DisplayName:        Connect MYHOSTNAME:1521:dbview
Activity.Kind:               Client
Activity.StartTime:          2024-09-19T07:26:16.4615973Z
Activity.Duration:           00:00:00.4114888
Activity.Tags:
    db.system: oracle
    db.user: scott
    db.name: dbview
    server.address: MYHOSTNAME
    server.port: 1521
StatusCode: Ok
Resource associated with Activity:
    service.name: DemoApp
    service.version: 1.0.0
    service.instance.id: 312ec47d-c0a1-4df3-b59f-f38aa225c39d
    telemetry.sdk.name: opentelemetry
    telemetry.sdk.language: dotnet
    telemetry.sdk.version: 1.9.0

Activity.TraceId:            29399ee7598dea88defd7c5421bc41d4
Activity.SpanId:             8487a50b9ace7808
Activity.TraceFlags:         Recorded
Activity.ParentSpanId:       95b69d7844ae0dd3
Activity.ActivitySourceName: Oracle.ManagedDataAccess.Core
Activity.ActivitySourceVersion: 23.6.0
Activity.DisplayName:        Open MYHOSTNAME:1521:dbview
Activity.Kind:               Client
Activity.StartTime:          2024-09-19T07:26:16.3584252Z
Activity.Duration:           00:00:00.5781012
Activity.Tags:
    db.system: oracle
    db.user: scott
    db.name: dbview
    server.address: MYHOSTNAME
    server.port: 1521
StatusCode: Ok
Resource associated with Activity:
    service.name: DemoApp
    service.version: 1.0.0
    service.instance.id: 312ec47d-c0a1-4df3-b59f-f38aa225c39d
    telemetry.sdk.name: opentelemetry
    telemetry.sdk.language: dotnet
    telemetry.sdk.version: 1.9.0

Activity.TraceId:            29399ee7598dea88defd7c5421bc41d4
Activity.SpanId:             89b3cd0b7b2d2f35
Activity.TraceFlags:         Recorded
Activity.ParentSpanId:       e0cf4916dc1e4328
Activity.ActivitySourceName: Oracle.ManagedDataAccess.Core
Activity.ActivitySourceVersion: 23.6.0
Activity.DisplayName:        SendExecuteRequest MYHOSTNAME:1521:dbview
Activity.Kind:               Client
Activity.StartTime:          2024-09-19T07:26:17.0031962Z
Activity.Duration:           00:00:00.0492725
Activity.Tags:
    db.system: oracle
    server.address: MYHOSTNAME
    server.port: 1521
    db.name: dbview
    db.user: scott
    db.statement: update Blogs set Blog_Name=? where Blog_Id=?
    db.odp.sql_id: 71jyh89d1cz9r
StatusCode: Ok
Resource associated with Activity:
    service.name: DemoApp
    service.version: 1.0.0
    service.instance.id: 312ec47d-c0a1-4df3-b59f-f38aa225c39d
    telemetry.sdk.name: opentelemetry
    telemetry.sdk.language: dotnet
    telemetry.sdk.version: 1.9.0

Activity.TraceId:            29399ee7598dea88defd7c5421bc41d4
Activity.SpanId:             e0cf4916dc1e4328
Activity.TraceFlags:         Recorded
Activity.ParentSpanId:       95b69d7844ae0dd3
Activity.ActivitySourceName: Oracle.ManagedDataAccess.Core
Activity.ActivitySourceVersion: 23.6.0
Activity.DisplayName:        ExecuteNonQuery MYHOSTNAME:1521:dbview
Activity.Kind:               Client
Activity.StartTime:          2024-09-19T07:26:16.9671882Z
Activity.Duration:           00:00:00.1279960
Activity.Tags:
    db.system: oracle
    server.address: MYHOSTNAME
    server.port: 1521
    db.name: dbview
    db.user: scott
    db.statement: update Blogs set Blog_Name=? where Blog_Id=?
    db.odp.rows_affected: 1
    db.odp.sql_id: 71jyh89d1cz9r
StatusCode: Ok
Resource associated with Activity:
    service.name: DemoApp
    service.version: 1.0.0
    service.instance.id: 312ec47d-c0a1-4df3-b59f-f38aa225c39d
    telemetry.sdk.name: opentelemetry
    telemetry.sdk.language: dotnet
    telemetry.sdk.version: 1.9.0

Activity.TraceId:            29399ee7598dea88defd7c5421bc41d4
Activity.SpanId:             8d2ebac56ee5c04f
Activity.TraceFlags:         Recorded
Activity.ParentSpanId:       95b69d7844ae0dd3
Activity.ActivitySourceName: Oracle.ManagedDataAccess.Core
Activity.ActivitySourceVersion: 23.6.0
Activity.DisplayName:        Close MYHOSTNAME:1521:dbview
Activity.Kind:               Client
Activity.StartTime:          2024-09-19T07:26:17.1450407Z
Activity.Duration:           00:00:00.0144354
Activity.Tags:
    db.system: oracle
    server.address: MYHOSTNAME
    server.port: 1521
    db.name: dbview
    db.user: scott
StatusCode: Ok
Resource associated with Activity:
    service.name: DemoApp
    service.version: 1.0.0
    service.instance.id: 312ec47d-c0a1-4df3-b59f-f38aa225c39d
    telemetry.sdk.name: opentelemetry
    telemetry.sdk.language: dotnet
    telemetry.sdk.version: 1.9.0

Activity.TraceId:            29399ee7598dea88defd7c5421bc41d4
Activity.SpanId:             95b69d7844ae0dd3
Activity.TraceFlags:         Recorded
Activity.ActivitySourceName: Test
Activity.DisplayName:        SampleTestActivity
Activity.Kind:               Internal
Activity.StartTime:          2024-09-19T07:26:16.2164456Z
Activity.Duration:           00:00:00.9785065
Resource associated with Activity:
    service.name: DemoApp
    service.version: 1.0.0
    service.instance.id: 312ec47d-c0a1-4df3-b59f-f38aa225c39d
    telemetry.sdk.name: opentelemetry
    telemetry.sdk.language: dotnet
    telemetry.sdk.version: 1.9.0

ODP.NET OpenTelemetry supports both dynamic and manual instrumentation. The next section describes how to set up each instrumentation method.

Dynamic and Manual Instrumentation

ODP.NET OpenTelemetry requires an exporter, such Jaeger or Zipkin, to visualize and analyze traces. These exporters are available as NuGet packages and will also include the OpenTelemetry SDK as a NuGet dependency.

ODP.NET OpenTelemetry manual instrumentation has the following requirements:

Oracle.ManagedDataAccess.OpenTelemetry package available from NuGet Gallery.
Exporter NuGet package, such as Console or Zipkin, which is used for instrumentation visualization.
OpenTelemetry SDK, which will be installed automatically with an exporter NuGet package.

Once the ODP.Net OpenTelemetry nuget package is installed, ODP.NET OpenTelemetry is enabled by invoking the TracerProviderBuilder AddOracleDataProviderInstrumentation extension method upon application startup. It accepts various options for configuring OpenTelemetry instrumentation.

Code sample: Enabling ODP.NET OpenTelemetry instrumentation with default options

using OpenTelemetry.Trace;

Sdk.CreateTracerProviderBuilder()
.AddOracleDataProviderInstrumentation()  // ODP.NET extension method 
.AddConsoleExporter()
.Build();

Code sample: Enabling ODP.NET OpenTelemetry instrumentation with all options enabled

using OpenTelemetry.Trace;

Sdk.CreateTracerProviderBuilder()
.AddOracleDataProviderInstrumentation(o =>
        {
          o.EnableConnectionLevelAttributes = true;
          o.RecordException = true;
          o.InstrumentOracleDataReaderRead = true;
          o.SetDbStatementForText = true;
        })
.AddConsoleExporter()
.Build();

After enabling ODP.NET OpenTelemetry, whenever a provider instrumented API is called, OpenTelemetry traces will be generated and sent to the configured exporter.

By default, instrumentation for potentially sensitive data is disabled, such as for SQL statements.

A second manual instrumentation method allows enabling ODP.NET OpenTelemetry without adding the ODP.NET OpenTelemetry NuGet package and without calling the AddOracleDataProviderInstrumentation() extension method. It requires adding the TracerProviderBuilder call AddSource(“Oracle.ManagedDataAccess.Core”) for ODP.NET Core or AddSource(“Oracle.ManagedDataAccess”) for managed ODP.NET.

ODP.NET OpenTelemetry instrumentation will then be enabled using default options. The defaults must be used as there is no way to modify option values. Changing these values require using ODP.NET OpenTelemetry NuGet package directly.

OpenTelemetry.Api NuGet package is installed automatically if the application adds any OpenTelemetry exporter NuGet packages, such as OpenTelemetry.Exporter.Console or OpenTelemetry.Exporter.Zipkin. OpenTelemetry.Api is necessary to call the AddSource() method.

Code sample: Enabling ODP.NET OpenTelemetry manual instrumentation via AddSource


Sdk.CreateTracerProviderBuilder()
         .AddSource("Oracle.ManagedDataAccess.Core")
         .AddConsoleExporter()
         .Build();

Through this instrumentation method, it is possible to enable and disable ODP.NET OpenTelemetry tracing programmatically at runtime dynamically. To enable, set OracleConfiguration OpenTelemetryTracing to true (default). To disable, set it to false.

OpenTelemetryTracing is available as a managed ODP.NET .NET configuration file setting as well. As a .NET config setting, it is read upon app startup only. Changes to this setting's value after startup are ignored.

Automatic Instrumentation

OpenTelemetry automatic instrumentation enables ODP.NET tracing without requiring code changes. Existing .NET apps can turn on tracing without having to recompile.

To use it, add the OpenTelemetry.AutoInstrumentation 1.7.0 NuGet package or higher to your project. Older package versions require setting environment variables to enable ODP.NET tracing. For example, the following environment variable settings turn on ODP.NET Core tracing using the console exporter.


OTEL_DOTNET_AUTO_TRACES_ADDITIONAL_SOURCES=Oracle.ManagedDataAccess.Core
OTEL_DOTNET_AUTO_TRACES_CONSOLE_EXPORTER_ENABLED=true

ODP.NET automatic instrumentation is on by default when using a newer OpenTelemetry.AutoInstrumentation NuGet package.

Instrumented ODP.NET APIs

The list of managed ODP.NET and ODP.NET Core APIs that are instrumented to support OpenTelemetry is as follows. Every ODP.NET database round trip originating from one of these APIs is instrumented as well.

OracleCommand
- ExecuteNonQuery()
- ExecuteNonQueryAsync(CancellationToken cancellationToken)
- ExecuteNonQueryAsync()
- All ExecuteReader overloads
- All ExecuteReaderAsync overloads
- ExecuteScalar()
- ExecuteScalarAsync(CancellationToken cancellationToken)
- ExecuteStream()
- ExecuteToStream(Stream)
- ExecuteXmlReader()
- All ExecuteXmlReaderAsync overloads
OracleConnection
- Close()
- Open()
- OpenAsync()
OracleDataAdapter
- All Fill overloads
OracleDataReader
- Read()
- ReadAsync()