Application Real-Time Monitoring Service:Filter specific spans with custom samplers

Prerequisites

Before you begin, make sure that you have:

• An application automatically instrumented with OpenTelemetry Java Agent. For setup instructions, see Use OpenTelemetry to submit the trace data of Java applications

• Apache Maven installed for building the extension JAR

Step 1: Create a Maven project

Create a new Maven project to build the agent extension.

Step 2: Add dependencies

Add the following dependencies to your pom.xml file.

<dependency>
  <groupId>com.google.auto.service</groupId>
  <artifactId>auto-service</artifactId>
  <version>1.1.1</version>
</dependency>

<dependency>
  <groupId>io.opentelemetry.javaagent</groupId>
  <artifactId>opentelemetry-javaagent</artifactId>
  <version>1.28.0</version>
  <!--Set the scope to compile.-->
  <scope>compile</scope>
</dependency>

<dependency>
  <groupId>io.opentelemetry</groupId>
  <artifactId>opentelemetry-sdk-trace</artifactId>
  <version>1.28.0</version>
</dependency>

<dependency>
  <groupId>io.opentelemetry</groupId>
  <artifactId>opentelemetry-sdk-extension-autoconfigure</artifactId>
  <version>1.28.0</version>
</dependency>

<dependency>
  <groupId>io.opentelemetry</groupId>
  <artifactId>opentelemetry-semconv</artifactId>
  <version>1.28.0-alpha</version>
</dependency>

Step 3: Implement the sampler

Create a class that implements the io.opentelemetry.sdk.trace.samplers.Sampler interface. Define your filtering rules in the shouldSample method and return a sampler name from getDescription.

• shouldSample -- Evaluate each span and return SamplingResult.create(SamplingDecision.DROP) for spans to discard, or SamplingResult.create(SamplingDecision.RECORD_AND_SAMPLE) for spans to keep.

• getDescription -- Return the sampler name.

The following example drops spans named spanName1 or spanName2, and spans with an http.target attribute of /api/checkHealth or /health/checks:

package org.example;

import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.context.Context;
import io.opentelemetry.sdk.trace.data.LinkData;
import io.opentelemetry.sdk.trace.samplers.Sampler;
import io.opentelemetry.sdk.trace.samplers.SamplingDecision;
import io.opentelemetry.sdk.trace.samplers.SamplingResult;
import io.opentelemetry.semconv.trace.attributes.SemanticAttributes;

import java.util.*;

public class SpanFilterSampler implements Sampler {

    // Span names to drop
    private static List<String> EXCLUDED_SPAN_NAMES = Collections.unmodifiableList(
        Arrays.asList("spanName1", "spanName2")
    );

    // HTTP target paths to drop
    private static List<String> EXCLUDED_HTTP_REQUEST_TARGETS = Collections.unmodifiableList(
        Arrays.asList("/api/checkHealth", "/health/checks")
    );

    @Override
    public SamplingResult shouldSample(Context parentContext, String traceId, String name,
            SpanKind spanKind, Attributes attributes, List<LinkData> parentLinks) {

        String httpTarget = attributes.get(SemanticAttributes.HTTP_TARGET) != null
            ? attributes.get(SemanticAttributes.HTTP_TARGET) : "";

        if (EXCLUDED_SPAN_NAMES.contains(name)
                || EXCLUDED_HTTP_REQUEST_TARGETS.contains(httpTarget)) {
            return SamplingResult.create(SamplingDecision.DROP);
        }
        return SamplingResult.create(SamplingDecision.RECORD_AND_SAMPLE);
    }

    @Override
    public String getDescription() {
        return "SpanFilterSampler";
    }
}

Step 4: Implement the sampler provider

Create a class that implements io.opentelemetry.sdk.autoconfigure.spi.traces.ConfigurableSamplerProvider. This provider registers your sampler with the Java Agent's auto-configuration mechanism.

• createSampler -- Return an instance of your sampler.

• getName -- Return the sampler name. The Java Agent uses this name to locate the sampler.

package org.example;

import com.google.auto.service.AutoService;
import io.opentelemetry.sdk.autoconfigure.spi.ConfigProperties;
import io.opentelemetry.sdk.autoconfigure.spi.traces.ConfigurableSamplerProvider;
import io.opentelemetry.sdk.trace.samplers.Sampler;

@AutoService(ConfigurableSamplerProvider.class)
public class SpanFilterSamplerProvider implements ConfigurableSamplerProvider {

    @Override
    public Sampler createSampler(ConfigProperties configProperties) {
        return new SpanFilterSampler();
    }

    @Override
    public String getName() {
        return "SpanFilterSampler";
    }
}

Step 5: Build the extension JAR

Package the project into a JAR file:

mvn clean package

The JAR file is generated in the target directory.

Step 6: Load the extension at startup

Specify your custom sampler when starting the application. Use either a JVM system property or an environment variable.

Option A: JVM system property

Add -Dotel.traces.sampler=<your-sampler-name> to the JVM startup parameters. Replace <your-sampler-name> with the value returned by the getName method.

java -javaagent:path/to/opentelemetry-javaagent.jar \
  -Dotel.javaagent.extensions=path/to/opentelemetry-java-agent-extension.jar \
  -Dotel.traces.sampler=<your-sampler-name> \
  -Dotel.exporter.otlp.headers=Authentication=<token> \
  -Dotel.exporter.otlp.endpoint=<endpoint> \
  -Dotel.metrics.exporter=none \
  -jar yourapp.jar

Option B: Environment variable

Set the OTEL_TRACES_SAMPLER environment variable to your sampler name:

export OTEL_JAVAAGENT_EXTENSIONS="path/to/opentelemetry-java-agent-extension.jar"
export OTEL_TRACES_SAMPLER="<your-sampler-name>"
export OTEL_EXPORTER_OTLP_HEADERS="Authentication=<token>"
export OTEL_EXPORTER_OTLP_ENDPOINT="<endpoint>"
export OTEL_METRICS_EXPORTER="none"

java -javaagent:path/to/opentelemetry-javaagent.jar \
  -jar yourapp.jar

Replace the following placeholders with your actual values:

Prerequisites

Before you begin, make sure that you have:

• An application manually instrumented with OpenTelemetry SDK for Java. For setup instructions, see Use OpenTelemetry to submit the trace data of Java applications

Step 1: Create the sampler class

Create a SpanFilterSampler class that implements the Sampler interface. The implementation is the same as the agent extension sampler in Step 3: Implement the sampler. Adapt the EXCLUDED_SPAN_NAMES and EXCLUDED_HTTP_REQUEST_TARGETS lists to match the spans you want to drop.

Step 2: Register the sampler with SdkTracerProvider

Call .setSampler(new SpanFilterSampler()) when building the SdkTracerProvider instance:

SdkTracerProvider sdkTracerProvider = SdkTracerProvider.builder()
    .setSampler(new SpanFilterSampler())  // Register the custom sampler
    .addSpanProcessor(BatchSpanProcessor.builder(OtlpGrpcSpanExporter.builder()
        .setEndpoint("<endpoint>")
        .addHeader("Authentication", "<token>")
        .build()).build())
    .setResource(resource)
    .build();

Step 3: Restart the application

Restart your application. The sampler evaluates every new span and drops those matching the filter rules.

Step 1: Create a sampler class

Create a class that implements the Sampler interface. Define your filtering logic in shouldSample:

const opentelemetry = require('@opentelemetry/api');

class SpanFilterSampler {
  shouldSample(spanContext, parentContext) {
    // Implement your custom sampling logic here.
  }
}

Step 2: Register the sampler with NodeTracerProvider

Pass the sampler instance to the NodeTracerProvider constructor:

const provider = new NodeTracerProvider({
  sampler: new SpanFilterSampler(),  // Register the custom sampler
  resource: new Resource({
    [SemanticResourceAttributes.HOST_NAME]: require("os").hostname(),
    [SemanticResourceAttributes.SERVICE_NAME]: "<your-service-name>",
  }),
});

Replace <your-service-name> with the name of your service, for example, my-node-app.

Start the application after updating the provider configuration.