All Products
Search
Document Center

Log Service:Import trace data from Python applications to Simple Log Service by using OpenTelemetry SDK for Python

Last Updated:Aug 25, 2023

This topic describes how to import trace data from Python applications to Simple Log Service by using OpenTelemetry SDK for Python.

Prerequisites

  • A trace instance is created. For more information, see Create a trace instance.

  • A Python development environment is set up. The Python version is 3.7 or later.

  • OpenTelemetry SDK for Python is installed in your Python development environment.

    If OpenTelemetry SDK for Python is not installed, you can run the following commands to install the SDK:

    pip install opentelemetry-api==1.12.0
    pip install opentelemetry-sdk==1.12.0
    pip install opentelemetry-exporter-otlp==1.12.0

Procedure

  1. Initialize an OpenTelemetry provider.

  2. Check whether the conditions for importing data in semi-automatic mode are met.

    • If the conditions are met, you can import trace data in semi-automatic mode.

      For scenarios in which trace data cannot be imported in semi-automatic mode, you must import the data in manual mode.

    • If the conditions are not met, you can import trace data in manual mode.

Step 1: Initialize an OpenTelemetry provider

You can initialize an OpenTelemetry provider by using the following code. You must replace the variables in the code with the actual values. For more information about the variables, see Variables.

# For Opentelemetry
import socket
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.trace.export import ConsoleSpanExporter
from opentelemetry.sdk.trace.export import SimpleSpanProcessor


class OpenTelemetrySLSProvider(object):

    def __init__(self, namespace="", service="", version="", endpoint='stdout',
                 project=None, instance=None, access_key_id=None, access_key_secret=None):
        '''
        :param namespace: Your service namespace
        :param service: Your Application Service Name
        :param version: Your Application Version
        :param endpoint: console or https://sls_endpoint:10010
        :param project: SLS project
        :param instance: SLS OTEL InstanceId
        :param access_key_id: Aliyun AccesskeyId
        :param access_key_secret:  Aliyun AccesskeySecret
        '''
        self.sls_otel_endpoint = endpoint
        self.sls_otel_project = project
        self.sls_otel_akid = access_key_id
        self.sls_otel_aksecret = access_key_secret
        self.sls_otel_instanceid = instance
        self.local_mode = False
        if endpoint == "stdout":
            self.local_mode = True
            self.resource = Resource(attributes={
                "host.name": socket.gethostname(),
                "service.name": service,
                "service.namespace": namespace,
                "service.version": version})
        else:
            self.resource = Resource(attributes={
                "host.name": socket.gethostname(),
                "service.namespace": namespace,
                "service.name": service,
                "service.version": version,
                "sls.otel.project": self.sls_otel_project,
                "sls.otel.akid": self.sls_otel_akid,
                "sls.otel.aksecret": self.sls_otel_aksecret,
                "sls.otel.instanceid": self.sls_otel_instanceid
            })

    def initTracer(self):
        trace.set_tracer_provider(TracerProvider(resource=self.resource))
        if self.local_mode:
            trace.get_tracer_provider().add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
        else:
            otlp_exporter = OTLPSpanExporter(endpoint=self.sls_otel_endpoint)
            trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(otlp_exporter))

# debug mode
#sls_ot_provider = OpenTelemetrySLSProvider(service="example", version="v0.1.0")

# write to sls
sls_ot_provider = OpenTelemetrySLSProvider(namespace="${service.namespace}", service="${service}", version="${version}",
                                         endpoint='${endpoint}',
                                         project="${project}",
                                         instance="${instance}",
                                         access_key_id="${access-key-id}",
                                         access_key_secret="${access-key-secret}"
                                         )
Table 1. Variables

Variable

Description

Example

${service.namespace}

The namespace to which the service belongs.

order

${service}

The name of the service. Specify the value based on your business requirements.

payment

${version}

The version of the service. We recommend that you specify a version in the va.b.c format.

v0.1.2

${endpoint}

The endpoint of the Simple Log Service project. Format: https://${project}.${region-endpoint}:Port.

  • ${project}: the name of the Simple Log Service project.

  • ${region-endpoint}: the Simple Log Service endpoint for the region where the project resides. You can access Simple Log Service by using an internal or public endpoint. An internal endpoint can be accessed over the classic network or a virtual private cloud (VPC). A public endpoint can be accessed over the Internet. For more information, see Endpoints.

  • Port: the port number. The value is fixed as 10010.

Note
  • If you set the variable to stdout (endpoint='stdout'), data is displayed to the standard output.

  • If you leave the variable empty, trace data is not uploaded to Simple Log Service.

https://test-project.cn-hangzhou.log.aliyuncs.com:10010

${project}

The name of the Simple Log Service project.

test-project

${instance}

The ID of the trace instance. For more information, see Create a trace instance.

test-traces

${access-key-id}

The AccessKey ID of your Alibaba Cloud account.

We recommend that you use the AccessKey pair of a RAM user that has only the write permissions on the Simple Log Service project. An AccessKey pair consists of an AccessKey ID and an AccessKey secret. For more information about how to grant the write permissions on a specified project to a RAM user, see Use custom policies to grant permissions to a RAM user. For more information about how to obtain an AccessKey pair, see AccessKey pair.

None

${access-key-secret}

The AccessKey secret of your Alibaba Cloud account.

We recommend that you use the AccessKey pair of a RAM user that has only the write permissions on the Simple Log Service project.

None

Step 2: Import data

You can import trace data to Simple Log Service in semi-automatic or manual mode. OpenTelemetry SDK for Python provides various types of instrumentation packages and supports the semi-automatic instrumentation of common frameworks. If you use one of the instrumentation packages, you must import data in semi-automatic mode. For more information, visit Instrumentation packages.

  • Semi-automatic instrumentation

    In this example, the flask and requests instrumentation packages are used.

    1. Install the instrumentation packages.

      pip install requests
      pip install flask
      
      pip install opentelemetry-instrumentation-flask
      
      pip install opentelemetry-instrumentation-requests
    2. Run code.

      Replace the variables in the following code with the actual values. For more information, see Variables.

      # for flask
      import flask
      import requests
      
      # for Opentelemetry instrumentation
      import socket
      from opentelemetry.instrumentation.flask import FlaskInstrumentor
      from opentelemetry.instrumentation.requests import RequestsInstrumentor
      
      # For Opentelemetry pip install opentelemetry-instrumentation-requests
      from opentelemetry import trace
      from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import (
          OTLPSpanExporter,
      )
      from opentelemetry.sdk.resources import Resource
      from opentelemetry.sdk.trace import TracerProvider
      from opentelemetry.sdk.trace.export import BatchSpanProcessor
      from opentelemetry.sdk.trace.export import ConsoleSpanExporter
      from opentelemetry.sdk.trace.export import SimpleSpanProcessor
      
      
      class OpenTelemetrySLSProvider(object):
      
          def __init__(self, namespace="", service="", version="", endpoint='stdout',
                       project=None, instance=None, access_key_id=None, access_key_secret=None):
              '''
              :param namespace: Your service namespace
              :param service: Your Application Service Name
              :param version: Your Application Version
              :param endpoint: console or https://sls_endpoint:10010
              :param project: SLS project
              :param instance: SLS OTEL InstanceId
              :param access_key_id: Aliyun AccesskeyId
              :param access_key_secret:  Aliyun AccesskeySecret
              '''
              self.sls_otel_endpoint = endpoint
              self.sls_otel_project = project
              self.sls_otel_akid = access_key_id
              self.sls_otel_aksecret = access_key_secret
              self.sls_otel_instanceid = instance
              self.local_mode = False
              if endpoint == "stdout":
                  self.local_mode = True
                  self.resource = Resource(attributes={
                      "host.name": socket.gethostname(),
                      "service.name": service,
                      "service.namespace": namespace,
                      "service.version": version})
              else:
                  self.resource = Resource(attributes={
                      "host.name": socket.gethostname(),
                      "service.name": service,
                      "service.version": version,
                      "service.namespace": namespace,
                      "sls.otel.project": self.sls_otel_project,
                      "sls.otel.akid": self.sls_otel_akid,
                      "sls.otel.aksecret": self.sls_otel_aksecret,
                      "sls.otel.instanceid": self.sls_otel_instanceid
                  })
      
          def initTracer(self):
              trace.set_tracer_provider(TracerProvider(resource=self.resource))
              if self.local_mode:
                  trace.get_tracer_provider().add_span_processor(
                      SimpleSpanProcessor(ConsoleSpanExporter()))
              else:
                  otlp_exporter = OTLPSpanExporter(endpoint=self.sls_otel_endpoint)
                  trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(otlp_exporter))
      
      
      # write to sls
      sls_ot_provider = OpenTelemetrySLSProvider(namespace="${service.namespace}", service="${service}", version="${version}",
                                                 endpoint='${endpoint}',
                                                 project="${project}",
                                                 instance="${instance}",
                                                 access_key_id="${access-key-id}",
                                                 access_key_secret="${access-key-secret}"
                                                 )
      # for console debug
      #sls_ot_provider = OpenTelemetrySLSProvider(service="example", version="v0.1.0")
      
      sls_ot_provider.initTracer()
      
      # flask init
      app = flask.Flask(__name__)
      
      # instrumentation init
      FlaskInstrumentor().instrument_app(app)
      RequestsInstrumentor().instrument()
      
      
      @app.route("/")
      def hello():
          tracer = trace.get_tracer(__name__)
          with tracer.start_as_current_span("request_server"):
              requests.get("http://www.taobao.com")
          return "hello"
      
      
      app.run(debug=True, port=5000)
    3. Access the service to trigger trace data generation and send the trace data to Simple Log Service.

      127.0.0.1:5000/hello
  • Manual instrumentation

    Run the following code. Replace the variables in the code with the actual values. For more information about the code, see Variables.

    # ot-manual-example.py
    import time
    
    # For Opentelemetry
    import socket
    from opentelemetry import trace
    from pentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
    from opentelemetry.sdk.resources import Resource
    from opentelemetry.sdk.trace import TracerProvider
    from opentelemetry.sdk.trace.export import BatchSpanProcessor
    from opentelemetry.sdk.trace.export import ConsoleSpanExporter
    from opentelemetry.sdk.trace.export import SimpleSpanProcessor
    
    
    class OpenTelemetrySLSProvider(object):
    
        def __init__(self, namespace="", service="", version="", endpoint='stdout',
                     project=None, instance=None, access_key_id=None, access_key_secret=None):
            '''
            :param service: Your service namespace
            :param service: Your Application Service Name
            :param version: Your Application Version
            :param endpoint: console or https://sls_endpoint:10010
            :param project: SLS project
            :param instance: SLS OTEL InstanceId
            :param access_key_id: Aliyun AccesskeyId
            :param access_key_secret:  Aliyun AccesskeySecret
            '''
            self.sls_otel_endpoint = endpoint
            self.sls_otel_project = project
            self.sls_otel_akid = access_key_id
            self.sls_otel_aksecret = access_key_secret
            self.sls_otel_instanceid = instance
            self.local_mode = False
            if endpoint == "stdout":
                self.local_mode = True
                self.resource = Resource(attributes={
                    "host.name": socket.gethostname(),
                    "service.name": service,
                    "service.namespace": namespace,
                    "service.version": version})
            else:
                self.resource = Resource(attributes={
                    "host.name": socket.gethostname(),
                    "service.name": service,
                    "service.version": version,
                    "service.namespace": namespace,
                    "sls.otel.project": self.sls_otel_project,
                    "sls.otel.akid": self.sls_otel_akid,
                    "sls.otel.aksecret": self.sls_otel_aksecret,
                    "sls.otel.instanceid": self.sls_otel_instanceid
                })
    
        def initTracer(self):
            trace.set_tracer_provider(TracerProvider(resource=self.resource))
            if self.local_mode:
                trace.get_tracer_provider().add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
            else:
                otlp_exporter = OTLPSpanExporter(endpoint=self.sls_otel_endpoint)
                trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(otlp_exporter))
    
    # write to sls
    sls_ot_provider = OpenTelemetrySLSProvider(namespace="${service.namespace}", service="${service}", version="${version}",
                                               endpoint='${endpoint}',
                                               project="${project}",
                                               instance="${instance}",
                                               access_key_id="${access-key-id}",
                                               access_key_secret="${access-key-secret}"
                                               )
    
    # for console debug
    #sls_ot_provider = OpenTelemetrySLSProvider(service="example", version="v0.1.0")
    
    # Trace Example
    sls_ot_provider.initTracer()
    tracer = trace.get_tracer(__name__)
    
    with tracer.start_as_current_span("foo"):
        print("Hello world!")
    
    
    labels = {"environment": "staging"}
    
    time.sleep(60)

FAQ

How do I check whether OpenTelemetry SDK for Python is installed as expected?

You can check whether the SDK is installed as expected by using the following sample code. Save the code as a tracing.py file. Then, run the tracing.py command. If the command output is returned as expected, the related dependencies of OpenTelemetry SDK for Python are installed.

# tracing-example-1.py
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
    ConsoleSpanExporter,
    SimpleSpanProcessor,
)

trace.set_tracer_provider(TracerProvider())
trace.get_tracer_provider().add_span_processor(
    SimpleSpanProcessor(ConsoleSpanExporter())
)

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("foo"):
    print("Hello world!")
python trace

What to do next