Alibaba Cloud Service Mesh:Trace applications inside and outside an ASM instance with Managed Service for OpenTelemetry

How it works

1. ExternalProxy runs outside the mesh. When it receives a request, it creates a root span, generates B3 trace headers, and forwards the request to the ASM ingress gateway.

2. The ingress gateway and sidecar proxies inside the mesh propagate the B3 headers and generate their own spans.

3. Both ExternalProxy and the mesh-side proxies report spans to Managed Service for OpenTelemetry through the Zipkin V1 API.

4. Managed Service for OpenTelemetry correlates spans by trace ID and assembles the complete call trace.

Step 1: Deploy the Bookinfo application inside the ASM instance

Deploy the Bookinfo sample application and its networking resources into the ACK cluster managed by the ASM instance.

1. Deploy the Bookinfo application in the ACK cluster. Download the YAML file from GitHub, then run:

      kubectl --kubeconfig=${DATA_PLANE_KUBECONFIG} apply -f bookinfo.yaml

2. Deploy virtual services for the Bookinfo application. Download the YAML file from GitHub, then run:

      kubectl --kubeconfig=${ASM_KUBECONFIG} apply -f virtual-service-all-v1.yaml

3. Deploy destination rules for the Bookinfo application. Download the YAML file from GitHub, then run:

      kubectl --kubeconfig=${ASM_KUBECONFIG} apply -f destination-rule-all.yaml

4. Deploy a gateway for the Bookinfo application. Download the YAML file from GitHub, then run:

      kubectl --kubeconfig=${ASM_KUBECONFIG} apply -f bookinfo-gateway.yaml

Replace the following placeholders with your actual values:

Step 2: Get the Managed Service for OpenTelemetry endpoint

Retrieve the Zipkin V1 endpoint that ExternalProxy uses to report spans.

1. Log on to the Managed Service for OpenTelemetry console. In the left-side navigation pane, click Overview.

2. On the Overview page, click the Access process tab, and then click View access point information.

3. In the upper part of the page, select the region where the ACK cluster is deployed.

   Important

   Select the same region as the ACK cluster. If you select a different region, tracing data from the Bookinfo application cannot reach Managed Service for OpenTelemetry.

4. On the Access point information tab, turn on Show Token for the Cluster Information parameter. Copy the public or private endpoint for Zipkin V1 based on where ExternalProxy runs:

   • Public endpoint -- Use this if ExternalProxy runs outside the Alibaba Cloud network (for example, on your local machine).

   • Private endpoint -- Use this if ExternalProxy runs within the same Alibaba Cloud VPC.

   

Step 3: Deploy the ExternalProxy application

ExternalProxy is a Python Flask application that creates a Zipkin span for each incoming request and forwards it to the Bookinfo Productpage service through the ASM ingress gateway.

1. Create a file named ExternalProxy.py with the following content: Before running the application, replace the following placeholders:

   Placeholder	Description
   ${XTRACE_ZIPKIN_V1_ENDPOINT}	Zipkin V1 endpoint from Step 2. If you use a custom tracing system, use your own endpoint instead
   ${INGRESS_GATE_WAY_IP}	IP address or hostname of the ASM ingress gateway

   Note

   The sample_rate=100 setting captures 100% of requests. This is suitable for testing and validation. In production, lower the sample rate (for example, to 1 or 10) to reduce performance overhead and tracing costs.

   Expand to view ExternalProxy.py

      import requests
      from flask import Flask
      from py_zipkin.zipkin import zipkin_span,create_http_headers_for_new_span
      from py_zipkin.util import generate_random_64bit_string
      from py_zipkin.util import ZipkinAttrs
      import time
   
      app = Flask(__name__)
   
      def do_stuff(trace_id, span_id, parent_span_id):
          time.sleep(2)
          headers = create_http_headers_for_new_span()
          headers["X-B3-TraceId"] = trace_id
          headers["X-B3-SpanId"] = span_id
          headers["X-B3-ParentSpanId"] = parent_span_id
          print "SEND TO INGRESS HEADERS : {0}".format(headers)
          r = requests.get('http://${INGRESS_GATE_WAY_IP}/productpage', headers=headers)
          return 'OK'
   
   
      def http_transport(encoded_span):
          # encoding prefix explained in https://github.com/Yelp/py_zipkin#transport
          body=encoded_span
          zipkin_url="${XTRACE_ZIPKIN_V1_ENDPOINT}"
          headers = {"Content-Type": "application/x-thrift"}
   
          # You'd probably want to wrap this in a try/except in case POSTing fails
          r=requests.post(zipkin_url, data=body, headers=headers)
          print(body)
   
   
      @app.route('/')
      def index():
          with zipkin_span(
              service_name='external-proxy',
              span_name='external-proxy/inbound',
              transport_handler=http_transport,
              port=5000,
              sample_rate=100,
          ) as inbound_span:
          do_stuff(inbound_span.zipkin_attrs.trace_id, inbound_span.zipkin_attrs.span_id, inbound_span.zipkin_attrs.parent_span_id)
   
          return 'OK', 200
   
      if __name__=='__main__':
          app.run(host="0.0.0.0",port=5000,debug=True)

2. Start ExternalProxy: Expected output:

      python ExternalProxy.py

       * Serving Flask app "main" (lazy loading)
       * Environment: production
         WARNING: This is a development server. Do not use it in a production deployment.
         Use a production WSGI server instead.
       * Debug mode: on
       * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
       * Restarting with stat
       * Debugger is active!
       * Debugger PIN: 317-792-686

3. Send a test request to ExternalProxy: Expected output: This confirms that ExternalProxy successfully called the Bookinfo Productpage service through the ingress gateway.

      curl localhost:5000

      OK

Step 4: View the trace

Both ExternalProxy and the mesh-side services report spans to Managed Service for OpenTelemetry. Verify the end-to-end trace in the console.

1. Log on to the Managed Service for OpenTelemetry console.

2. In the left-side navigation pane, click Applications. In the upper part of the page, select the region where the ACK cluster resides, then click ExternalProxy in the application list.

3. In the left-side navigation pane, click Application Details. Click the Traces tab, then click a trace ID to view the full call trace. The trace shows the complete request path: ExternalProxy (outside the mesh) to the Bookinfo Productpage service (inside the mesh), with each hop represented as a span. A connected end-to-end trace confirms that header propagation is working correctly.

Troubleshooting

Related topics

• Create an ASM instance

• Add a cluster to an ASM instance

• Create an ingress gateway

• Collect ASM tracing data to Managed Service for OpenTelemetry