Install a Python agent on ACK and ACS with ack-onepilot - Cloud Monitor

ack-onepilot component

The ack-onepilot component automatically prepares the Application Monitoring agent package and configures the agent's reporting environment within a container. For more information, see How the ack-onepilot component works.

After installation, ack-onepilot automatically upgrades the agent to the latest version when your application restarts. To manage agent versions manually instead, see Control agent versions.

Prerequisites

An ACK cluster or an ACS cluster is available.
ACK cluster: Create a dedicated ACK cluster (new creation suspended), Create a managed ACK cluster, or Create an ACK Serverless cluster.
ACS cluster: Create an ACS cluster.
A namespace is available. For more information, see Manage namespaces and quotas. In this topic, the arms-demo namespace is used.
Verify your Python and framework versions. For more information about the requirements, see Python components and frameworks supported by Application Monitoring.

Notes

If your application starts with uvicorn, choose one of the following configuration methods.
- Method 1: Add the following statement to the first line of the uvicorn entry file to import the Python agent.
```
from aliyun.opentelemetry.instrumentation.auto_instrumentation import sitecustomize
```
- Method 2: Replace uvicorn with gunicorn and add the aliyun-instrument prefix to the gunicorn command.
  
  Example:
```
uvicorn app:app --workers 4 --port=9090 --host 0.0.0.0
```
  Change the command to:
```
aliyun-instrument gunicorn -w 4 -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8000 app:app
```
  Note
  The aliyun-instrument command initializes the ARMS Python agent and enables zero-touch instrumentation.
If your application starts with uWSGI, see Integrate the Python agent when you start a Django or Flask project with uWSGI.
If you use gevent coroutines, you must set the GEVENT_ENABLE=true environment variable.

For example, if your code includes the following:
```
from gevent import monkey
monkey.patch_all()
```
You must set the environment variable as follows:
```
GEVENT_ENABLE=true
```

Step 1: Install the ack-onepilot component

Log on to the Container Service for Kubernetes (ACK) console and click the name of the target cluster on the Clusters page.
In the left-side navigation pane, click Add-ons, and then search for ack-onepilot in the upper-right corner.

Important
The ack-onepilot component must be version 5.1.0 or later. Version 5.1.0 automatically downloads and injects the Python agent package when you enable monitoring, providing zero-code instrumentation without manual modifications to the startup command. Version 5.1.0 is in canary release. To use it, contact our support team in the DingTalk group (ID: 159215000379).
On the ack-onepilot card, click Install.
Note
- By default, the ack-onepilot component supports up to 1,000 pods. For every additional 1,000 pods in the cluster, increase the CPU resources for ack-onepilot by 0.5 cores and the memory by 512 MB.
- If you are installing ack-onepilot in an ACS cluster, set the accessKey and accessKeySecret parameters to your AccessKey ID and AccessKey secret at the bottom of the installation card. For more information, see Create an AccessKey pair. Make sure that the Alibaba Cloud account has the AliyunARMSFullAccess and AliyunSTSAssumeRoleAccess permissions.
In the panel that appears, we recommend using the default settings and clicking OK.

Note
After installation, you can upgrade, configure, or uninstall the ack-onepilot component on the Add-ons page.

(Optional) Step 2: Modify the Dockerfile

Important

For ack-onepilot version 5.1.0 and later, the component automatically downloads and injects the Python agent package when you enable monitoring, providing zero-code instrumentation without manual modifications to the startup command.

Download the agent installer from the PyPI repository.
```
pip3 install aliyun-bootstrap
```
Use aliyun-bootstrap to install the agent.
```
# The region ID of your Alibaba Cloud account.
ARMS_REGION_ID=xxx aliyun-bootstrap -a install
```
Note
To install a specific version of the Python agent, run the following command:
```
# Replace ${version} with the actual version number.
aliyun-bootstrap -a install -v ${version}
```
For a list of all released Python agent versions, see Python Agent Release Notes.
Start the application with the ARMS Python agent.
```
aliyun-instrument python app.py
```
Build the image.

Dockerfile before modification

# Use the Python 3.10 base image
FROM docker.m.daocloud.io/python:3.10
# Set the working directory
WORKDIR /app
# Copy the requirements.txt file to the working directory
COPY requirements.txt .
# Use pip to install dependencies
RUN pip install --no-cache-dir -r requirements.txt
COPY ./app.py /app/app.py
# Expose port 8000 of the container
EXPOSE 8000
CMD ["python","app.py"]

Dockerfile after modification

# Use the official Python 3.10 base image
FROM docker.m.daocloud.io/python:3.10
# Set the working directory
WORKDIR /app
# Copy the requirements.txt file to the working directory
COPY requirements.txt .
# Use pip to install dependencies
RUN pip install --no-cache-dir -r requirements.txt
#########################Install the Aliyun Python agent###############################
# The region ID of your Alibaba Cloud account
RUN pip3 install aliyun-bootstrap && ARMS_REGION_ID=xxx aliyun-bootstrap -a install 
##########################################################
COPY ./app.py /app/app.py
# Expose port 8000 of the container
EXPOSE 8000
#########################################################
CMD ["aliyun-instrument","python","app.py"]

Step 3: Grant ARMS permissions

Managed ACK cluster

If your managed ACK cluster has an ARMS Addon Token, skip to Step 4. Otherwise, grant permissions manually as follows.

Check whether an ARMS Addon Token exists

Log on to the Container Service for Kubernetes (ACK) console and click the name of your cluster on the Clusters page.
In the left-side navigation pane, choose Configurations > Secrets. At the top of the page, select kube-system from the Namespace drop-down list and check if addon.arms.token exists.

Note

If an ARMS Addon Token exists, ARMS handles authentication automatically. Managed ACK clusters have this token by default, but some older clusters might not. Check for the token first; if it does not exist, grant permissions manually.

Manually add permission policies

Log on to the Container Service for Kubernetes (ACK) console and click the name of the target cluster on the Clusters page.
On the Cluster Information page, click the Basic Information tab. In the Cluster Resources section, click the link next to Worker RAM Role.
On the Permissions tab, click Add Permissions.
In the Add Permissions panel, add the following two permission policies, and then click OK.
- AliyunTracingAnalysisFullAccess: Grants full access to Tracing Analysis.
- AliyunARMSFullAccess: Grants full access to ARMS.

Dedicated and registered clusters

To monitor applications in a dedicated ACK cluster or a registered cluster from ACK One, ensure that the Alibaba Cloud account has the AliyunARMSFullAccess and AliyunSTSAssumeRoleAccess permissions. For more information, see Grant permissions to a RAM user.

After installing the ack-onepilot component, provide the AccessKey pair for an Alibaba Cloud account with ARMS permissions.

Method 1: Specify the AccessKey pair in Helm

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the target cluster. In the left-side navigation pane, choose Applications > Helm. Find the ack-onepilot component and click Update in the Actions column.
In the YAML editor, find the accessKey and accessKeySecret fields. Replace their values with your AccessKey ID and AccessKey secret, and then click OK.

Note
For more information about how to obtain an AccessKey pair, see Create an AccessKey pair.
Restart the application Deployment.

Method 2: Reference the AccessKey pair from a Secret

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the target cluster. In the left-side navigation pane, choose Configurations > Secrets.
Select the ack-onepilot namespace, and then create a Secret to store your AccessKey pair.

Note
For more information about how to obtain an AccessKey pair, see Create an AccessKey pair.

Set the Secret name to ack-onepilot-aksk and the type to Opaque. In the Data section, add two key-value pairs: ak for your AccessKey ID and sk for your AccessKey secret.
In the left-side navigation pane, choose Workloads > Stateless. Click the ack-onepilot component, which is typically named ack-onepilot-ack-onepilot and located in the ack-onepilot namespace.
In the upper-right corner of the ack-onepilot-ack-onepilot page, click Edit. In the Environment Variables section, add ONE_PILOT_ACCESSKEY and ONE_PILOT_ACCESSKEY_SECRET. Configure them to reference the values stored in the Secret, and then click Update.

Select the ack-onepilot-aksk Secret. The ONE_PILOT_ACCESSKEY variable must reference the ak key, and the ONE_PILOT_ACCESSKEY_SECRET variable must reference the sk key.

ACK Serverless and ECI clusters

To monitor applications in an ACK Serverless cluster or a cluster that uses Elastic Container Instance (ECI), grant permissions on the Cloud Resource Access Authorization page, and then restart all pods under the ack-onepilot component.

Step 4: Enable application monitoring

Note

An init container downloads and injects the Python agent package. With default resource settings, the init container takes about 10 seconds to run. It consumes 0.5 CPU cores and 250 MB of memory by default, which are released after initialization. You can adjust the resource quota and execution speed by using the following environment variables:

PYTHON_INIT_RESOURCE_REQUESTS_CPU: The CPU request for the Python application's init container. Example: 500m.
PYTHON_INIT_RESOURCE_REQUESTS_MEM: The memory request for the Python application's init container. Example: 250Mi.
PYTHON_INIT_RESOURCE_LIMIT_CPU: The CPU limit for the Python application's init container. Example: 1000m.
PYTHON_INIT_RESOURCE_LIMIT_MEM: The memory limit for the Python application's init container. Example: 500Mi.

Use the following YAML to create a stateless application (Deployment) with Application Monitoring enabled.

Expand to view the complete sample YAML file (Python)

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: arms-python-client
  name: arms-python-client
  namespace: arms-demo
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: arms-python-client
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: arms-python-client
        aliyun.com/app-language: python # Required for Python applications. Specifies that this is a Python application.
        armsPilotAutoEnable: 'on'
        armsPilotCreateAppName: "arms-python-client"    # The display name of the application in ARMS.
        armsPilotAppWorkspace: "workspace"   # If you do not specify a workspace, data is reported to the default workspace.
    spec:
      containers:
        - image: registry.cn-hangzhou.aliyuncs.com/arms-default/python-agent:arms-python-client
          imagePullPolicy: Always
          name: client
          resources:
            requests:
              cpu: 250m
              memory: 300Mi
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: arms-python-server
  name: arms-python-server
  namespace: arms-demo
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: arms-python-server
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: arms-python-server
        aliyun.com/app-language: python # Required for Python applications. Specifies that this is a Python application.
        armsPilotAutoEnable: 'on'
        armsPilotCreateAppName: "arms-python-server"    # The display name of the application in ARMS.
        armsPilotAppWorkspace: "workspace"  # Replace with the name of your workspace.
    spec:
      containers:
        - env:
          - name: CLIENT_URL
            value: 'http://arms-python-client-svc:8000'
        - image: registry.cn-hangzhou.aliyuncs.com/arms-default/python-agent:arms-python-server
          imagePullPolicy: Always
          name: server
          resources:
            requests:
              cpu: 250m
              memory: 300Mi
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: arms-python-server
  name: arms-python-server-svc
  namespace: arms-demo
spec:
  internalTrafficPolicy: Cluster
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack
  ports:
    - name: http
      port: 8000
      protocol: TCP
      targetPort: 8000
  selector:
    app: arms-python-server
  sessionAffinity: None
  type: ClusterIP
apiVersion: v1
kind: Service
metadata:
  name: arms-python-client-svc
  namespace: arms-demo
  uid: 91f94804-594e-495b-9f57-9def1fdc7c1d
spec:
  internalTrafficPolicy: Cluster
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack
  ports:
    - name: http
      port: 8000
      protocol: TCP
      targetPort: 8000
  selector:
    app: arms-python-client
  sessionAffinity: None
  type: ClusterIP

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the target cluster. In the left-side navigation pane, choose Workloads > Deployments.
On the Stateless page, find the target application and choose > View in YAML in the Actions column.

To create a new application, click Create from YAML.

In the YAML file, add the following labels to the spec.template.metadata section.

labels:
  aliyun.com/app-language: python # Required for Python applications. Specifies that this is a Python application.
  armsPilotAutoEnable: 'on'
  armsPilotCreateAppName: "deployment-name"    # The display name of the application in ARMS.
  armsPilotAppWorkspace: "workspace"    # Replace with the name of your workspace.

Important

Zero-code agent injection may modify the versions of some dependencies, such as Protobuf. To prevent errors from these changes, use one of the following methods.

If you have already installed a dependency such as Protobuf, prioritize your installed version by running the following commands.

# Check the package installation paths.
python3 -m site
# The output is similar to the following:
sys.path = [
    '/app',
    '/usr/local/lib/python3.9',
    '/usr/local/lib/python3.9/site-packages'
]
# Set the PYTHONPATH environment variable to prioritize your dependency versions.
# Use a colon (:) to separate different paths.
export PYTHONPATH=/app:/usr/local/lib/python3.9:/usr/local/lib/python3.9/site-packages

If Method 1 does not resolve the issue, manually install the Python agent. After installing the agent, force-reinstall your required dependency version:

pip3 install xx==${your_original_dependency_version} --force-reinstall

Important

If you do not want to use the zero-code injection capability for Python, you can disable it with the following label:

labels:
  armsAutoInstrumentationEnable: "off"  # Disables the zero-code injection capability for Python applications.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: arms-python-client
  name: arms-python-client
  namespace: arms-demo
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: arms-python-client
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: arms-python-client
        aliyun.com/app-language: python
        armsPilotAutoEnable: 'on'
        armsPilotCreateAppName: "arms-python-client"
        armsPilotAppWorkspace: "workspace"
    spec:
      containers:
        - image: registry.cn-hangzhou.aliyuncs.com/arms-default/python-agent:arms-python-client

Click Update.

View monitoring data

Log on to the Cloud Monitor 2.0 console and select the target workspace. In the left-side navigation pane, choose All Features > Application Observability > Application Monitoring.
On the Application List page, click an application's name to view its monitoring data.