Simple Log Service:Recopilación y análisis de registros de texto de ECS mediante LoongCollector

Requisitos previos

Activar SLS y preparar una cuenta

• Activar SLS: Si es la primera vez que lo utiliza, inicie sesión en la consola de Simple Log Service y siga las instrucciones en pantalla para activar el servicio.

• Preparar una cuenta:

  • Cuenta de Alibaba Cloud: Cuenta con todos los permisos de forma predeterminada.

  • Usuario RAM: Si utiliza un usuario RAM, debe grant las directivas de permisos necesarias:

    • AliyunLogFullAccess: Otorga permisos para crear y gestionar recursos de SLS, como proyectos y logstores.

    • AliyunECSFullAccess: Otorga permisos para instalar el agente de recopilación en una instancia de ECS.

    • AliyunOOSFullAccess: Otorga permisos para instalar automáticamente el agente de recopilación en una instancia de ECS mediante Operation Orchestration Service (OOS).

    En un entorno de producción, puede create custom permission policies para obtener un control más granular sobre los permisos del usuario RAM.

Preparar una instancia de ECS

Asegúrese de que el security group ECS instance permita el tráfico de salida por el puerto 80 (HTTP) y el puerto 443 (HTTPS).

Generar registros simulados

1. Log on to the ECS instance.

2. Cree un archivo de script llamado generate_nginx_logs.sh y pegue el siguiente contenido. Este script escribe una entrada de registro de acceso estándar de Nginx en /var/log/nginx/access.log cada 5 segundos.

   generate_nginx_logs.sh

   #!/bin/bash
   
   #==============================================================================
   # Script Name: generate_nginx_logs.sh
   # Script Description: Simulates an NGINX server and continuously writes logs to access.log.
   #==============================================================================
   
   # --- Configurable Parameters ---
   
   # Log file path
   LOG_FILE="/var/log/nginx/access.log"
   
   # --- Mock Data Pools ---
   
   # Random IP address pool
   IP_ADDRESSES=(
       "192.168.1.10" "10.0.0.5" "172.16.31.40" "203.0.113.15"
       "8.8.8.8" "1.1.X.X" "91.198.XXX.XXX" "114.114.114.114"
       "180.76.XX.XX" "223.5.5.5"
   )
   
   # HTTP request method pool
   HTTP_METHODS=("GET" "POST" "PUT" "DELETE" "HEAD")
   
   # Common request path pool
   REQUEST_PATHS=(
       "/index.html" "/api/v1/users" "/api/v1/products?id=123" "/images/logo.png"
       "/static/js/main.js" "/static/css/style.css" "/login" "/admin/dashboard"
       "/robots.txt" "/sitemap.xml" "/non_existent_page.html"
   )
   
   # HTTP status code pool (You can adjust the weights, for example, add more 200s to increase their probability)
   HTTP_STATUSES=(200 200 200 200 201 301 404 404 500 502 403)
   
   # Common User-Agent pool
   USER_AGENTS=(
       "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36"
       "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36"
       "Mozilla/5.0 (iPhone; CPU iPhone OS 14_6 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1 Mobile/15E148 Safari/604.1"
       "Mozilla/5.0 (Linux; Android 11; SM-G991U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.120 Mobile Safari/537.36"
       "curl/7.68.0"
       "Googlebot/2.1 (+http://www.google.com/bot.html)"
   )
   
   # Common Referer pool
   REFERERS=(
       "https://www.google.com/"
       "https://www.bing.com/"
       "https://github.com/"
       "https://stackoverflow.com/"
       "-"
       "-"
       "-"
   )
   # Check and create the log directory
   LOG_DIR=$(dirname "$LOG_FILE")
   if [ ! -d "$LOG_DIR" ]; then
       echo "Log directory '$LOG_DIR' does not exist. Attempting to create..."
       # Use sudo to create the directory because root permissions are usually required
       sudo mkdir -p "$LOG_DIR"
       if [ $? -ne 0 ]; then
           echo "Error: Failed to create directory '$LOG_DIR'. Check permissions or create it manually."
           exit 1
       fi
       echo "Directory created successfully."
   fi
   
   # Check write permissions for the log file
   trap 'echo -e "\n\nScript interrupted. Stopping log generation..."; exit 0;' SIGINT
   
   # --- Core Function ---
   
   # Define a function to randomly select an element from an array
   # Usage: random_element "array_name"
   function random_element() {
       local arr=("${!1}")
       echo "${arr[$((RANDOM % ${#arr[@]}))]}"
   }
   
   # Catch the Ctrl+C interrupt signal for a graceful exit
   trap 'echo -e "\n\nScript interrupted. Stopping log generation..."; exit 0;' SIGINT
   
   # --- Main Loop ---
   
   echo "Start generating mock NGINX logs to $LOG_FILE ..."
   echo "A log entry is generated every 5 seconds."
   echo "Press Ctrl+C to stop."
   sleep 2
   
   # Infinite loop to continuously generate logs
   while true; do
       # 1. Get the current time in the default NGINX format: [dd/Mon/YYYY:HH:MM:SS +ZZZZ]
       timestamp=$(date +'%d/%b/%Y:%H:%M:%S %z')
   
       # 2. Randomly select data from the pools
       ip=$(random_element IP_ADDRESSES[@])
       method=$(random_element HTTP_METHODS[@])
       path=$(random_element REQUEST_PATHS[@])
       status=$(random_element HTTP_STATUSES[@])
       user_agent=$(random_element USER_AGENTS[@])
       referer=$(random_element REFERERS[@])
   
       # 3. Generate a random response body size (in bytes)
       bytes_sent=$((RANDOM % 5000 + 100)) # A random number between 100 and 5100
   
       # 4. Concatenate into a complete NGINX combined format log entry
       # Format: $remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"
       log_line="$ip - - [$timestamp] \"$method $path HTTP/1.1\" $status $bytes_sent \"$referer\" \"$user_agent\""
   
       # 5. Append the log line to the file
       # echo "$log_line" >> "$LOG_FILE"
       echo "$log_line" | sudo tee -a "$LOG_FILE" > /dev/null
       
       # 6. Wait for 5 seconds before the next loop
       sleep 5
   done

3. Otorgue permisos de ejecución: chmod +x generate_nginx_logs.sh.

4. Ejecute el script en segundo plano: nohup ./generate_nginx_logs.sh &.

Siguientes pasos

Ahora que ha recopilado, consultado, visualizado y monitoreado registros, explore los siguientes recursos para obtener más información:

• Projects y logstores: conceptos básicos de recursos.

• Data collection methods: elija el método adecuado para su escenario.

• Storage resource hierarchy: planifique el ciclo de vida de los recursos y la asignación de shards.

Preguntas frecuentes