Query logs with the GetLogs API - Simple Log Service - Centro de documentación de Alibaba Cloud

Llame a la operación GetLogs para consultar los registros recopilados. Este tema proporciona ejemplos del SDK de Python para escenarios comunes de consulta y análisis.

Requisitos previos

Se creó un usuario de RAM y se le otorgaron los permisos obligatorios. Create a RAM user and grant permissions.
Las variables de entorno ALIBABA_CLOUD_ACCESS_KEY_ID y ALIBABA_CLOUD_ACCESS_KEY_SECRET están configuradas. Configure environment variables on Linux, macOS, and Windows.
Importante
- El par de AccessKey de una cuenta de Alibaba Cloud tiene permisos sobre todas las operaciones de la API. Utilice el par de AccessKey de un usuario de RAM para las llamadas a la API y las operaciones diarias.
- No codifique su AccessKey ID o AccessKey Secret directamente en el código del proyecto. Las credenciales expuestas comprometen todos los recursos de su cuenta.
El SDK de Log Service para Python está instalado. Install Log Service SDK for Python.
Los registros se recopilaron en el logstore de destino. Data collection overview.
Está familiarizado con los parámetros de la operación GetLogs. GetLogs.

Notas de uso

Este tema utiliza el endpoint público de la región China (Hangzhou) como ejemplo: https://cn-hangzhou.log.aliyuncs.com.

Si accede a Log Service desde otros servicios de Alibaba Cloud en la misma región que su Project, utilice el endpoint interno: https://cn-hangzhou-intranet.log.aliyuncs.com.

Regiones y endpoints compatibles: Endpoints.
Llame a is_completed() en el objeto de respuesta para comprobar si los resultados de la consulta son precisos.
- Si is_completed() devuelve true, la consulta finalizó y los resultados son exactos.
- Si is_completed() devuelve false, los resultados son imprecisos e incompletos. Repita la solicitud para obtener resultados completos. Possible causes of imprecise queries.

Ejemplo de registro sin procesar

body_bytes_sent:1750
host:www.example.com
http_referer:www.example.com
http_user_agent:Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; it-it) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27
http_x_forwarded_for:203.0.XX.XX
remote_addr:203.0.XX.XX
remote_user:p288
request_length:13741
request_method:GET
request_time:71
request_uri:/request/path-1/file-1
http_code:200
time_local:11/Aug/2021:06:52:27
upstream_response_time:0.66

Ejemplos

Los siguientes ejemplos en Python muestran escenarios comunes de consulta y análisis de registros.

Importante

El parámetro query en GetLogs acepta instrucciones de búsqueda y análisis:

Cuando query contiene solo una instrucción de búsqueda, line controla la cantidad de registros devueltos.
Cuando query contiene una instrucción de búsqueda y análisis, line se ignora. Utilice LIMIT para controlar la cantidad de filas devueltas. LIMIT clause.

Sintaxis de la instrucción de búsqueda: Basic syntax.

Ejemplo 1: Consulta de registros mediante una palabra clave

Busque registros con la palabra clave path-0/file-5:

# encoding: utf-8
import time
import os
from aliyun.log import *

def main():
    # The endpoint for Log Service. For more information, see Endpoints. 
    # This example uses the endpoint for the China (Hangzhou) region. Replace it with your actual endpoint.
    endpoint = 'cn-hangzhou.log.aliyuncs.com'
    # This example retrieves the AccessKey ID and AccessKey Secret from environment variables.
    access_key_id = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_ID', '')
    access_key = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_SECRET', '')

    # The names of the Project and Logstore.
    project = 'your-project-name'
    logstore = 'your-logstore-name'

    # Create a Log Service client.
    client = LogClient(endpoint, access_key_id, access_key)

    # Query logs by using the keyword 'path-0/file-5'.
    query = 'path-0/file-5'

    # The from_time and to_time parameters specify the time range for the query in UNIX timestamp format.
    from_time = int(time.time()) - 3600
    to_time = time.time() + 3600

    print("ready to query logs from logstore %s" % logstore)

    # In this example, the query parameter specifies the search statement, and the line parameter limits the results to 3 log entries.
    request = GetLogsRequest(project, logstore, from_time, to_time, '', query=query, line=3, offset=0, reverse=False)
    response = client.get_logs(request)
    # Print the query results.
    print('-------------Query is started.-------------')
    for log in response.get_logs():
        print(log.contents.items())
    print('-------------Query is finished.-------------')

if __name__ == '__main__':
    main()

Respuesta:

ready to query logs from logstore your-logstore-name
-------------Query is started.-------------
dict_items([ ('remote_user', 'nhf3g'), ('time_local', '14/Feb/2022:06:49:28'),  ('request_uri', '/request/path-0/file-5')...])
dict_items([ ('remote_user', 'ysu'), ('time_local', '14/Feb/2022:06:49:38'),  ('request_uri', '/request/path-0/file-5')...])
dict_items([ ('remote_user', 'l_k'), ('time_local', '14/Feb/2022:06:49:38'),  ('request_uri', '/request/path-0/file-5')...])
-------------Query is finished.-------------

Process finished with exit code 0

Ejemplo 2: Consulta de registros al especificar un campo

Busque registros donde request_method sea POST, con line establecido en 3:

# encoding: utf-8
import time
import os
from aliyun.log import *

def main():
    # The endpoint for Log Service. This example uses the endpoint for the China (Hangzhou) region. 
    # Replace the value with the actual endpoint.
    endpoint = 'cn-hangzhou.log.aliyuncs.com'
    # This example retrieves the AccessKey ID and AccessKey Secret from environment variables.
    access_key_id = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_ID', '')
    access_key = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_SECRET', '')

    # The names of the Project and Logstore.
    project = 'your-project-name'
    logstore = 'your-logstore-name'

    # Create a Log Service client.
    client = LogClient(endpoint, access_key_id, access_key)

    # Query logs by a specific field.
    # Query for logs where the request method is POST.
    query = 'request_method:POST'

    # The from_time and to_time parameters specify the time range for the query in UNIX timestamp format.
    from_time = int(time.time()) - 3600
    to_time = time.time() + 3600

    print("ready to query logs from logstore %s" % logstore)

    # In this example, the query parameter specifies the search statement, and the line parameter limits the results to 3 log entries.
    request = GetLogsRequest(project, logstore, from_time, to_time, '', query=query, line=3, offset=0, reverse=False)
    response = client.get_logs(request)
    # Print the query results.
    print('-------------Query is started.-------------')
    for log in response.get_logs():
        print(log.contents.items())
    print('-------------Query is finished.-------------')

if __name__ == '__main__':
    main()

Respuesta:

ready to query logs from logstore your-logstore-name
-------------Query is started.-------------
dict_items([ ('remote_user', 'tv0m'), ('time_local', '14/Feb/2022:06:59:08'), ('request_method', 'POST')...])
dict_items([ ('remote_user', '6joc'), ('time_local', '14/Feb/2022:06:59:08'), ('request_method', 'POST')...])
dict_items([ ('remote_user', 'da8'), ('time_local', '14/Feb/2022:06:59:08'), ('request_method', 'POST')...])
-------------Query is finished.-------------

Process finished with exit code 0

Ejemplo 3: Análisis de registros mediante una instrucción SQL

Cuente las solicitudes POST con request_method:POST|select COUNT(*) as pv:

# encoding: utf-8
import time
import os
from aliyun.log import *

def main():
    # The endpoint for Log Service. This example uses the endpoint for the China (Hangzhou) region. 
    # Replace the value with the actual endpoint.
    endpoint = 'cn-hangzhou.log.aliyuncs.com'
    # This example retrieves the AccessKey ID and AccessKey Secret from environment variables.
    access_key_id = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_ID', '')
    access_key = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_SECRET', '')

    # The names of the Project and Logstore.
    project = 'your-project-name'
    logstore = 'your-logstore-name'

    # Create a Log Service client.
    client = LogClient(endpoint, access_key_id, access_key)

    # Analyze logs by using an analysis statement.
    # Query for logs where the request method is POST and count the number of page views (PVs).
    query = 'request_method:POST|select COUNT(*) as pv'

    # The from_time and to_time parameters specify the time range for the query in UNIX timestamp format.
    from_time = int(time.time()) - 3600
    to_time = time.time() + 3600

    print("ready to query logs from logstore %s" % logstore)

    # In this example, the query parameter specifies a search and analysis statement. The line parameter is ignored. 
    # The number of results is determined by the analysis statement, which returns one row.
    request = GetLogsRequest(project, logstore, from_time, to_time, '', query=query, line=3, offset=0, reverse=False)
    response = client.get_logs(request)
    # Print the query results.
    print('-------------Query is started.-------------')
    for log in response.get_logs():
        print(log.contents.items())
    print('-------------Query is finished.-------------')

if __name__ == '__main__':
    main()

Respuesta:

ready to query logs from logstore nginx-moni
-------------Query is started.-------------
dict_items([('pv', '2918')])
-------------Query is finished.-------------

Process finished with exit code 0

Ejemplo 4: Análisis de registros mediante la cláusula GROUP BY

Cuente las solicitudes POST agrupadas por host, ordenadas por PV de forma descendente, con request_method:POST|select host, COUNT(*) as pv group by host order by pv desc limit 5:

# encoding: utf-8
import time
import os
from aliyun.log import *

def main():
    # The endpoint for Log Service. This example uses the endpoint for the China (Hangzhou) region. 
    # Replace the value with the actual endpoint.
    endpoint = 'cn-hangzhou.log.aliyuncs.com'
    # This example retrieves the AccessKey ID and AccessKey Secret from environment variables.
    access_key_id = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_ID', '')
    access_key = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_SECRET', '')

    # The names of the Project and Logstore.
    project = 'your-project-name'
    logstore = 'your-logstore-name'

    # Create a Log Service client.
    client = LogClient(endpoint, access_key_id, access_key)

    # Count POST requests, group them by host, and sort them by PV count.
    # The LIMIT clause in the analysis statement restricts the output to 5 results.
    query = 'request_method:POST|select host, COUNT(*) as pv group by host order by pv desc limit 5'

    # The from_time and to_time parameters specify the time range for the query in UNIX timestamp format.
    from_time = int(time.time()) - 3600
    to_time = time.time() + 3600

    print("ready to query logs from logstore %s" % logstore)

    # The query parameter specifies the search and analysis statement. 
    # The line and reverse parameters are ignored. The number and order of results are determined by the analysis statement.
    request = GetLogsRequest(project, logstore, from_time, to_time, '', query=query, line=3, offset=0, reverse=False)
    response = client.get_logs(request)
    # Print the query results.
    print('-------------Query is started.-------------')
    for log in response.get_logs():
        print(log.contents.items())
    print('-------------Query is finished.-------------')

if __name__ == '__main__':
    main()

Respuesta:

ready to query logs from logstore nginx-moni
-------------Query is started.-------------
dict_items([('host', 'www.example.com'), ('pv', '7')])
dict_items([('host', 'www.example.org'), ('pv', '6')])
dict_items([('host', 'www.example.net'), ('pv', '6')])
dict_items([('host', 'www.example.edu'), ('pv', '5')])
dict_items([('host', 'www.aliyundoc.com'), ('pv', '4')])
-------------Query is finished.-------------

Process finished with exit code 0

Ejemplo 5: Escritura de campos específicos de registros obtenidos en un archivo local

Consulte registros con la palabra clave path-0/file-5 y escriba un valor de campo específico en un archivo local (log.txt):

# encoding: utf-8
import time
import os
from aliyun.log import *

def main():
    # The endpoint for Log Service. This example uses the endpoint for the China (Hangzhou) region. 
    # Replace the value with the actual endpoint.
    endpoint = 'cn-hangzhou.log.aliyuncs.com'
    # This example retrieves the AccessKey ID and AccessKey Secret from environment variables.
    access_key_id = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_ID', '')
    access_key = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_SECRET', '')

    # The names of the Project and Logstore.
    project = 'your-project-name'
    logstore = 'your-logstore-name'

    # Create a Log Service client.
    client = LogClient(endpoint, access_key_id, access_key)

    # Query logs by using the keyword 'path-0/file-5'.
    query = 'path-0/file-5'

    # The from_time and to_time parameters specify the time range for the query in UNIX timestamp format.
    from_time = int(time.time()) - 3600
    to_time = time.time() + 3600

    print("ready to query logs from logstore %s" % logstore)

    # In this example, the query parameter specifies the search statement, and the line parameter limits the results to 3 log entries.
    request = GetLogsRequest(project, logstore, from_time, to_time, '', query=query, line=3, offset=0, reverse=False)
    response = client.get_logs(request)
    # Print the query results.
    print('-------------Query is started.-------------')
    for log in response.get_logs():
        print(log.contents.items())
    print('-------------Query is finished.-------------')
    
    # Extract the value of the 'remote_user' key from each log and save it to a local file.
    print('-------------Start writing logs to a local file.-------------')
    for loglocal in response.get_logs():
        filename = 'log.txt'
        with open(filename, mode='a') as fileobject:
            fileobject.write(loglocal.contents.get('remote_user')+'\n')
    print('-------------Finished writing logs to the local file.-------------')

if __name__ == '__main__':
    main()

Respuesta:

ready to query logs from logstore your-logstore-name
-------------Query is started.-------------
dict_items([ ('remote_user', 'nhf3g'), ('time_local', '14/Feb/2022:06:49:28'),  ('request_uri', '/request/path-0/file-5')...])
dict_items([ ('remote_user', 'ysu'), ('time_local', '14/Feb/2022:06:49:38'),  ('request_uri', '/request/path-0/file-5')...])
dict_items([ ('remote_user', 'l_k'), ('time_local', '14/Feb/2022:06:49:38'),  ('request_uri', '/request/path-0/file-5')...])
-------------Query is finished.-------------
-------------Start writing logs to a local file.-------------
-------------Finished writing logs to the local file.-------------

Process finished with exit code 0

Contenido de log.txt generado en el mismo directorio que GetLogsTest.py:

nhf3g
ysu
l_k

Referencias

Si falla una llamada a la API, consulte API error codes para resolver el error.
Log Service también admite los SDK comunes de Alibaba Cloud. Log Service SDK Center - Alibaba Cloud OpenAPI Portal.
Log Service proporciona una CLI para automatizar configuraciones. Log Service command-line interface (CLI).