Paginate query and analysis results in Simple Log Service - Simple Log Service

Cuando los resultados de una consulta son extensos, utilice la paginación para controlar cuántos registros se devuelven por solicitud. Las instrucciones de búsqueda se pagan mediante los parámetros offset y line; las instrucciones analíticas se pagan con la cláusula LIMIT.

Métodos de paginación

La operación GetLogs admite paginación tanto para instrucciones de búsqueda como para instrucciones analíticas, aunque cada una utiliza un método distinto. Para obtener el recuento total de registros antes de consultar, llame a GetHistograms.

Instrucción de búsqueda: consulta datos de registros sin procesar mediante palabras clave. Establezca los parámetros offset y line en GetLogs para paginar. Log search overview.
Instrucción analítica: analiza los resultados de la consulta con SQL. Utilice la cláusula LIMIT para paginar. Overview of log query and analysis | LIMIT clause.

Ejemplos de paginación

A continuación se muestra cómo paginar resultados de consulta y de análisis.

Paginar resultados de consulta

En una consulta paginada, el valor de offset se incrementa en cada solicitud hasta que GetLogs devuelve 0 resultados y marca la consulta como completada.

Pseudocódigo de paginación

offset = 0                           # The line from which the system starts to read logs. The value 0 indicates that the system reads logs from line 0. 
line = 100                          # The number of rows that are returned for the current API request. Up to 100 rows can be returned. If you set this parameter to a value greater than 100, only 100 rows are returned. Read 100 lines at a time. 
query = "status:200"                 # Query all logs in which the value of the status field is 200. 
while True:
     response = get_logstore_logs(query, offset, line) # Call the operation to read logs. 
     process (response)                                 # Call custom logic to process the returned result. 
     if response.get_count() == 0 && response.is_complete()   
         The read process is complete, and the current loop ends.
     else
        offset += 100                          # The value of the offset parameter increases to 100. The next 100 lines are read.

Ejemplo de código en Python

Overview of Simple Log Service SDK for Python.

# The Simple Log Service endpoint. 
endpoint = '' 
# In this example, the AccessKey ID and AccessKey secret are obtained from environment variables. 
accessKeyId = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_ID', '')
accessKey = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_SECRET', '')
# The name of the project. 
project = ''     
# The name of the Logstore. 
logstore = ''  
client = LogClient(endpoint, accessKeyId, accessKey)
topic = ""
From = int(time.time()) - 600
To = int(time.time())
log_line = 100
offset = 0
while True:
    res4 = None
    for retry_time in range(0, 3): 
        req4 = GetLogsRequest(project, logstore, From, To, topic=topic, line=log_line, offset=offset)
        res4 = client.get_logs(req4)
        if res4 is not None and res4.is_completed():
            break
        time.sleep(1)
     offset += 100 
     if res4.is_completed() and res4.get_count() == 0:
         break;
     if res4 is not None:
         # The execution result. 
        res4.log_print()

Ejemplo de código en Java

Overview of Simple Log Service SDK for Java.

int log_offset = 0;
int log_line = 100;   // The number of lines that are read at a time. Up to 100 rows can be returned. If you want to read more than 100 lines, use the offset parameter. The offset and line parameters take effect only for a search statement that uses keywords. If you use an analytic statement, the offset and line parameters are invalid. If you want an analytic statement to return more than 100 rows, use the LIMIT clause. 
while (true) {
    GetLogsResponse res4 = null;
    // For each offset, 100 lines are read at a time. If the read operation fails, up to three retries are allowed. 
    for (int retry_time = 0; retry_time < 3; retry_time++) {
          GetLogsRequest req4 = new GetLogsRequest(project, logstore, from, to, topic, query, log_offset,
               log_line, false);
          res4 = client.GetLogs(req4);
          if (res4 != null && res4.IsCompleted()) {
                    break;
                }
                Thread.sleep(200);
            }
            System.out.println("Read log count:" + String.valueOf(res4.GetCount()));
            log_offset += log_line;
            if (res4.IsCompleted() && res4.GetCount() == 0) {
                        break;
            }
 }

Paginar resultados de análisis

Utilice la cláusula LIMIT en SQL para paginar los resultados del análisis. Por ejemplo, si * | select count(1) , url group by url devuelve 1,000 entradas, divida la consulta en dos solicitudes de 500 cada una:

* | select count(1) , url  group by url  limit 0, 500
* | select count(1) , url  group by url  limit 500, 500

Pseudocódigo de paginación

offset = 0     // The line from which the system starts to read logs. The value 0 indicates that the system reads logs from line 0. 
line = 500    // The number of rows that are returned for the current API request. Up to 1,000,000 rows can be returned. If many lines are read at a time, the network latency increases, and the client-side processing speed decreases. Read 500 lines at a time. 
query = "* | select count(1) , url  group by url  limit "
while True:
real_query = query + offset + "," +  line
response = get_logstore_logs(real_query) // Call the operation to read logs. 
process (response)                       // Call custom logic to process the returned result. 
if response.get_count() == 0   
    The read process is complete, and the current loop ends.
else
    offset += 500                        // The value of the offset parameter increases to 500. The next 500 lines are read.

Ejemplo de código en Python

Overview of Simple Log Service SDK for Python.

# The Simple Log Service endpoint.
endpoint = '' 
# In this example, the AccessKey ID and AccessKey secret are obtained from environment variables. 
accessKeyId = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_ID', '')
accessKey = os.environ.get('ALIBABA_CLOUD_ACCESS_KEY_SECRET', '')
# The name of the project. 
project = ''     
# The name of the Logstore. 
logstore = ''  
client = LogClient(endpoint, accessKeyId, accessKey)
topic = ""
origin_query = "* | select *  limit "
From = int(time.time()) - 600
To = int(time.time())
log_line = 100
offset = 0
while True:
    res4 = None
    query = origin_query + str(offset) + " , " + str(log_line)
    for retry_time in range(0, 3): 
        req4 = GetLogsRequest(project, logstore, From, To, topic=topic, query=query)
        res4 = client.get_logs(req4)
        if res4 is not None and res4.is_completed():
            break
        time.sleep(1)
    offset += 100 
    if res4.is_completed() and res4.get_count() == 0:
        break;
    if res4 is not None:
    # The execution result. 
        res4.log_print()

Ejemplo de código en Java

Overview of Simple Log Service SDK for Java.

int log_offset = 0;
int log_line = 500;
String origin_query = "* | select count(1) , url  group by url  limit "
while (true) {
            GetLogsResponse res4 = null;
            // For each offset, 500 lines are read at a time. If the read operation fails, up to three retries are allowed. 
            query = origin_query + log_offset + "," + log_line;
            for (int retry_time = 0; retry_time < 3; retry_time++) {
                GetLogsRequest req4 = new GetLogsRequest(project, logstore, from, to, topic, query);
                res4 = client.GetLogs(req4);

                if (res4 != null && res4.IsCompleted()) {
                    break;
                }
                Thread.sleep(200);
            }
            System.out.println("Read log count:" + String.valueOf(res4.GetCount()));
            log_offset += log_line;
            if (res4.GetCount() == 0) {
                        break;
            }
}