如何创建/修改/查询/删除索引 - Simple Log Service

An index is an inverted storage structure that maps keywords to data rows for fast lookups, similar to a data catalog. You can query and analyze log data only after you configure indexes. This topic provides sample code for creating, modifying, querying, and deleting indexes by using Simple Log Service SDK for Java.

Prerequisites

Simple Log Service is activated.
Simple Log Service SDK for Python is initialized.

Simple Log Service SDK for Java is installed. For more information, see Install the Java SDK.
You have written logs to a Logstore. For more information, see Data collection overview.

Precautions

In this example, the public Simple Log Service endpoint for the China (Hangzhou) region is used. Endpoint: https://cn-hangzhou.log.aliyuncs.com.

If you want to access Simple Log Service from other Alibaba Cloud services that reside in the same region as your project, you can use the internal Simple Log Service endpoint, which is https://cn-hangzhou-intranet.log.aliyuncs.com.

For more information about the supported regions and endpoints of Simple Log Service, see Endpoint.

Raw log

body_bytes_sent:1750
host:www.example.com
http_referer:www.example.com
http_user_agent:Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; it-it) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27
http_x_forwarded_for:203.0.XX.XX
remote_addr:203.0.XX.XX
remote_user:p288
request_length:13741
request_method:GET
request_time:71
request_uri:/request/path-1/file-1
http_code:200
time_local:11/Aug/2021:06:52:27
upstream_response_time:0.66

Sample code that is used to create indexes

You can manage indexes in the Simple Log Service console with ease. For more information, see Create indexes.

The following sample code creates an index with full-text indexing enabled and field indexing enabled for the request_method and status fields. This example is based on the raw log.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.Index;
import com.aliyun.openservices.log.exception.LogException;
public class CreateIndex {
    public static void main(String[] args) throws LogException {
        // This example obtains the AccessKey ID and AccessKey secret from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // Specify the project name.
        String projectName = "ali-test-project";
        // Specify the Logstore name.
        String logstoreName = "ali-test-logstore";
        // Set the endpoint of Simple Log Service. This example uses the endpoint of the China (Hangzhou) region. Replace the value with the actual endpoint.
        String host = "https://cn-hangzhou.log.aliyuncs.com";
        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);
        try {
            // Define the index.
            // Before you create an index, you must plan the full-text and field index configurations. In this example, a full-text index is enabled and a field index is created for the request_method and status fields.
            String logstoreindex = "{\"line\": {\"token\": [\",\", \" \", \"'\", \"\\\"\", \";\", \"=\", \"(\", \")\", \"[\", \"]\", \"{\", \"}\", \"?\", \"@\", \"&\", \"<\", \">\", \"/\", \":\", \"\\n\", \"\\t\", \"\\r\"], \"caseSensitive\": false, \"chn\": false}, \"keys\": {\"request_method\": {\"type\": \"text\", \"token\": [\",\", \" \", \"'\", \"\\\"\", \";\", \"=\", \"(\", \")\", \"[\", \"]\", \"{\", \"}\", \"?\", \"@\", \"&\", \"<\", \">\", \"/\", \":\", \"\\n\", \"\\t\", \"\\r\"], \"caseSensitive\": false, \"alias\": \"\", \"doc_value\": true, \"chn\": false}, \"status\": {\"type\": \"long\", \"alias\": \"\", \"doc_value\": true}}, \"log_reduce\": false, \"max_text_len\": 2048}";
            Index index = new Index();
            System.out.println("ready to create index");
            index.FromJsonString(logstoreindex);
            client.CreateIndex(projectName, logstoreName, index);
            System.out.println(String.format("create index for %s success", logstoreName));
        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Expected result:

ready to create index
create index for ali-test-logstore success

Sample code that is used to modify indexes

You can manage indexes in the Simple Log Service console with ease. For more information, see Create indexes.

The following sample code modifies an index. In this example, full-text indexing is enabled, field indexing is enabled for the request_method and status fields, and caseSensitive is set to True for the request_method field. This example is based on the raw log.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.Index;
import com.aliyun.openservices.log.exception.LogException;
public class UpdateIndex {
    public static void main(String[] args) throws LogException {
        // This example obtains the AccessKey ID and AccessKey secret from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // Specify the project name.
        String projectName = "ali-test-project";
        // Specify the Logstore name.
        String logstoreName = "ali-test-logstore";
        // Set the endpoint of Simple Log Service. This example uses the endpoint of the China (Hangzhou) region. Replace the value with the actual endpoint.
        String host = "https://cn-hangzhou.log.aliyuncs.com";
        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);
        try {
            // Update the index.
            String logstoreindex = "{\"line\": {\"token\": [\",\", \" \", \"'\", \"\\\"\", \";\", \"=\", \"(\", \")\", \"[\", \"]\", \"{\", \"}\", \"?\", \"@\", \"&\", \"<\", \">\", \"/\", \":\", \"\\n\", \"\\t\", \"\\r\"], \"caseSensitive\": false, \"chn\": false}, \"keys\": {\"request_method\": {\"type\": \"text\", \"token\": [\",\", \" \", \"'\", \"\\\"\", \";\", \"=\", \"(\", \")\", \"[\", \"]\", \"{\", \"}\", \"?\", \"@\", \"&\", \"<\", \">\", \"/\", \":\", \"\\n\", \"\\t\", \"\\r\"], \"caseSensitive\": true, \"alias\": \"\", \"doc_value\": true, \"chn\": false}, \"status\": {\"type\": \"long\", \"alias\": \"\", \"doc_value\": true}}, \"log_reduce\": false, \"max_text_len\": 2048}";
            Index index = new Index();
            System.out.println("ready to update index");
            index.FromJsonString(logstoreindex);
            client.UpdateIndex(projectName, logstoreName, index);
            System.out.println(String.format("update index for %s success", logstoreName));
        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Expected result:

ready to update index
update index for ali-test-logstore success

Sample code that is used to query indexes

The following sample code queries the index configuration of a specified Logstore.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.common.Index;
import com.aliyun.openservices.log.exception.LogException;
import com.aliyun.openservices.log.response.GetIndexResponse;
public class ListIndex {
    public static void main(String[] args) throws LogException {
        // This example obtains the AccessKey ID and AccessKey secret from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // Specify the project name.
        String projectName = "ali-test-project";
        // Specify the Logstore name.
        String logstoreName = "ali-test-logstore";
        // Set the endpoint of Simple Log Service. This example uses the endpoint of the China (Hangzhou) region. Replace the value with the actual endpoint.
        String host = "https://cn-hangzhou.log.aliyuncs.com";
        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);
        try {
            Index index = new Index();
            System.out.println("ready to get index");
            GetIndexResponse response = client.GetIndex(projectName, logstoreName);
            index = response.GetIndex();
            // Print the index configuration.
            System.out.println("The index is: " + index.ToJsonString());
            System.out.println(String.format("get index for %s success", logstoreName));
        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Expected result:

ready to get index
The index is: {"log_reduce":false,"line":{"caseSensitive":false,"chn":false,"token":[","," ","'","\"",";","=","(",")","[","]","{","}","?","@","&","<",">","/",":","\n","\t","\r"]},"keys":{"request_method":{"doc_value":true,"caseSensitive":true,"chn":false,"alias":"","type":"text","token":[","," ","'","\"",";","=","(",")","[","]","{","}","?","@","&","<",">","/",":","\n","\t","\r"]},"status":{"doc_value":true,"alias":"","type":"long"}},"ttl":30,"max_text_len":2048}
get index for ali-test-logstore success

Sample code that is used to delete indexes

The following sample code deletes the index of a specified Logstore.

import com.aliyun.openservices.log.Client;
import com.aliyun.openservices.log.exception.LogException;
public class DeleteIndex {
    public static void main(String[] args) throws LogException {
        // This example obtains the AccessKey ID and AccessKey secret from environment variables.
        String accessId = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID");
        String accessKey = System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET");
        // Specify the project name.
        String projectName = "ali-test-project";
        // Specify the Logstore name.
        String logstoreName = "ali-test-logstore";
        // Set the endpoint of Simple Log Service. This example uses the endpoint of the China (Hangzhou) region. Replace the value with the actual endpoint.
        String host = "https://cn-hangzhou.log.aliyuncs.com";
        // Create a Simple Log Service client.
        Client client = new Client(host, accessId, accessKey);
        try {
            System.out.println("ready to delete index");
            client.DeleteIndex(projectName, logstoreName);
            System.out.println(String.format("delete index for %s success", logstoreName));
        } catch (LogException e) {
            System.out.println("LogException e :" + e.toString());
            System.out.println("error code :" + e.GetErrorCode());
            System.out.println("error message :" + e.GetErrorMessage());
            throw e;
        }
    }
}

Expected result:

ready to delete index
delete index for ali-test-logstore success

References

Error codes.
In addition to its native SDK, SLS also supports the common Alibaba Cloud SDKs. For more information, see Simple Log Service_SDK Center_Alibaba Cloud OpenAPI Explorer.
Simple Log Service CLI.
For more information about index-related API operations, see the following topics:
Alibaba Cloud Simple Log Service SDK for Java on GitHub.