Import data from MaxCompute to Log Service

Import data

1. Log on to the Log Service console.
2. In the Import Data section, click the MaxCompute - Data Import card.
3. Select the newly created project and Logstore. Then, click Next.
4. Configure data import.
   1. On the Specify Data Source tab, set the parameters. The following table describe the parameters.

      Parameter	Description
      Config Name	The name of the Logtail configuration file.
      MaxCompute Project	The name of the MaxCompute project where the data to be imported resides.
      Table	The name of the table where the data to be imported resides.
      Partition Description	The description of MaxCompute table partitions. You must specify this parameter if you import data from a partitioned table. For more information, see Partition. Separate multiple-level partitions with commas (,).
      AccessKey ID	The AccessKey ID that you use to access MaxCompute. Note The Alibaba Cloud account or RAM user to which the AccessKey ID belongs must be granted access to the MaxCompute project.
      AccessKey Secret	The AccessKey secret that you use to access MaxCompute.
      Endpoint	The endpoint of the MaxCompute project. For more information, see Endpoints.
      Tunnel Endpoint	The tunnel endpoint that you use to access the MaxCompute project. For more information, see Endpoints.

   2. Click Preview to preview the data.
   3. Click Next.
   4. On the Specify Data Type tab, set the required parameters. The following table describes the parameters.

      Parameter	Description
      Use System Time	If you turn on the Use System Time switch, the time field of a parsed log entry is the system time when the log entry is imported. If you turn off the Use System Time switch, you must manually configure the time field and format. Note We recommend that you turn on the Use System Time switch. You can configure an index for the time field and use the index for log queries. If you import data that is generated earlier than the current time minus the data retention period of the Logstore, the data cannot be queried in the Log Service console. For example, if the data retention period is seven days and you import data that was generated seven days ago, no results can be found in the Log Service console.
      Time Field	If you turn off the Use System Time switch, you must specify a field to extract the log time.
      Time Format	If you turn off the Use System Time switch, you must specify a time format by using the Java SimpleDateFormat class. The time format is used to parse time fields, or parse strings that are extracted by regular expressions. For more information, see Class SimpleDateFormat and Time formats. Note The Java SimpleDateFormat class does not support Unix timestamps. If you want to use Unix timestamps, you can set the time format to epoch.
      Time Zone	If you turn off the Use System Time switch, you must specify a time zone to parse the time zone of the log time. This parameter is invalid if the time zone information already exists in the log format.

   5. Optional:After you specify the required parameters, click Test.
   6. After the test succeeds, click Next.
   7. Optional:On the Specify Scheduling Interval tab, set the required parameters. The following table describes the parameters.

      Parameter	Description
      Import Interval	The interval at which the MaxCompute data is imported to Log Service.
      Import Now	If you turn on the Import Now switch, the MaxCompute data is immediately imported.

   8. Click Next.
5. Preview data, configure indexes, and then click Next.
   By default, full-text indexing is enabled for Log Service. You can also configure field indexes based on collected logs in manual or automatic mode. For more information, see Configure indexes.

   Note If you want to query and analyze logs, you must enable full-text indexing or field indexing. If you enable both full-text indexing and field indexing, the system uses only field indexes.

View the data import configurations

After you create data import configurations, you can view the configuration details and relevant statistical report in the Log Service console.

1. In the Projects section, click the target project.
2. Find the Logstore to which the data import configurations belong, choose Data Import > Data Import, and then click the name of the data import configurations.
3. On the Import Configuration Overview page, view the configuration details and statistical report.

References

On the Import Configuration Overview page, you can perform the following operations:

• Modify the data import configurations

  Click Modify Settings to modify the data import configurations. For more information, see Configure data import.

• Delete the data import configurations.
  You can click Delete Configuration to delete the data import configurations.

  Note The data import configurations cannot be recovered after they are deleted.