All Products
Search
Document Center

Export logs

Last Updated: Sep 26, 2021

In order to meet your needs of implementing self-processing and personalized analysis on original log data, Mobile Analysis Service (MAS) supports exporting the tracking log data uploaded by the client offline, namely, exporting the log data from mPaaS ODPS (Open Data Processing Service) tables to your ODPS tables.

Currently, MAS provides two export methods: routine export and manual export.

  • Routine export: Synchronize the source table to the target table in the early morning every day by running a scheduled task. The exported data is T-1 data.
  • Manual export: Export the historical data and the data failed to be exported in routine export offline by manually supplementing data. Support supplementing the historical data in the last 90 days.
Note: MAS log data export may incur corresponding costs. For shared executor group, you will be charged by the traffic generated in exporting log data. For exclusive executor group, the costs include exclusive resource fees and data traffic fees. For specific charging methods, see Postpaid mode.

Create log export tasks

You can create two types of tasks: routine task and data supplement task.

  • Routine task: Synchronize the newly added log data generated on the previous day daily. The task is executed every day from 2:00 to 8:00 in the morning.
  • Data supplement task: Supplement the data of specified date when the routine task fails. You can supplement the historical data in the last 90 days at most.

Create a routine task

The routine task is triggered by a task template.

Follow the steps below to complete task creation:

  1. Log in to the mPaaS console, and select the target App from the App list.
  2. On the left navigation pane, click the Mobile Analysis Service > Log management menu.
  3. Enter the Export logs > Manage task templates tab page, and click Create a task template. In the pop-up template creation window, configure the template information. The configuration items include:
    Configuration items Required Description
    Task template name Yes Customize the template name.
    Executor group Yes The task executor runs in the form of cluster group. The executor group falls into shared executor group (default, for all customers) and exclusive executor group.
    Target database AccessID Yes Access ID of the target MaxCompute (formerly known as ODPS) database, namely the Alibaba Cloud account.
    Target database AccessSecret Yes Key of the target MaxCompute database
    Target database ODPSProject Yes Project name of the target MaxCompute database
    Target database region Yes The region where the target MaxCompute database is. The system will automatically generate Endpoint according to the MaxCompute region table.
    Source table Yes Support exporting the data of following MAS log tables:
    • uniform_behavior: Behavior log table
    • uniform_behavior_auto: Automation log table
    • monitor_performance: Performance log table
    • uniform_exception: Exception log table
    Note: Only one task template is allowed for each log type.
    Target table Yes The table to which the data will be exported. Input the target table name of the corresponding log type, and create the table in MaxCompute based on the table creation statements.
  4. After configuring the template information, click OK to complete the creation. The newly created template will appear in the task template list.

    The Run switch of the newly created template is turned on by default, which means that the export task will be generated periodically. The task will be displayed in the Manage tasks list, and the task type is Routine. If the Run switch is turned off, the routine export tasks will no longer be generated in the future.

Create a data supplement task

The data supplement task also depends on the task template.

Follow the steps below to complete task creation:

  1. On the left navigation pane, click the Mobile Analysis Service > Log management menu.
  2. Enter the Export logs > Manage tasks tab page, and click Create a data supplement task.
  3. In the pop-up task creation window, select Supplement date and Task template, and then click OK to complete the task creation.

    After the task is created, it will be executed immediately. The newly created data supplement task will be displayed in the task management list, and the task type is Data supplement.

Manage task templates

Whether it is the scheduled export of daily data or the manual export of historical data, the export task is executed at a specific time period of the day, that is, the daily log data export will generate a separate task.

MAS supports creating and managing the log export task with the same source table and target table based on templates so as to reduce repeated log export configuration tasks and improve work efficiency.

The task template list displays information such as template name, task type, executor group, target database region, template creation time, and running status. Task template management includes the operations of adding, running, viewing, editing and deletion templates.

Create task templates

For how to create a task template, see Create a routine task.

View template details

In the task template list, select the target template, and click View on the right to view the template details.

Edit templates

In the task template list, select the target template, click Edit on the right, modify the template information and confirm the modification. The modified template will take effect on the tasks generated later, and will not have any impact on the tasks that have been executed before.

Delete templates

In the task template list, select the target template, click Delete on the right, and confirm to delete the corresponding template.

Manage log export tasks

The task management page displays all log tasks generated periodically or in batches by task templates, including task ID, task type, executor group, task template name, task execution time, execution status and other information.

On the Manage tasks page, select the task you want to view from the task list, and click View log to view the current task execution details. If the task execution fails, you can view the failure cause through the log. The task in the unexecuted state cannot be viewed in the log.

Support filtering log export tasks by specifying time period, task type and execution state.

Manage executor groups

The executor group is an execution program that performs export tasks. MAS adopts a grouped cluster architecture for log export executors to support large volume data export, serve more users, and ensure that the system can respond to your needs stably in a timely manner. By default, the shared executor group is used to export logs. If you have higher requirements on timeliness and performance, you can submit a ticket to apply for the exclusive executor group.

The executor list displays all shared and exclusive executors used in the current application log export, including the executor group name, executor group type, number of cumulative executed tasks, cumulative export data volume, number of tasks in execution, number of tasks in queue, and running status.

Where:

  • Executor group: Fall into two types, shared and exclusive
  • Cumulative executed tasks : The total number of the tasks executed by the current executor group
  • Cumulative exported data : The total volume of the log data exported by the current executor group
  • Tasks in execution : The number of the tasks being executed by the current executor group
  • Tasks in queue: The number of the tasks in queue in the current executor group
  • Running status: The running status of the executor, smooth or blocked. If there is one or more tasks in queue, it is regarded as blocked.

Task overview

The task overview displays the log export task execution data. You can query the log data export information within a specified time period. Note that the start/end date selected here corresponds to the actual execution date of the task.

Task dashboard

Display the task execution statistics data, including the number of successful tasks, the number of failed tasks, and the number of tasks in execution. At the same time, a line chart of task execution success and failure data is provided to facilitate you to understand the task execution trend over a period of time.

Executor group dashboard

The task execution status of the shared executor groups and the exclusive executor groups are respectively displayed, including the cumulative times of successful task execution and the cumulative volume of exported data.