Simple Log Service provides the time series forecasting feature to automatically and intelligently predict time series data. You can learn about the future trends of time series data based on the prediction results and know the status of the system or key business metrics in advance. This topic describes the background information, workflow, scheduling and execution scenarios, and usage notes of time series forecasting.
Background information
A variety of time series data is generated when a service is running. The data records the changes of various service metrics over time. The monitoring and analysis of time series data is widely used in system monitoring and fault location. You can use the following two methods to monitor and analyze time series data:
Analyze the generated time series data to detect possible exceptions. To achieve the goal, you can use the intelligent inspection feature. For more information, see How intelligent inspection works.
This method is mainly used to identify and locate problems in time after service errors occur.
Predict future time series data and estimate the future trends in the data. To achieve the goal, you can use the time series forecasting feature.
This method is mainly used to provide early warning for abnormal trends of key service metrics.
Time series forecasting is suitable for the following scenarios:
Predict the trends of key service metrics, such as QPS and the number of online users, for a period of time in the future. If the value of a metric exceeds the threshold, alerts are triggered. This allows O&M engineers to troubleshoot potential issues before errors occur.
Predict the future fluctuations of various system metrics, such as CPU utilization and disk utilization. You can use the future fluctuation trends of metrics to adjust the system more reliably. For example, you can scale out the system if the CPU utilization is too high.
The future trends of time series data is affected by various factors, such as emergencies, unstable conditions, and internal defects of forecasting algorithms. Therefore, the prediction results are not 100% accurate. The prediction results can be used only for reference during decision-making.
Workflow
A time-series forecasting job uses SQL statements to extract or aggregate time series metrics. The job regularly pulls data based on scheduling rules, writes the prediction results to the destination Logstore (internal-ml-log), and displays the prediction results in a dashboard. This helps you quickly view the results of time series forecasting.
Basic concepts
The following table lists the basic concepts involved in time series forecasting.
Concept | Description |
Job | A time-series forecasting job includes data features and algorithm model parameters. |
Instance | A time-series forecasting job creates a time-series forecasting instance based on the configuration of the job. The instance pulls data at regular intervals, runs the algorithm model, and then distributes the analysis result based on the configuration of the job. For more information about the impacts of different operations on the scheduling and running of instances, see Scheduling and execution scenarios.
|
Instance ID | Each instance is identified by a unique ID. |
Creation time | Each instance is created at a specific point in time. In most cases, an instance is created for a time-series forecasting job based on the scheduling rules of the job. If historical data needs to be processed or the delay caused by the timeout of the previous instance is offset, an instance is immediately created. |
Start time | Each instance starts to run at a specific point in time. If the job to which an instance belongs is retried, the start time is the most recent time at which the instance starts to run. |
End time | Each instance stops running at a specific point in time. If the job to which an instance belongs is retried, the end time is the most recent time at which the instance stops running. |
Status | Each instance is in a specific state at a specific point in time. Valid values:
|
Data feature configurations | Data feature configurations include time, entities, and features. For more information, see Create a time-series forecasting job. |
Algorithm configurations | Algorithm configurations include cycle, holiday, and forecasting (including length of the sequence to be tested, confidence, number of samples, forecasting frequency, and observation duration). For more information, see Create a time-series forecasting job. |
Forecasting results | Forecasting results are displayed in the built-in dashboard. |
Scheduling and execution scenarios
Each job can create one or more instances. Only one instance can run in a job at a time regardless of whether the instance is run on schedule or is retried due to an anomaly. You cannot concurrently run multiple instances in a single job.
The following table describes the scheduling and execution scenarios.
Scenario | Description |
Scenario 1: Execute a time-series forecasting job from the current point in time | The time-series forecasting job reads historical data based on the configured observation length and then predicts the time series for a period of time in the future. |
Scenario 2: Execute a time-series forecasting job from a historical point in time | If you configure a job to start at a previous point in time, algorithm models analyze data from the historical point in time to the current point in time. |
Scenario 3: Modify the scheduling configurations | After you modify the scheduling configurations of a job, the job generates an instance based on the new configurations. Algorithm models record the point in time before which all historical data is analyzed and continue to analyze the most recent data. |
Scenario 4: Retry a failed instance | If an instance fails to run due to issues such as insufficient permissions, unavailable source Logstore, unavailable destination Logstore, or invalid configurations, Simple Log Service can automatically retry to run the instance. If an instance is stuck in the STARTING state, the configuration of the instance may fail. Simple Log Service generates an error log record and sends the record to the internal-etl-log Logstore. You can verify the configuration of the instance and start the instance again. After the instance is scheduled and run, Simple Log Service changes the status of the instance to SUCCEEDED or FAILED based on the retry result. |
Usage notes
We recommend that you specify the metrics you want to monitor based on your business requirements. This way, you can improve the efficiency of time series forecasting. Take note of the following information:
Specify the format of the data uploaded to the specified Logstore, define the formats and fields of data, and specify the observation granularity.
Obtain the metric data changes of the entities that you specify and understand the stability and periodicity of the metric data. These operations help you configure the parameters of an algorithm model.
Align the time windows of time-series forecasting jobs at a point in time that is accurate to seconds, minutes, or hours.