You can edit the parameters of a Realtime Compute job to customize the log download level and path for that job.

Note Only Realtime Compute V3.2.0 and later allow you to customize the log download level and path.

Procedure

  1. Log on to the Realtime Compute development platform.
  2. In the top navigation bar, click Development.
  3. In the Development section of the left-side navigation pane, double-click the target folder to find the target job.
  4. Double-click the target job to go to the job edit page.
  5. On the right side of the job edit page, click the Parameters tab. In the dialog box that appears, enter log4j configuration information.
  6. Stop the job. For more information, see Terminate a job.
  7. Start the job. For more information, see Start a job.

Precautions

  • The Object Storage Service (OSS) path must be the same as that configured for the Realtime Compute cluster in exclusive mode.
  • Logs are downloaded by using log4j, and system.out logs are not generated.
  • You can only configure Log Service and OSS as custom appenders by using the following fixed statements:
    • Log Service: log4j.appender.loghub = com.alibaba.blink.log.loghub.BlinkLogHubAppender
    • OSS: log4j.appender.oss=com.alibaba.blink.log.oss.BlinkOssAppender
  • The specified Log Service or OSS can interact with the cluster where the job resides.
  • If the specified log output mode is incorrect, the job can be started, but the logs cannot be printed based on the custom configuration.

Scenario 1

  • Description

    Change the log level to DEBUG and send logs to an OSS bucket.

  • Configuring job parameters
    Notice If you set the log4j.rootLogger parameter, you may fail to view log information or troubleshoot related issues on the Realtime Compute development platform. Use this parameter with caution.
    # Change the log level to DEBUG and export logs to a specific file in an OSS bucket.
    log4j.rootLogger=DEBUG, file, oss
    
    # Fixed statement. Configure the OSS appender.
    log4j.appender.oss=com.alibaba.blink.log.oss.BlinkOssAppender
    
    # The endpoint.
    log4j.appender.oss.endpoint=oss-cn-hangzhou****.aliyuncs.com
    
    # The AccessKey ID.
    log4j.appender.oss.accessId=U****4ZF  
    
    # The AccessKey secret.
    log4j.appender.oss.accessKey=hsf****DeLw   
    
    # The OSS bucket name.
    log4j.appender.oss.bucket=et**** 
    
    # The subdirectory used to store logs.                          
    log4j.appender.oss.subdir=/luk****/test/
Note After you edit the job parameters, restart the job. Logs generated can be viewed in the OSS bucket.

Scenario 2

  • Description

    Exclude logs of the specified types and send ERROR logs to the specified Logstore of Log Service.

  • Configuring job parameters
    # Disable the export of logs of the specified type to the specified path.
    log4j.logger.org.apache.hadoop=OFF
    
    # Fixed statement. Configure the LogHub appender.                                       
    log4j.appender.loghub = com.alibaba.blink.log.loghub.BlinkLogHubAppender
    
    # Send only ERROR logs to Log Service.
    log4j.appender.loghub.Threshold = ERROR
    # The name of a project in Log Service.
    log4j.appender.loghub.projectName = blink-errdumpsls-test
    
    # The name of a Logstore in Log Service.
    log4j.appender.loghub.logstore = logstore-3
    
    # The endpoint of Log Service.
    log4j.appender.loghub.endpoint = http://cn-shanghai****.sls.aliyuncs.com
    # The AccessKey ID.
    log4j.appender.loghub.accessKeyId = Tq****WR
    
    # The AccessKey secret.
    log4j.appender.loghub.accessKey = MJ****nfVx