All Products
Search

This Product

    E-MapReduce:Use Jindo AuditLog

    Document Center

    E-MapReduce:Use Jindo AuditLog

    Last Updated:Mar 25, 2026

    Jindo AuditLog allows you to audit operations in the namespaces that are in block storage mode. It records addition, deletion, and renaming operations in the namespaces.

    With AuditLog, you can:

    • Analyze namespace access patterns to understand how your cluster is being used

    • Detect abnormal requests and unauthorized operations

    • Track and investigate errors affecting specific paths or users

    Prerequisites

    Before you begin, make sure you have:

    Audit log format

    AuditLog records each file system operation as a single log entry. Use these fields to determine which users or IP addresses are most active, whether unauthorized operations occurred, and which paths were affected.

    Example log entry:

    2020-07-09 18:29:24.689 allowed=true ugi=hadoop (auth:SIMPLE) ip=127.0.0.1 ns=test-block cmd=CreateFileletRequest src=jfs://test-block/test/test.snappy.parquet dst=null perm=::rwxrwxr-x
    FieldDescriptionExample entry
    TimeThe timestamp in yyyy-MM-dd hh:mm:ss.SSS format.2020-07-09 18:29:24.689
    allowedWhether the operation was permitted. Valid values: true, false.true
    ugiThe user who performed the operation, including the authentication method.hadoop (auth:SIMPLE)
    ipThe client IP address.127.0.0.1
    nsThe name of the namespace in block storage mode.test-block
    cmdThe operation command.CreateFileletRequest
    srcThe source path.jfs://test-block/test/test.snappy.parquet
    dstThe destination path. This parameter can be left empty.null
    permThe operation permissions on the file.::rwxrwxr-x

    Configure AuditLog

    Important

    Make sure the OSS bucket already exists before enabling AuditLog. Set namespace.auditlog.oss.uri in the format oss://<yourbucket>/auditLog, replacing <yourbucket> with your bucket name. Each log file is capped at 5 GB; configure an OSS lifecycle rule to manage retention automatically.

    1. Go to the SmartData service.

      1. Log on to the Alibaba Cloud EMR console.

      2. In the top navigation bar, select the region where your cluster resides. Select the resource group as required. By default, all resources of the account appear.

      3. Click the Cluster Management tab.

      4. On the Cluster Management page that appears, find the target cluster and click Details in the Actions column.

      5. In the left-side navigation pane, click Cluster Service and then SmartData.

    2. Go to the namespace tab for the SmartData service.

      1. Click the Configure tab.

      2. Click the namespace tab in the Service Configuration section. namespace

    3. Add the AuditLog configuration.

      1. In the upper-right corner of the Service Configuration section, click Custom Configuration.

      2. In the Add Configuration Item dialog box, configure the following parameters.

        ParameterDescriptionRequired
        namespace.auditlog.enable
        • true: Enable AuditLog.
        • false: Disable AuditLog.
        		Yes
        namespace.auditlog.oss.uriThe OSS bucket where the log files generated by AuditLog are stored.

        Set this parameter in the format of oss://<yourbucket>/auditLog.

        Replace <yourbucket> with the OSS bucket name.

        		Yes
        namespace.auditlog.oss.accessKeyThe AccessKey ID used to access the OSS bucket.No
        namespace.auditlog.oss.accessSecretThe AccessKey secret used to access the OSS bucket.No
        namespace.auditlog.oss.endpointThe endpoint of the OSS bucket.No

      3. In the upper-right corner of the Service Configuration section, click Deploy Client Configuration.

      4. In the Cluster Activities dialog box, specify Description and click OK.

      5. In the Confirm message, click OK.

    4. Restart Jindo Namespace Service.

      1. Select Restart Jindo Namespace Service from the Actions drop-down list in the upper-right corner.

      2. In the Cluster Activities dialog box, specify Description and click OK.

      3. In the Confirm message, click OK.

    5. Set a retention period for log files.

      1. Log on to the OSS console.

      2. In the left-side navigation pane, click Buckets. On the page that appears, click the name of the created bucket.

      3. In the left-side navigation pane, choose Basic Settings > Lifecycle. In the Lifecycle section, click Configure.

      4. Click Create Rule. In the Create Rule dialog box, configure the parameters.

      5. Click OK.

    Analyze log files

    JindoFS allows you to use Shell commands to analyze the log files generated by AuditLog. Run jindo auditlog as a MapReduce job in the EMR console to find the most active commands or IP addresses in the log files.

    ParameterDescriptionRequired
    --srcThe OSS bucket where AuditLog stores log files. Defaults to the value of namespace.auditlog.oss.uri.No
    --nsThe namespace to analyze. Defaults to all namespaces in block storage mode.No
    --typeThe analysis target: ip for the most active IP addresses, cmd for the most active commands.Yes
    --minThe time range in minutes. Specify either --min or --day, not both.No
    --dayThe time range in days. --day 1 means the current day.No

    Example command:

    jindo auditlog --src oss://<yourbucket>/auditlog/ --ns test --type ip --day 1 --top 2

    Example output:

    16 openFileStatusRequest
6  deleteFileletRequest