Content

Target customers: 1. Developers, maintenance engineers, and operations engineers that need to parse the complex logs uploaded to Log Service. These logs include gateway, service, router, and complex application logs. The logs are uploaded through syslog, logtail simple mode, or Log Service SDK. 2. Operations engineers and analysts that need to process the complex JSON data or text in the retrieved logs. For example, they may need to extract data, add data, or mask sensitive data. 3. Developers and operations engineers that need to process data before importing the data. For example, they may need to filter data, mask sensitive data, or standardize data. Features released: The ETL function has the following features: 1. Realtime stream processing for auto load balancing and data recovery. 2. Batch process history data. 3. Custom orchestration: dispatch, transform, split events, keep events, drop events, keep fields, drop fields, auto key-value pair extraction, and rename fields. 4. Use built-in conversion modules: imputed value processing, RegExp, CSV, dictionary, and external CSV mapping, auto key-value pair extraction, JSON code expansion, extraction, filtering, splitting, and merging. 5. Support seamless Python database loading and referencing simple UDFs.