Platform for AI (PAI) - Machine Learning Designer provides data preprocessing operators for LLM and common templates
Feb 28 2024
Platform for AI (PAI)Machine Learning Designer provides data preprocessing operators for LLM and common templates
Content
Intended customers: customers who require Large Language Model (LLM) pre-training and supervised fine-tuning (SFT). New features: High-quality data preprocessing is an important step in LLM application. Machine Learning Designer of Platform for AI (PAI) provides commonly used high-performance operators for data preprocessing, such as deduplication, standardization, and sensitive information masking. You can use large-scale distributed computing capabilities based on MaxCompute when you preprocess data for LLMs to improve efficiency and improve the reliability and performance of LLMs.
