EMR Serverless Spark centralizes Spark configuration management — covering global defaults, security-related information and files, and custom configuration files. Configuration changes take effect dynamically to meet diverse business requirements.
Configuration management provides two methods:
Manage Spark configuration templates: Define workspace-level default configurations that apply across all tasks. Supported operations include creating, editing, deleting, and restoring templates to their defaults.
-
Manage custom configuration files: Create and manage task-specific configuration files that override defaults for particular execution environments.
All task types in a workspace — including batch jobs, sessions, and Gateway — automatically apply these custom configuration files. No manual references are required.