Document Center
E-MapReduce
Product Introduction
What is E-MapReduce?
Benefits
Architecture
Scenarios
Versioning
Pricing
Overview
Price of Subscription E-MapReduce, not including ECS (cent/month per core)
Price of Pay-As-You-Go E-MapReduce, not including ECS (cent/hour per core)
Expiration and overdue payments
Renewal
Quick Start
Step 1: Select configuration
Step 2: Prerequisites
Step 3: Create a cluster
Step 4: Create and run a job
Cluster Planning and Configurations
Cluster planning
Role authorization
User management
Instance types
Components deployed on different instance nodes
Gateway clusters
ECS instances
Storage guide
D1 series
Enable access between classic networks and VPCs
Disaster recovery in E-MapReduce clusters
Configure clusters
Create a cluster
Cluster list and details
Expand a cluster
Release a cluster
Cluster renewal
Connect to a cluster using SSH
Service list
Cluster scripts
Access links and ports
Security groups
Create a Gateway
Manage cluster resources
Auto Scaling
Overview
Configure Auto Scaling by time
Scale By Rule
Preemptible instances in Auto Scaling
Auto Scaling records
VPC
MetaService
Metadata management
Manage Hive metadata
Manage Kafka metadata
Modify configurations
Upgrade node configuration
Disk expansion
Convert your billing method
Remove abnormal nodes
Multiple host groups
Third-party software
Bootstrap actions
Software configuration
Configure parameters
Monitoring and Alarms
Alarm management
Cluster alarm management
Data Development
Manage a workflow project
Edit a job
Ad hoc queries
Edit a workflow
Cluster template
Event codes for CloudMonitor
Jobs
Job operations
Time and date variables
Configure a Hive SQL job
Configure a Hadoop MapReduce job
Configure a Hive job
Configure a Pig job
Configure a Spark job
Configure a Spark SQL job
Configure a Shell job
Configure a Sqoop job
Configure a Flink job
Configure a Spark Streaming job
Old EMR Scheduling (Soon will be unavailable)
Execution plans
Create an execution plan
Manage an execution plan
Execution plan list
View job results and logs
Parallel execution of multiple execution plans
Notebooks
Introduction
Operations
Examples
Query bank employee information
Video playback data
Create a job
Open Source Components
Hue
Oozie
Zeppelin
ZooKeeper
Kafka
Quick start
Cross-cluster access to Kafka
Kafka Ranger
Kafka SSL
Common Kafka problems
Kafka Manager
Druid
Introduction to Druid
Quick start
Ingestion Spec
Kafka Indexing Service
LOG Indexing Service
Tranquility
Superset
Common Druid problems
Presto
What is Presto?
Quick start
System structure
Basic concepts
Command line tool
Uses JDBC
Implement authentication with ApacheDS
Instructions
Overview
SQL manual
Data types
Common functions and operators
Decimal function
Binary functions
Bitwise functions
String functions
Mathematical functions and operators
Date and time processing functions
Conditional expressions
Regular expression
Comparison functions and operators
Aggregate functions
Conversion functions
Logical operators
SQL statements
SQL statement overview
ALTER SCHEMA
ALTER TABLE
CALL
COMMIT
CREATE SCHEMA
CREATE TABLE
CREATE TABLE AS
CREATE VIEW
DEALLOCATE PREPARE
DELETE
DESCRIBE
DESCRIBE INPUT
DESCRIBE OUTPUT
DROP SCHEMA
DROP TABLE
DROP VIEW
EXECUTE
EXPLAIN
EXPLAIN ANALYZE
GRANT
INSERT
PREPARE
RESET SESSION
REVOKE
ROLLBACK
SELECT clause
GROUP BY clause
HAVING clause
Joins
LIMIT clause
ORDER BY clause
SELECT
TABLESAMPLE
UNNEST
WITH clause
Subquery
Set operations
SET SESSION
SHOW CATALOGS
SHOW COLUMNS
SHOW CREATE TABLE
SHOW CREATE VIEW
SHOW FUNCTIONS
SHOW GRANTS
SHOW PARTITIONS
SHOW SCHEMAS
SHOW SESSION
SHOW TABLES
START TRANSACTION
USE
VALUES
Common connectors
JMX connector
Kafka connector
System connector
TensorFlow
Knox
Flume
Use Flume
Configure Flume
Use LogHub Source to move data from non-EMR clusters to HDFS of EMR clusters
Sqoop
Component authorization
HDFS authorization
YARN authorization
Hive authorization
HBase authorization
Kafka authorization
Ranger
Introduction to Ranger
Integrate Ranger into HDFS
Integrate Ranger into Hive
HBase configurations
Data masking in Hive
Kafka Configuration
Kerberos authentication
Introduction to Kerberos
Compatible with the MIT Kerberos authentication protocol
RAM authentication
LDAP authentication
Execution plan authentication
Cross-region access
Developer Guide
Use Python on E-MapReduce
Preparations
Development preparations
Configure the OSS URI to use E-MapReduce
Use sample projects
Spark
Preparations
Parameter description
Operate OSS files
Use Spark to access OSS
Use MaxCompute in Spark
Use Spark Streaming to consume MQ data
Consume Table Store data in Spark
Use Spark Streaming to consume MNS data
Use Spark to write data to HBase
Use Spark Streaming to process Kafka data
Use Spark and write data to MySQL
Configure Spark-Submit parameters
Hadoop
Parameter description
Create and run MapReduce jobs
Create and run a Hive job
Create and run a Pig job
Create a Hadoop streaming job
Process Table Store data in Hive
Process Table Store data in MR
Create an HBase cluster and use the HBase storage service
Back up HBase
Guide for E-MapReduce cluster operations
Kernel Enhancement
JindoFS user guide
Best Practices
Use E-MapReduce to collect metrics from a Kafka client
Use E-MapReduce to process offline jobs
Submit Storm topologies to process data in Kafka on E-MapReduce
Use ES-Hadoop on E-MapReduce
Use Mongo-Hadoop on E-MapReduce
Deep learning with Analytics Zoo on E-MapReduce
Adaptive execution of Spark SQL
E-MapReduce data migration solution
Use EMR Data Science clusters for deep learning
Use Flink jobs to process OSS data
Submit a Flink job to process OSS data using EMR
Connect to ApsaraDB for HBase using E-MapReduce Hive
Use EMR for real-time MySQL binlog transmission
Run Flume on a Gateway node to synchronize data
OSS ACL
Configure a network connection for using Sqoop to transfer data from a database to an EMR cluster
Use E-MapReduce to submit a Spark Streaming job for consuming Kafka data
Use Kafka Connect to migrate data
API Reference
List of operations by function
Description of calls
Request structure
Sign signatures
Common parameters
Responses
Cluster
CreateClusterV2
CreateClusterTemplate
CreateClusterWithTemplate
DeleteClusterTemplate
DescribeClusterTemplate
DescribeClusterV2
DescribeEmrMainVersion
ListClusterHost
ListClusters
ListClusterTemplates
ListClusterServiceQuickLink
ListEmrAvailableConfig
ListEmrAvailableResource
ListEmrMainVersion
ModifyClusterTemplate
ModifyClusterName
ReleaseCluster
ResizeClusterV2
Cluster services
ModifyResourcePoolSchedulerType
ModifyResourceQueue
ModifyClusterServiceConfig
CreateResourcePool
CreateResourceQueue
DeleteResourcePool
DeleteResourceQueue
RefreshClusterResourcePool
RunClusterServiceAction
DescribeClusterOperationHostTaskLog
ListClusterOperationHostTask
ListClusterOperationHost
ModifyResourcePool
ListClusterServiceConfigHistory
DescribeClusterServiceConfig
DescribeClusterResourcePoolSchedulerType
ListResourcePool
ListClusterServiceComponentHealthInfo
ListClusterHostComponent
ListClusterOperation
ListClusterService
DescribeClusterService
AddClusterService
DescribeClusterServiceConfigTag
Data development
CloneFlow
CloneFlowJob
CreateFlowCategory
CreateFlowForWeb
CreateFlowJob
CreateFlowProject
CreateFlowProjectClusterSetting
CreateFlowProjectUser
DeleteFlow
DeleteFlowCategory
DeleteFlowJob
DeleteFlowProject
DeleteFlowProjectClusterSetting
DeleteFlowProjectUser
DescribeFlow
DescribeFlowCategory
DescribeFlowCategoryTree
DescribeFlowInstance
DescribeFlowJob
DescribeFlowNodeInstance
DescribeFlowNodeInstanceContainerLog
DescribeFlowNodeInstanceLauncherLog
DescribeFlowProject
DescribeFlowProjectClusterSetting
KillFlowJob
ListFlow
ListFlowCluster
ListFlowClusterAll
ListFlowClusterAllHosts
ListFlowClusterHost
ListFlowInstance
ListFlowJob
ListFlowJobHistory
ListFlowNodeInstanceContainerStatus
ListFlowNodeSqlResult
ListFlowProject
ListFlowProjectClusterSetting
ListFlowProjectUser
ModifyFlowForWeb
ModifyFlowProject
ModifyFlowJob
RerunFlow
ResumeFlow
SubmitFlow
SubmitFlowJob
SuspendFlow
Old EMR Scheduling
Job
CreateJob
ModifyJob
DeleteJob
DescribeJob
ListJobs
Execution plan
CreateExecutionPlan
DescribeExecutionPlan
ListExecutionPlans
ModifyExecutionPlan
DeleteExecutionPlan
RunExecutionPlan
ResumeExecutionPlanScheduler
SuspendExecutionPlanScheduler
ListExecutionPlanInstances
ListJobExecutionInstances
KillExecutionJobInstance
ListJobInstanceWorkers
Data type
ClusterInfo
EcsInstance
EcsOrder
BootstrapAction
ExecutionPlanInfo
ExecutionPlanInstance
JobInfo
JobInstance
JobInstanceWorker
SoftwareInfo
FailReason
Enumerations
SDK Reference
E-MapReduce SDK release notes
Create an AccessKey
Java SDK
Download SDKs
Get started
Reference project
Python SDK
Install SDK
Sample code
FAQ
Q&A about E-MapReduce
Error messages
The FAQ about product use
Job exception
Use execution plans
O&M FAQ
Errors and Troubleshooting
Appendix
Cluster management page
Notes on E-MapReduce versions
Error code list
Status list
Configure cluster ports
General Reference
Guidelines for the use of documentation
Alibaba Cloud documentation
Instructions for open source documentation
Open-Source Documentation Code of Conduct
Editing Open-Source Documentation
Alibaba Cloud console
Regions and zones
Service limits
Best practice of primary account security
Create an AccessKey
ICP Filing
Glossary
all-products-head
This Product
This Product
All Products
E-MapReduce
E-MapReduce
All Products
Product Introduction
Download PDF
What is E-MapReduce?
Benefits
Architecture
Scenarios
Versioning
Show more
Show less
Pricing
Download PDF
Overview
Price of Subscription E-MapReduce, not including ECS (cent/month per core)
Price of Pay-As-You-Go E-MapReduce, not including ECS (cent/hour per core)
Expiration and overdue payments
Renewal
Show more
Show less
Quick Start
Download PDF
Step 1: Select configuration
Step 2: Prerequisites
Step 3: Create a cluster
Step 4: Create and run a job
Show more
Show less
Cluster Planning and Configurations
Download PDF
Cluster planning
Configure clusters
Modify configurations
Third-party software
Show more
Show less
Monitoring and Alarms
Alarm management
Show more
Show less
Data Development
Download PDF
Manage a workflow project
Edit a job
Ad hoc queries
Edit a workflow
Cluster template
Event codes for CloudMonitor
Jobs
Old EMR Scheduling (Soon will be unavailable)
Show more
Show less
Open Source Components
Download PDF
Hue
Oozie
Zeppelin
ZooKeeper
Kafka
Druid
Presto
TensorFlow
Knox
Flume
Sqoop
Component authorization
Kerberos authentication
Show more
Show less
Developer Guide
Download PDF
Use Python on E-MapReduce
Preparations
Spark
Hadoop
Create an HBase cluster and use the HBase storage service
Back up HBase
Guide for E-MapReduce cluster operations
Show more
Show less
Kernel Enhancement
Download PDF
JindoFS user guide
Show more
Show less
Best Practices
Download PDF
Use E-MapReduce to collect metrics from a Kafka client
Use E-MapReduce to process offline jobs
Submit Storm topologies to process data in Kafka on E-MapReduce
Use ES-Hadoop on E-MapReduce
Use Mongo-Hadoop on E-MapReduce
Deep learning with Analytics Zoo on E-MapReduce
Adaptive execution of Spark SQL
E-MapReduce data migration solution
Use EMR Data Science clusters for deep learning
Use Flink jobs to process OSS data
Submit a Flink job to process OSS data using EMR
Connect to ApsaraDB for HBase using E-MapReduce Hive
Use EMR for real-time MySQL binlog transmission
Run Flume on a Gateway node to synchronize data
OSS ACL
Configure a network connection for using Sqoop to transfer data from a database to an EMR cluster
Use E-MapReduce to submit a Spark Streaming job for consuming Kafka data
Use Kafka Connect to migrate data
Show more
Show less
API Reference
Download PDF
List of operations by function
Description of calls
Cluster
Cluster services
Data development
Old EMR Scheduling
Data type
Enumerations
Show more
Show less
SDK Reference
E-MapReduce SDK release notes
Create an AccessKey
Java SDK
Python SDK
Show more
Show less
FAQ
Download PDF
Q&A about E-MapReduce
Error messages
The FAQ about product use
Job exception
Use execution plans
O&M FAQ
Show more
Show less
Errors and Troubleshooting
Download PDF
Appendix
Show more
Show less
Free Trial
Free Trial