All Products
Search
Document Center

E-MapReduce:Overview

Last Updated:Aug 14, 2023

This topic introduces the common keywords in Spark Streaming SQL and describes how to use these keywords.

Background information

Spark Streaming SQL is developed based on Spark Structured Streaming. All syntax and limits are compliant with Spark Structured Streaming. Therefore, you can use SQL statements to develop a streaming query job.

Usage notes

Whether an E-MapReduce (EMR) cluster supports the Spark Streaming SQL feature depends on the version of Spark.

  • The version of Spark that is used by EMR V3.X series is Spark 2.x. Only EMR V3.21.0 and later support Spark Streaming SQL.

  • The version of Spark that is used by EMR V4.X series and EMR V5.X series is Spark 3.x. Only EMR V5.8.0 and later support Spark Streaming SQL.

Common keywords

Type

Keyword

DDL

CREATE TABLE, CREATE TABLE AS SELECT, CREATE SCAN, and CREATE STREAM

DML

INSERT INTO and MERGE INTO

SELECT clause

SELECT FROM, WHERE, GROUP BY, JOIN, and UNION ALL

Use keywords as field names

If you want to use a keyword as a field name, enclose the keyword in backticks (`). Example: `value`.