This topic describes how to use Spark SQL to perform data analysis and interactive development on the Loghub data source.

CREATE TABLE syntax

CREATE TABLE tbName(columnName dataType [,columnName dataType]*)
USING loghub
OPTIONS(propertyName=propertyValue[,propertyName=propertyValue]*);

Parameters

Parameter Description Required
sls.project The name of the Log Service project. Yes
sls.store The name of the Logstore. Yes
access.key.id The AccessKey ID. Yes
access.key.secret The AccessKey secret. Yes
endpoint The endpoint of the Logstore, such as http://cn-shanghai.log.aliyun.com. Yes

Table schema

When you create a Loghub data table, you must explicitly define the fields in the data table.
  • Valid CREATE TABLE syntax:
    spark-sql> CREATE TABLE loghub_table_test(content string)
             > USING loghub
             > OPTIONS
             > (...)
    
    spark-sql> DESC loghub_table_test;
    content  string  NULL
    Time taken: 0.436 seconds, Fetched 1 row(s)
  • Invalid CREATE TABLE syntax:
    spark-sql> CREATE TABLE loghub_table_test
             > USING loghub
             > OPTIONS
             > (...)
    The following error is displayed:
    java.lang.IllegalArgumentException: requirement failed: Unable to infer the schema. The schema 
    specification is required to create the table. ;