To connect PrestoDB to Tablestore, configure the Catalog and Schema for the Presto Server. Once connected, use the SQL examples in this topic to perform data operations.
Server configuration
Configure the Catalog and Schema for the Presto Server.
The Catalog configuration defines the connector settings and the Schema configuration mode. Configure the Schema using either dynamic configuration with a meta table or static configuration with a local file.
With dynamic configuration (meta table mode), no Schema configuration is needed before starting the Presto Server. Create the Schema and mapping tables after starting the SQL command-line interface (CLI).
Catalog configuration
The Catalog configuration file is located at etc/catalog/tablestore.properties in the PrestoDB installation folder.
connector.name=tablestore
tablestore.schema-mode=meta-table
#tablestore.schema-mode=file
tablestore.schema-file=/users/test/tablestore/presto/tablestore.schema
tablestore.meta-instance=metastoreinstance
tablestore.endpoint=http://metastoreinstance.cn-hangzhou,ots.aliyuncs.com/
tablestore.accessid=****************
tablestore.accesskey=**************************
tablestore.meta-table=meta_table
tablestore.auto-create-meta-table=true
|
Configuration item |
Example |
Required |
Description |
|
connector.name |
tablestore |
Yes |
The name of the connector. Must be set to |
|
tablestore.schema-mode |
meta-table |
Yes |
The Schema configuration mode. Valid values:
|
|
tablestore.schema-file |
/users/test/tablestore/presto/tablestore.schema |
Yes, conditional |
Required only when The full path to the local static file. For more information, see Schema configuration. |
|
tablestore.meta-instance |
metastoreinstance |
Yes, conditional |
Required only when The Tablestore instance used to store metadata. For more information, see Instances. Important
Make sure the configured instance exists in your Alibaba Cloud account. |
|
tablestore.endpoint |
http://metastoreinstance.cn-hangzhou,ots.aliyuncs.com/ |
Yes, conditional |
Required only when The endpoint of the Tablestore instance used to store metadata. For more information, see Endpoints. |
|
tablestore.accessid |
|
Yes, conditional |
Required only when The AccessKey ID and AccessKey secret of the Alibaba Cloud account or Resource Access Management (RAM) user that has access permissions for the metadata instance. |
|
tablestore.accesskey |
|
Yes, conditional |
|
|
tablestore.meta-table |
meta_table |
Yes, conditional |
Required only when The name of the Tablestore table used to store metadata. |
|
tablestore.auto-create-meta-table |
true |
No |
Applicable only when Specifies whether to automatically create the metadata table. Defaults to |
Schema configuration
Configure the Schema in one of two modes: dynamic configuration with a meta table (recommended) or static configuration with a local file.
Dynamic configuration with a meta table
No Schema configuration is needed before starting the Presto Server.
After connecting and starting the SQL CLI, use the
create schemaandcreate tablecommands to create the Schema and mapping tables. For more information, see Create a Schema and Create a mapping table.
Local static file configuration
The path to the local static file must match the full path specified for tablestore.schema-file in the Catalog configuration.
Create a file on the server with the following content:
{
"account" : {
"accessId" : "xxxxxxxx", ----- <Required> The AccessKey ID of the Alibaba Cloud account or RAM user.
"accessKey" : "xxxxxxxxxxxxxxx", ----- <Required> The AccessKey secret of the Alibaba Cloud account or RAM user.
},
"instances" : {
"mydb" : { ----- <Required> The Schema name in PrestoDB.
"instanceName" : "myinstance", ----- <Required> The name of the Tablestore instance.
"endpoint" : "http://myinstance.cn-hangzhou.ots.aliyuncs.com", ----- <Required> The endpoint of the Tablestore instance.
"tables": {
"mytable" : { ----- <Required> The table name in PrestoDB.
"originName" : "SampleTable", ----- <Optional> The name of the corresponding table in Tablestore (case-sensitive). If not configured, it is the same as the table name in Presto.
"columns" : [ ----- <Required> The metadata of the table. It must include all primary key columns, and the order of the primary key columns must be the same as the primary key order of the table.
{"name" : "gid", "type" : "bigint"},
{"name" : "uid", "type" : "bigint"},
{"name" : "c1", "type" : "boolean", "originName" : "col1"},
{"name" : "c2", "type" : "bigint", "originName" : "col2"},
{"name" : "C3", "type" : "varchar", "originName" : "col3"}
]
},
"anotherTable" : {
"originName" : "sampleTable2",
"columns" : [
{"name" : "gid", "type" : "bigint"},
{"name" : "uid", "type" : "bigint"},
{"name" : "a", "type" : "varchar"},
{"name" : "b", "type" : "varchar"},
{"name" : "c", "type" : "boolean"},
{"name" : "d", "type" : "bigint"},
{"name" : "e", "type" : "varchar"}
]
}
}
}
}
}
|
Configuration item |
Required |
Description |
|
|
account |
accessId |
Yes |
The AccessKey ID and AccessKey secret of the Alibaba Cloud account or RAM user. |
|
accessKey |
Yes |
||
|
instances(map(<schema_name> -> <schema_info>)) |
Yes |
A JSON map that maps Schema names to Schema information. Important
Schema names are not case-sensitive in PrestoDB. |
|
|
<schema_info> |
instanceName |
Yes |
The Tablestore instance name that corresponds to this Schema. For more information, see Instances. |
|
endpoint |
Yes |
The endpoint of the Tablestore instance. For more information, see Endpoints. |
|
|
tables(map(<table_name> -> <table_info>)) |
Yes |
A list of tables mounted to PrestoDB. Important
|
|
|
<table_info> |
originName |
No |
The actual table name in Tablestore.
If the connector cannot find the corresponding table in Tablestore, read and write operations on the table fail. |
|
columns(list([<column_info>])) |
Yes |
The metadata of the table, including all defined columns. Important
|
|
|
<column_info> |
name |
Yes |
The column name. Important
Column names are not case-sensitive in PrestoDB. |
|
type |
Yes |
The column type. Supported PrestoDB types that map to Tablestore column types: Bigint, Varchar, Varbinary, Double, and Boolean. Important
The column type in PrestoDB must match the column type in Tablestore. |
|
|
originName |
No |
The name of the corresponding column in the Tablestore data table. |
|
Common SQL examples
The following SQL examples show how to work with Tablestore data through PrestoDB. All examples use a Schema named testdb connected to the Tablestore instance myinstance, and a mapping table named exampletable with the schema (pk varchar, c1 double, c2 varchar, c3 varchar).
Create a Schema
A Schema configures access to a Tablestore instance and handles authentication. The following example creates a Schema named testdb to access the Tablestore instance myinstance.
CREATE SCHEMA tablestore.testdb
WITH (
endpoint = 'https://myinstance.cn-hangzhou.ots.aliyuncs.com',
instance_name = 'myinstance',
access_id = '************************',
access_key = '********************************'
);
|
Parameter |
Example |
Required |
Description |
|
endpoint |
https://myinstance.cn-hangzhou.ots.aliyuncs.com |
Yes |
The endpoint of the Tablestore instance. For more information, see Endpoints. |
|
instance_name |
myinstance |
Yes |
The name of the Tablestore instance. For more information, see Instances. |
|
access_id |
**** |
Yes |
The AccessKey ID of the Alibaba Cloud account or RAM user. |
|
access_key |
** |
Yes |
The AccessKey secret of the Alibaba Cloud account or RAM user. |
Create a mapping table
A mapping table corresponds to a physical table in a Tablestore instance.
Note the following when creating a mapping table:
Make sure field types in the mapping table match those in the Tablestore data table. For more information, see Field type mapping.
The mapping table name must match the table name in Tablestore.
The
table_nameproperty in the CREATE TABLE statement maps the PrestoDB table to the actual table in Tablestore. Multiple mapping tables with different names can point to the same Tablestore data table.The mapping table must include all primary key columns of the data table, but can include only a subset of attribute columns.
The names and order of primary key columns in the mapping table must match those in the Tablestore data table. Map each attribute column to its Tablestore counterpart using the
origin_nameparameter.
The following example creates a mapping table named exampletable that maps to the Tablestore data table of the same name.
CREATE TABLE if not exists exampletable
(
pk varchar,
c1 double with (origin_name = 'col1'),
c2 varchar with (origin_name = 'col2'),
c3 varchar with (origin_name = 'col3')
) WITH (
table_name = 'exampletable'
);
Schema operations
After creating a Schema, list all Schemas or select a Schema to use.
List all Schemas
show schemas;
Sample response:
Schema
--------------------
information_schema
testdb
testdb1
(3 rows)
Use a Schema
Select a Schema before performing operations on its tables.
Select a Schema before performing operations on the tables within it.
The following example selects the testdb Schema.
use testdb;
Sample response:
USE
Mapping table operations
After creating a mapping table, list tables in a Schema, view table metadata, or delete a mapping table.
List tables in a Schema
After selecting a Schema, list its tables.
show tables;
Sample response:
Table
-----------------
exampletable
sampletable
sampletabletest
table1
testtable
(5 rows)
Describe a table
The following example describes the exampletable table.
describe exampletable;
Sample response:
Column | Type | Extra | Comment
--------+---------+-------+---------
pk | varchar | |
c1 | double | |
c2 | varchar | |
c3 | varchar | |
(4 rows)
Delete a mapping table
The following example deletes the table1 mapping table.
drop table table1;
Sample response:
DROP TABLE
Data operations
After creating a mapping table, write data to and read data from the corresponding Tablestore table.
PrestoDB does not support UPDATE or DELETE operations on Tablestore tables.
Write data
-
Insert a single row
The following example inserts a row into
exampletable.insert into exampletable values('101', 22.0, 'Mary', '10002');Sample response:
INSERT: 1 row -
Batch import data
ImportantBefore importing data in a batch, make sure the destination table exists and its schema matches the source table.
The following example copies rows where
c1 < 50fromexampletabletosampletable.insert into sampletable select pk, c1, c2, c3 from exampletable where c1 < 50;Sample response:
INSERT: 3 rows
Read data
-
Query all rows
The following example reads all data from
exampletable.select * from exampletable;Sample response:
pk | c1 | c2 | c3 -----+------+------+------- 100 | 11.0 | Lily | 10001 101 | 22.0 | Mary | 10002 102 | 12.0 | Jim | 10003 (3 rows) -
Query rows with conditions
The following example queries rows in
exampletablewherec1 < 15andc3 = '10001'.select * from exampletable where c1 < 15 and c3 = '10001';Sample response:
pk | c1 | c2 | c3 -----+------+------+------- 100 | 11.0 | Lily | 10001 (1 row)
Analyze data
-
Calculate column average
The following example calculates the average of the
c1column inexampletable.select avg(c1) as Average from exampletable;Sample response:
Average --------- 15.0 (1 row) -
Count total rows
The following example counts all rows in
exampletable.select count(*) as total from exampletable;Sample response:
total ------- 3 (1 row)