HoloWeb supports one-click synchronization of Object Storage Service (OSS) data. Use the visual interface to quickly import data from OSS tables into Hologres internal tables for querying. This method delivers better query performance than creating a foreign table to query the data directly.
Prerequisites
-
You have logged on to an instance. For more information, see Log on to an instance.
-
You have created a foreign server. For more information, see Create a foreign server.
Procedure
Log on to the Hologres management console.
In the top navigation bar, select a region.
In the left-side navigation pane, click Go to HoloWeb.
-
After you log on to the destination database, click Metadata Management > OSS Data Lake Acceleration > One-click OSS Data Import to go to the One-click Data Import from OSS Data Lake page.
-
Configure the parameters for the Create OSS Data Import Task task. The SQL Script section automatically displays the SQL statement that corresponds to your settings.
Category
Parameter name
Description
Select Instance
Instance Name
The name of the instance that you have logged on to.
DLF Source Table
Server Name
The name of the foreign server that you created.
Database Name
The name of the Data Lake Formation (DLF) database that contains the OSS data to import.
Table Name
The name of the OSS table to import.
Hologres Destination Table
Database Name
The name of the destination database in Hologres.
Schema Name
The default value is public. You can select another schema for which you have permissions from the drop-down list.
Table Name
The name of the new Hologres internal table.
You do not need to create the table manually. After you enter the source DLF table name, the system automatically populates the same name for the Hologres internal table. You can rename the table.
If an internal table with the same name already exists:
-
For a non-partitioned table: The original table and its data are deleted and recreated when the task runs.
-
For a partitioned table: The original table and its data are not deleted. A new child partition table is created based on the partition value, and data is inserted into it. If the schema of the DLF table is different from the existing internal table, an error occurs.
Destination Table Description
The description for the new Hologres internal table. You can modify it.
Parameter Settings
GUC Parameters
The Grand Unified Configuration (GUC) parameters to set.
You usually need to enter these parameters to control the read and write concurrency of the job. The format is:
set <GUC>=<values>;. To set multiple GUC parameters, enter each one on a new line.Import Settings
Fields
The fields from the DLF table to import.
You can import some or all of the fields.
Partition Configuration
-
Partition Field: Select a partition field.
-
Data Timestamp: If the DLF table is partitioned by date, you can select a specific partition date. The system will import data for the specified date into the Hologres table.
Index Configuration
-
Storage Mode:
-
Column-oriented Storage: Suitable for various complex queries.
-
Row-oriented Storage: Suitable for point queries and scans based on a primary key.
-
Row-column Storage: Suitable for various scenarios, such as Online Analytical Processing (OLAP) complex analysis, high-concurrency point queries, and point queries not based on a primary key.
If you do not specify a storage mode, the system uses column store mode by default.
For more information, see Table storage formats: Column store, row store, and column and row store.
-
-
Data Lifecycle: The time to live (TTL) of the table data. The default is Permanent storage. If you specify a TTL, the DPI engine deletes the data at some point after the TTL expires, provided the data has not been modified within the specified period.
-
Binlog: Specifies whether to enable binary logging for the table. For more information, see Subscribe to Hologres binary logs.
NoteHologres V0.9 and later support the binary logging feature at the table level.
-
Lifecycle of Binary Logs: The TTL of the binary log in seconds. The default is 30 days (2,592,000 seconds). For more information, see Subscribe to Hologres binary logs.
After you complete these configurations, you can set properties for the fields to import from the DLF table. These properties include distribution column, segment column, cluster column, dictionary encoding column, and bitmap column.
-
-
Click Submit in the upper-right corner of the page. You can then view the execution details of the data import task on the One-click OSS Data Import page.
-
(Optional) In the Actions column for the import task, you can view its Details, Rerun, or Delete the task, or view its Execution History.
Query OSS data
After the OSS data import task runs successfully, Hologres automatically creates an internal table. Run the following SQL statement in Hologres to query the data:
SELECT * FROM <tablename>;
In the statement, tablename is the name of the Hologres internal table that you specified when you created the OSS data import task.