Use ST_ImportFrom to import external files to a raster object - PolarDB

Imports an Object Storage Service (OSS) object, a MinIO object, or a Hadoop Distributed File System (HDFS) file into a Ganos raster object.

Syntax

raster ST_ImportFrom(cstring chunkTableName,
                     cstring url,
                     cstring storageOption default '{}',
                     cstring importOption default '{}');

Parameters

Parameter	Description
`chunkTableName`	The name of the chunk table that stores the raster data. Must comply with PolarDB table naming conventions.
`url`	The URL of the source file. For supported URL formats, see ST_CreateRast.
`storageOption`	A JSON string that controls how the raster data is stored. Default: `'{}'`.
`importOption`	A JSON string that controls the import process. Default: `'{}'`.

storageOption parameters

Storage structure

Parameter	Type	Default	Description
`chunking`	boolean	`true`	Specifies whether to store data as chunks.
`chunkdim`	string	Same as source	The chunk dimensions in `w,h,b` format. Takes effect only when `chunking` is `true`.

Encoding format

Parameter	Type	Default	Description
`compression`	string	`lz4`	The compression algorithm. Valid values: `none`, `jpeg`, `zlib`, `png`, `lzo`, `lz4`, `snappy`, `zstd`, `jp2k`.
`quality`	integer	`75`	The image quality after compression. Applies only to JPEG and JPEG 2000 compression.
`interleaving`	string	Same as source	The band interleaving method. Valid values: `bip` (Band Interleaved by Pixel), `bil` (Band Interleaved by Line), `bsq` (Band Sequential).
`blockendian`	string	`NDR`	The byte order for each data chunk. Valid values: `NDR` (little endian), `XDR` (big endian).
`celltype`	string	Same as source	The pixel type. Valid values: `1bb` (1 bit), `2bui` (2-bit unsigned integer), `4bui` (4-bit unsigned integer), `8bsi` (8-bit signed integer), `8bui` (8-bit unsigned integer), `16bsi` (16-bit signed integer), `16bui` (16-bit unsigned integer), `32bsi` (32-bit signed integer), `32bui` (32-bit unsigned integer), `32bf` (32-bit float), `64bsi` (64-bit signed integer), `64bui` (64-bit unsigned integer), `64bf` (64-bit float).

importOption parameters

Parameter	Type	Default	Description
`mapping_oss_file`	boolean	`false`	Specifies whether to load the OSS object into an in-memory file before reading. Some drivers do not optimize data stored in OSS, which causes a large number of small requests and reduces performance. Enabling this option maps the OSS object to a memory object to improve performance. Use the Global User Configuration (GUC) parameter ganos.raster.memory_oss_file_max_size to set the maximum file size eligible for in-memory loading.
`parallel`	integer	Value of the GUC parameter ganos.parallel.degree	The degree of parallelism. Valid values: 1 to 64.

Description

ST_ImportFrom creates a raster object and imports data from OSS, MinIO, or HDFS into it. The function stores the imported data in a chunk table named by the chunkTableName parameter.

To check which file formats are supported before importing, call ST_RasterDrivers:

SELECT * FROM ST_RasterDrivers();

Examples

All examples use OSS:// URLs. Replace <ak>, <ak_secret>, and the bucket path with your actual credentials and file location.

Import with default settings

-- Minimum required arguments: chunk table name and file URL
SELECT ST_ImportFrom(
    'chunk_table',
    'OSS://<ak>:<ak_secret>@oss-cn-beijing-internal.aliyuncs.com/mybucket/data/image.tif'
);

Import a NetCDF file with a named subset

-- Append :<subset_name> to the URL to select a subset within a NetCDF file
SELECT ST_ImportFrom(
    'chunk_table',
    'OSS://<ak>:<ak_secret>@oss-cn-beijing-internal.aliyuncs.com/mybucket/data/image.nc:hcc'
);

Control chunk size and compression

-- storageOption: set chunk dimensions to 128x128 pixels with 3 bands, disable compression
SELECT ST_ImportFrom(
    'chunk_table',
    'OSS://<ak>:<ak_secret>@oss-cn-beijing-internal.aliyuncs.com/mybucket/data/image.tif',
    '{"chunkdim":"(128,128,3)", "compression":"none"}'
);

Import with parallel processing

-- importOption: use 4 parallel workers to speed up the import
SELECT ST_ImportFrom(
    'chunk_table',
    'OSS://<ak>:<ak_secret>@oss-cn-beijing-internal.aliyuncs.com/mybucket/data/image.nc:hcc',
    '{}',
    '{"parallel": 4}'
);