Query and Export OSS Data via FILES Function in Lindorm OLAP - Lindorm

The FILES function in OLAP resource groups lets you read from and write to Object Storage Service (OSS) files directly in SQL — no data pipeline or external tool required. It acts as a table: use it as the destination of an INSERT statement to export data to OSS, or as the source of a SELECT statement to query data from OSS.

Prerequisites

Before you begin, make sure you have:

An OLAP resource group in Lindorm
An OSS bucket with the data you want to query, or an OSS path where you want to write output
An AccessKey ID and AccessKey secret with read or write access to the bucket

Syntax

FILES(
  "path"               = "s3://<bucket_name>/<path>/",
  "format"             = "parquet" | "csv",
  "compression"        = "uncompressed" | "gzip" | "snappy" | "zstd",
  "single"             = "true" | "false",
  "aws.s3.access_key"  = "<AccessKey ID>",
  "aws.s3.secret_key"  = "<AccessKey secret>",
  "aws.s3.endpoint"    = "<OSS endpoint>"
)

All parameters are specified as "key" = "value" pairs.

Parameters

Parameter	Description	Required	Default
`path`	OSS path. Must start with `s3://`. Example: `"s3://bucket_name/path/to/dir/"`	Yes	—
`format`	File format. Supported values: `parquet`, `csv`	Yes	—
`compression`	Compression algorithm. Supported values: `uncompressed`, `gzip`, `snappy`, `zstd`	No	Auto-detected when reading
`single`	Whether to write all output to a single file. If data exceeds 512 MB, the data is stored in multiple files.	No	`false`
`aws.s3.access_key`	OSS AccessKey ID	Yes	—
`aws.s3.secret_key`	OSS AccessKey secret	Yes	—
`aws.s3.endpoint`	OSS endpoint. Example: `"oss-cn-hangzhou-internal.aliyuncs.com"`	Yes	—

Export data to OSS

Use INSERT INTO FILES(...) to write query results to an OSS path.

INSERT INTO FILES(
  "path"               = "s3://bucket_name/path/to/dir/",
  "format"             = "parquet",
  "compression"        = "snappy",
  "single"             = "true",
  "aws.s3.access_key"  = "your_ak",
  "aws.s3.secret_key"  = "your_sk",
  "aws.s3.endpoint"    = "oss-cn-hangzhou-internal.aliyuncs.com"
) SELECT * FROM sometable;

This writes the query results to the specified OSS path.

Query OSS data

Use SELECT * FROM FILES(...) to query data directly from OSS files.

Do not point path to a folder alone. Specify individual files or use a glob pattern to match files — for example, /dir/* matches all files in a folder, and *.parquet matches files by extension.

SELECT * FROM FILES(
  "path"               = "s3://bucket_name/path/to/dir/*.parquet",
  "format"             = "parquet",
  "aws.s3.access_key"  = "your_ak",
  "aws.s3.secret_key"  = "your_sk",
  "aws.s3.endpoint"    = "oss-cn-hangzhou-internal.aliyuncs.com"
);

When reading, compression is optional. The system detects the compression format automatically.