Object Storage Service:Access OSS Tables with the Trino native S3 file system

Modify trino-filesystem-s3

Locate the S3 location validation logic and add oss to the set of allowed schemes:

// Before modification
checkArgument(Set.of("s3", "s3a", "s3n").contains(location.scheme().get()), "Wrong scheme for S3 location: %s", location);

// After modification
checkArgument(Set.of("s3", "s3a", "s3n", "oss").contains(location.scheme().get()), "Wrong scheme for S3 location: %s", location);

Modify trino-filesystem-manager

Add a binding for the oss scheme in the file system factory bindings:

factories.addBinding("s3").to(Key.get(TrinoFileSystemFactory.class, FileSystemS3.class));

factories.addBinding("s3a").to(Key.get(TrinoFileSystemFactory.class, FileSystemS3.class));

factories.addBinding("s3n").to(Key.get(TrinoFileSystemFactory.class, FileSystemS3.class));
// Add this line after the existing s3/s3a/s3n bindings:
factories.addBinding("oss").to(Key.get(TrinoFileSystemFactory.class, FileSystemS3.class));

Build and replace the JAR packages

After building the two modules, replace the original JAR packages in your Trino deployment directory with the new ones and restart the Trino service.

# Build trino-filesystem-s3
mvn -pl lib/trino-filesystem-s3 -am package -DskipTests

# Build trino-filesystem-manager
mvn -pl lib/trino-filesystem-manager -am package -DskipTests

# Replace the JAR packages (assuming Trino is installed in /opt/trino)
cp lib/trino-filesystem-s3/target/trino-filesystem-s3-*.jar /opt/trino/lib/
cp lib/trino-filesystem-manager/target/trino-filesystem-manager-*.jar /opt/trino/lib/

Method 1: Use ossutil

1. Install or upgrade ossutil

Install ossutil 2.3.0 or later. If you have an older version of ossutil installed, run the following command to upgrade to the latest version:

ossutil update -f

2. Configure credentials

Run the ossutil config command and follow the prompts to enter your AccessKey ID, AccessKey Secret, and region.

3. Create a table bucket

ossutil tables-api create-table-bucket --name <table_bucket_name> --endpoint http://<endpoint> --region <region>

After the command succeeds, the output includes the Table Bucket ARN. Record this ARN for later use.

4. Create a namespace

ossutil tables-api create-namespace --table-bucket-arn <table_bucket_arn> --namespace <namespace_name> --endpoint http://<endpoint>

5. Create a table

You can create an Iceberg table in one of the following ways:

• Create it using another compute engine, such as Spark.

• Create it using ossutil: save the table schema to a JSON file, and then call create-table.

  The following example schema file, named schema.json, defines three fields:

  {
    "iceberg": {
      "schema": {
        "fields": [
          {"name": "event_id", "type": "string", "required": true},
          {"name": "event_time", "type": "string"},
          {"name": "event_type", "type": "string"}
        ]
      }
    }
  }

  Create a table based on the schema file:

  ossutil tables-api create-table --table-bucket-arn <bucket_arn> --namespace <namespace_name> --name <table_name> --format ICEBERG --metadata file://<file_path> --endpoint http://<endpoint>

Method 2: Use the AWS CLI

OSS Tables is compatible with the S3 Tables API, so you can also use the AWS CLI to manage a table bucket.

1. Install the AWS CLI

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

2. Configure credentials

Run the aws configure command and follow the prompts to enter your AccessKey ID, AccessKey Secret, and region.

3. Create a table bucket

aws s3tables --endpoint http://<endpoint> create-table-bucket --region <region> --name <table_bucket_name>

After the command succeeds, the output includes the Table Bucket ARN.

4. Create a namespace

aws s3tables --endpoint http://<endpoint> create-namespace --table-bucket-arn <table_bucket_arn> --namespace <namespace_name>

5. Create a table

• Create a table using another compute engine, such as Spark.

• Use the AWS CLI. Save the complete parameters to a JSON file, such as create-table.json, and then call create-table.

  {
      "tableBucketARN": "<bucket_arn>",
      "namespace": "<namespace_name>",
      "name": "<table_name>",
      "format": "ICEBERG",
      "metadata": {
          "iceberg": {
              "schema": {
                  "fields": [
                       {"name": "event_id", "type": "string","required": true},
                       {"name": "event_time", "type": "string"},
                       {"name": "event_type", "type": "string"}
                  ]
              }
          }
      }
  }

  aws s3tables --endpoint http://<endpoint> create-table --cli-input-json file://<file_path>

6. Manage background maintenance tasks

OSS Tables supports automatic background maintenance for Iceberg tables, such as file cleanup and file compaction. You can use the AWS CLI to query and configure these maintenance tasks.

Query the status of a table maintenance job:

aws s3tables get-table-maintenance-job-status \
   --table-bucket-arn="<bucket_arn>" \
   --namespace="<namespace_name>" \
   --name="<table_name>" 

Configure a bucket-level maintenance policy (file cleanup):

aws s3tables put-table-bucket-maintenance-configuration \
   --table-bucket-arn <table_bucket_arn> \
   --type icebergUnreferencedFileRemoval \
   --value '{"status":"enabled","settings":{"icebergUnreferencedFileRemoval":{"unreferencedDays":4,"nonCurrentDays":10}}}' 

Configure a table-level maintenance policy (small file compaction):

aws s3tables put-table-maintenance-configuration \
   --table-bucket-arn <bucket_arn> \
   --type icebergCompaction \
   --namespace <namespace_name> \
   --name <table_name> \

Create a catalog properties file

In Trino's etc/catalog/ directory, create a properties file (for example, oss_tables.properties):

connector.name=iceberg
iceberg.catalog.type=rest
iceberg.rest-catalog.uri=https://<region>-internal.oss-tables.aliyuncs.com/iceberg
iceberg.rest-catalog.warehouse=<Table Bucket ARN>
iceberg.rest-catalog.security=SIGV4
iceberg.rest-catalog.signing-name=osstables
iceberg.rest-catalog.view-endpoints-enabled=false
fs.hadoop.enabled=false
fs.native-s3.enabled=true
s3.endpoint=https://oss-<region>-internal.aliyuncs.com
s3.region=<region>
s3.aws-access-key=<AccessKey ID>
s3.aws-secret-key=<AccessKey Secret>
s3.path-style-access=true

Parameters

Example: Create a table

CREATE TABLE ${catalog}.${namespace}.orders (
    order_id    BIGINT,
    customer    VARCHAR,
    amount      DECIMAL(10,2),
    order_date  DATE,
    created_at  TIMESTAMP(6)
)
WITH (
    format = 'PARQUET',
    partitioning = ARRAY['day(order_date)'],
);

Example: Query data

SELECT * FROM ${catalog}.${namespace}.orders limit 10;

Resources

• Table Bucket ARN: acs:osstables:<region>:<alibaba_cloud_account_id>:bucket/<bucket_name>

• Table ARN: acs:osstables:<region>:<alibaba_cloud_account_id>:bucket/<bucket_name>/table/<table_id>

Actions

The following table lists the actions supported by OSS Tables and indicates whether they allow cross-account access.

Iceberg REST operation permissions

The following table lists the OSS actions required for each Iceberg REST Catalog operation.