OSS Tables
OSS Tables是阿里雲Object Storage Service提供的結構化資料湖儲存服務,基於Apache Iceberg開放表格式,為AI訓練資料管理和OLAP分析情境提供即開即用的低成本、高效能結構化資料存放區能力。OSS Tables內建自動化資料維護機制(壓縮、快照管理、未引用檔案清理),實現零營運的Serverless資料湖體驗。
申請開通
當前OSS Tables,在華東1(杭州)、華東2(上海)、華東6(烏蘭察布)、新加坡、德國(法蘭克福)地區開放免費邀測,接受企業實名認證客戶申請,請登入阿里雲帳號後,前往Table Bucket 列表頁面申請邀測資格。參與邀測的使用者請隨時關注產品收費通知,避免產生非預期費用。
工作原理
OSS Tables採用三層結構(Table Bucket - Namespace - Table)組織資料,提供從資源隔離到資料表管理的完整層次化管理能力。
Table Bucket:頂層資源容器,是管理和隔離表資源的雲資源單元。Table Bucket的類型為customer,用於承載使用者自建的結構化資料表。Table Bucket擁有獨立的ARN(Alibaba Cloud Resource Name),格式為
acs:osstables:{region}:{uid}:bucket/{bucket-name},Namespace:邏輯隔離層,類似關聯式資料庫中的Schema概念,用於在同一個Table Bucket內對不同業務域的資料表進行分組和許可權隔離。
Table:實際的資料表,儲存格式為Apache Iceberg。每張Table擁有獨立的ARN,格式為
acs:osstables:{region}:{uid}:bucket/{bucket-name}/table/{table-id},支援細粒度的許可權控制。Table在建立時需要指定Schema,支援long、string、timestamptz、boolean、int、float、double、decimal(P,S)、date、time、timestamp、uuid、binary、fixed[L]等欄位類型。
Table Bucket
Table Bucket是OSS Tables的頂層資源容器,用於儲存和管理Apache Iceberg格式的表。
建立Table Bucket
同一個阿里雲帳號在同一個地區最多建立10個Table Bucket,Table Bucket名稱在同一地區內全域唯一。
控制台
登入OSS控制台,在左側導覽列選擇Table Bucket 列表。
單擊建立 Table Bucket。
在建立彈窗中配置以下參數:
名稱:輸入Table Bucket名稱,長度為3~32個字元,只允許小寫字母、數字和連字號(-),不能以連字號(-)開頭或結尾,不能以
oss、acs、aliyun、alibaba、aws為首碼。地區:選擇地區,例如華北2(北京)。
資料目錄格式:預設為iceberg。
Endpoint:根據所選地區自動產生。
冗餘類型:當前為本地冗餘。
服務端加密方式:選擇不加密或AES256。
單擊確定。
ossutil
使用ossutil建立Table Bucket的命令樣本如下:
ossutil tables-api create-table-bucket --name mytablebucketSDK
Python
使用Python SDK建立Table Bucket的程式碼範例如下:
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="create table bucket sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--name', help='The name of the table bucket.', required=True)
parser.add_argument('--sse-algorithm', help='The server-side encryption algorithm.')
parser.add_argument('--kms-key-arn', help='The KMS key ARN for encryption.')
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
encryption_configuration = None
if args.sse_algorithm is not None or args.kms_key_arn is not None:
encryption_configuration = oss_tables.models.EncryptionConfiguration(
sse_algorithm=args.sse_algorithm,
kms_key_arn=args.kms_key_arn,
)
result = client.create_table_bucket(oss_tables.models.CreateTableBucketRequest(
name=args.name,
encryption_configuration=encryption_configuration,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id},'
f' arn: {result.arn}')
if __name__ == "__main__":
main()Go
使用Go SDK建立Table Bucket的程式碼範例如下:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
name string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&name, "name", "", "The name of the bucket.")
}
func main() {
flag.Parse()
if len(name) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, bucket name required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.CreateTableBucket(context.TODO(), &tables.CreateTableBucketRequest{
Name: oss.Ptr(name),
})
if err != nil {
log.Fatalf("failed to create table bucket %v", err)
}
log.Printf("create table bucket result:%#v\n", result)
}Java
使用Java SDK建立Table Bucket的程式碼範例如下:
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class CreateTableBucketSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String name = "mytablebucket";
String sseAlgorithm = "AES256";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
CreateTableBucketRequest.Builder requestBuilder = CreateTableBucketRequest.newBuilder()
.name(name);
EncryptionConfiguration encryptionConfig = EncryptionConfiguration.newBuilder()
.sseAlgorithm(sseAlgorithm)
.build();
requestBuilder.encryptionConfiguration(encryptionConfig);
CreateTableBucketResult result = client.createTableBucket(requestBuilder.build());
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Created table bucket with ARN: %s%n", result.arn());
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用CreateTableBucket介面建立Table Bucket。
Table Bucket建立後,系統會預設啟用自動資料維護配置(icebergUnreferencedFileRemoval),自動清理Iceberg表中未引用的檔案。預設參數為:unreferencedDays=3(檔案未被引用超過3天后可被清理)、nonCurrentDays=10(非目前的版本檔案保留10天)。
查詢Table Bucket
控制台
登入OSS控制台,在左側導覽列選擇Table Bucket 列表。
單擊目標Table Bucket名稱,進入Table Bucket詳情頁。
在詳情頁查看以下基本資料:
Bucket 擁有者賬戶 ID:Table Bucket所屬的阿里雲帳號ID。
資源標識符(ARN):Table Bucket的全域唯一資源標識。
Bucket 類型:顯示為Customer,表示由使用者建立。
建立時間:Table Bucket的建立時間。
儲存類型:顯示為標準儲存-本地冗餘。
服務端加密方式:顯示當前加密配置。
ossutil
使用ossutil查詢Table Bucket資訊的命令樣本如下:
ossutil tables-api get-table-bucket --table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucketSDK
Python
使用Python SDK查詢Table Bucket資訊的程式碼範例如下:
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="get table bucket sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.get_table_bucket(oss_tables.models.GetTableBucketRequest(
table_bucket_arn=args.table_bucket_arn,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id},'
f' arn: {result.arn},'
f' name: {result.name},'
f' owner account id: {result.owner_account_id},'
f' created at: {result.created_at},'
f' table bucket id: {result.table_bucket_id},'
f' type: {result.type}')
if __name__ == "__main__":
main()Go
使用Go SDK查詢Table Bucket資訊的程式碼範例如下:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.GetTableBucket(context.TODO(), &tables.GetTableBucketRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
})
if err != nil {
log.Fatalf("failed to get table bucket %v", err)
}
log.Printf("get table bucket result:%#v\n", result)
}Java
使用Java SDK查詢Table Bucket資訊的程式碼範例如下:
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class GetTableBucketSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
GetTableBucketResult result = client.getTableBucket(GetTableBucketRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.build());
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Table bucket ARN: %s%n", result.arn());
System.out.printf("Table bucket name: %s%n", result.name());
System.out.printf("Table bucket type: %s%n", result.type());
System.out.printf("Owner account ID: %s%n", result.ownerAccountId());
System.out.printf("Table bucket ID: %s%n", result.tableBucketId());
System.out.printf("Created at: %s%n", result.createdAt());
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用GetTableBucket介面查詢Table Bucket的詳細資料。
列舉Table Bucket
控制台
登入OSS控制台,在左側導覽列選擇 Table Bucket 列表。
在列表頁查看當前帳號下所有Table Bucket的名稱、地區、建立時間等資訊。
ossutil
使用ossutil列舉Table Bucket的命令樣本如下:
ossutil tables-api list-table-bucketsSDK
Python
使用Python SDK列舉Table Bucket的程式碼範例如下:
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="list table buckets sample")
parser.add_argument('--region', help='The region in which the table buckets are located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--prefix', help='The prefix used to filter the table buckets.')
parser.add_argument('--max-buckets', help='The maximum number of buckets to return.', type=int)
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.list_table_buckets(oss_tables.models.ListTableBucketsRequest(
prefix=args.prefix,
max_buckets=args.max_buckets,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id},'
f' continuation token: {result.continuation_token}')
if result.table_buckets:
for i, bucket in enumerate(result.table_buckets):
print(f'bucket {i + 1}:'
f' arn: {bucket.arn},'
f' name: {bucket.name},'
f' id: {bucket.table_bucket_id},'
f' owner account id: {bucket.owner_account_id},'
f' created at: {bucket.created_at}')
if __name__ == "__main__":
main()Go
使用Go SDK列舉Table Bucket的程式碼範例如下:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
}
func main() {
flag.Parse()
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
p := client.NewListTableBucketsPaginator(&tables.ListTableBucketsRequest{})
var i int
log.Println("Table Buckets:")
for p.HasNext() {
i++
page, err := p.NextPage(context.TODO())
if err != nil {
log.Fatalf("failed to get page %v, %v", i, err)
}
// Log the objects found
for _, b := range page.TableBuckets {
log.Printf("Table Bucket %v,%v,%v,%v,%v,%v\n", oss.ToString(b.Name), oss.ToString(b.Arn), oss.ToString(b.OwnerAccountId), oss.ToString(b.TableBucketId), oss.ToString(b.CreatedAt), oss.ToString(b.Type))
}
}
}Java
使用Java SDK列舉Table Bucket的程式碼範例如下:
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class ListTableBucketsSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
int maxBuckets = 10;
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
ListTableBucketsRequest request = ListTableBucketsRequest.newBuilder()
.maxBuckets(maxBuckets)
.build();
ListTableBucketsResult result = client.listTableBuckets(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Continuation token: %s%n", result.continuationToken());
System.out.printf("Number of buckets: %d%n", result.tableBuckets().size());
for (int i = 0; i < result.tableBuckets().size(); i++) {
TableBucketSummary bucket = result.tableBuckets().get(i);
System.out.printf("Bucket %d:%n", i + 1);
System.out.printf(" ARN: %s%n", bucket.arn());
System.out.printf(" Name: %s%n", bucket.name());
System.out.printf(" ID: %s%n", bucket.tableBucketId());
System.out.printf(" Owner Account ID: %s%n", bucket.ownerAccountId());
System.out.printf(" Created At: %s%n", bucket.createdAt());
}
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用ListTableBuckets介面列舉Table Bucket。
刪除Table Bucket
刪除Table Bucket前,必須先刪除該Table Bucket下的所有Table和Namespace。不支援刪除非空的Table Bucket。刪除操作無法復原,請謹慎操作。
控制台
登入OSS控制台。在左側導覽列選擇Table Bucket 列表。
在目標Table Bucket對應的操作列,單擊刪除。
在確認彈窗中單擊確定。
ossutil
使用ossutil刪除Table Bucket的命令樣本如下:
ossutil tables-api delete-table-bucket --table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucketSDK
Python
使用Python SDK刪除Table Bucket的程式碼範例如下:
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="delete table bucket sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.delete_table_bucket(oss_tables.models.DeleteTableBucketRequest(
table_bucket_arn=args.table_bucket_arn,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id}')
if __name__ == "__main__":
main()Go
使用Go SDK刪除Table Bucket的程式碼範例如下:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.DeleteTableBucket(context.TODO(), &tables.DeleteTableBucketRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
})
if err != nil {
log.Fatalf("failed to delete table bucket %v", err)
}
log.Printf("delete table bucket result:%#v\n", result)
}Java
使用Java SDK刪除Table Bucket的程式碼範例如下:
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class DeleteTableBucketSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
DeleteTableBucketRequest request = DeleteTableBucketRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.build();
DeleteTableBucketResult result = client.deleteTableBucket(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.println("Successfully deleted table bucket.");
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用DeleteTableBucket介面刪除Table Bucket。
Namespace
Namespace用於在Table Bucket內對錶進行邏輯分組和隔離管理,類似關聯式資料庫中的Schema概念。每個Table Bucket下最多建立10,000個Namespace。
通過OSS管理主控台操作時,Namespace的管理整合在Table的操作流程中:建立Table時在命名空間下拉式功能表中選擇已有命名空間或單擊建立命名空間輸入名稱建立新的Namespace。通過ossutil、SDK和API可以獨立管理Namespace。
Namespace命名規範:
長度為1~255個字元。
只允許小寫字母、數字和底線(_)。
不能以底線開頭或結尾。
名稱在同一個Table Bucket內唯一。
建立Namespace
ossutil
ossutil tables-api create-namespace \
--table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket \
--namespace mynamespaceSDK
Python
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="create namespace sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--namespace', help='The namespace to create, comma separated if multiple.', required=True)
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
namespaces = args.namespace.split(",")
result = client.create_namespace(oss_tables.models.CreateNamespaceRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=namespaces,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id}')
print(f'successfully created namespace(s) in table bucket: {args.table_bucket_arn}')
if __name__ == "__main__":
main()Go
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.CreateNamespace(context.TODO(), &tables.CreateNamespaceRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: []string{"my_space"},
})
if err != nil {
log.Fatalf("failed to create namespace %v", err)
}
log.Printf("create namespace result:%#v\n", result)
}Java
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
import java.util.Arrays;
import java.util.List;
public class CreateNamespaceSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
String namespace = "mynamespace";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
List<String> namespaces = Arrays.asList("mynamespace");
CreateNamespaceRequest request = CreateNamespaceRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.namespace(namespaces)
.build();
CreateNamespaceResult result = client.createNamespace(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Successfully created namespace(s) in table bucket: %s%n", tableBucketARN);
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用CreateNamespace介面建立Namespace。
查詢Namespace
ossutil
ossutil tables-api get-namespace \
--table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket \
--namespace mynamespaceSDK
Python
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="get namespace sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--namespace', help='The namespace to get.', required=True)
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.get_namespace(oss_tables.models.GetNamespaceRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=args.namespace,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id},'
f' namespace: {result.namespace},'
f' namespace id: {result.namespace_id},'
f' table bucket id: {result.table_bucket_id},'
f' owner account id: {result.owner_account_id},'
f' created at: {result.created_at},'
f' created by: {result.created_by}')
if __name__ == "__main__":
main()Go
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.GetNamespace(context.TODO(), &tables.GetNamespaceRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: oss.Ptr("my_space"),
})
if err != nil {
log.Fatalf("failed to get namespace %v", err)
}
log.Printf("get namespace result:%#v\n", result)
}Java
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class GetNamespaceSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
String namespace = "mynamespace";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
GetNamespaceRequest request = GetNamespaceRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.namespace(namespace)
.build();
GetNamespaceResult result = client.getNamespace(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Successfully retrieved namespace: %s from table bucket: %s%n", result.namespaceId(), tableBucketARN);
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用GetNamespace介面查詢Namespace的詳細資料。
列舉Namespace
ossutil
ossutil tables-api list-namespaces \
--table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucketSDK
Python
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="list namespaces sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--prefix', help='The prefix to filter namespaces.')
parser.add_argument('--max-namespaces', type=int, help='The maximum number of namespaces to return.')
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.list_namespaces(oss_tables.models.ListNamespacesRequest(
table_bucket_arn=args.table_bucket_arn,
prefix=args.prefix,
max_namespaces=args.max_namespaces,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id},'
f' continuation token: {result.continuation_token}')
if result.namespaces:
for i, ns in enumerate(result.namespaces):
print(f'namespace {i + 1}:'
f' namespace: {ns.namespace},'
f' namespace id: {ns.namespace_id},'
f' owner account id: {ns.owner_account_id},'
f' created at: {ns.created_at},'
f' created by: {ns.created_by}')
if __name__ == "__main__":
main()Go
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
p := client.NewListNameSpacesPaginator(&tables.ListNamespacesRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
})
var i int
log.Println("Namespaces:")
for p.HasNext() {
i++
page, err := p.NextPage(context.TODO())
if err != nil {
log.Fatalf("failed to get page %v, %v", i, err)
}
for _, b := range page.Namespaces {
log.Printf("Namespace %v,%v,%v,%v,%v,%v\n", oss.ToString(b.CreatedAt), oss.ToString(b.CreatedBy), oss.ToString(b.OwnerAccountId), oss.ToString(b.TableBucketId), oss.ToString(b.NamespaceId), b.Namespace)
}
}
}Java
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class ListNamespacesSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
int maxNamespaces = 10;
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
ListNamespacesRequest request = ListNamespacesRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.maxNamespaces(maxNamespaces)
.build();
ListNamespacesResult result = client.listNamespaces(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Successfully listed namespaces for table bucket: %s%n", tableBucketARN);
System.out.printf("Continuation token: %s%n", result.continuationToken());
for (int i = 0; i < result.namespaces().size(); i++) {
NamespaceSummary ns = result.namespaces().get(i);
if (ns.namespace() != null && !ns.namespace().isEmpty()) {
System.out.printf("Namespace %%d: %%s (ID: %%s)%%n", i+1, ns.namespace().get(0), ns.namespaceId());
} else {
System.out.printf("Namespace %%d: (no namespace name) (ID: %%s)%%n", i+1, ns.namespaceId());
}
}
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用ListNamespaces介面列舉Namespace。
刪除Namespace
刪除Namespace前需先刪除該Namespace下的所有Table。
ossutil
ossutil tables-api delete-namespace \
--table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket \
--namespace mynamespaceSDK
Python
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="delete namespace sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--namespace', help='The namespace to delete.', required=True)
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.delete_namespace(oss_tables.models.DeleteNamespaceRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=args.namespace,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id}')
print(f'successfully deleted namespace: {args.namespace}')
if __name__ == "__main__":
main()Go
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.DeleteNamespace(context.TODO(), &tables.DeleteNamespaceRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: oss.Ptr("my_space"),
})
if err != nil {
log.Fatalf("failed to delete namespace %v", err)
}
log.Printf("delete namespace result:%#v\n", result)
}Java
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class DeleteNamespaceSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
String namespace = "mynamespace";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
DeleteNamespaceRequest request = DeleteNamespaceRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.namespace(namespace)
.build();
DeleteNamespaceResult result = client.deleteNamespace(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Successfully deleted namespace: %s from table bucket: %s%n", namespace, tableBucketARN);
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用DeleteNamespace介面刪除Namespace。
Table
Table是實際的資料表,儲存格式為Apache Iceberg。
建立Table
如果目標Namespace尚不存在,需要先建立Namespace,每個Table Bucket下最多建立10,000個Namespace,每個Table Bucket下最多建立10,000個Table。
Schema定義
目前控制台不支援Iceberg的巢狀型別(Nested Types),包括struct、list、map,需要通過 API 管理帶巢狀型別的資料表。
建立Table時,通過Schema定義表的欄位結構。每個欄位包含名稱(name)、類型(type)和是否必填(required)等屬性。
支援的欄位類型如下表所示。
類型 | 說明 |
boolean | 布爾值,取值為true或false。 |
int | 32位有符號整數。 |
long | 64位有符號整數。 |
float | 32位IEEE 754浮點數。 |
double | 64位IEEE 754浮點數。 |
decimal(P,S) | 固定精度小數,P為精度(precision),S為標度(scale)。 |
date | 日期類型,不含時間和時區資訊。 |
timestamp | 時間戳記類型,微秒精度,不含時區資訊。 |
timestamptz | 時間戳記類型,微秒精度,含時區資訊。 |
string | UTF-8編碼的字串。 |
binary | 可變長度位元組數組。 |
uuid | 通用唯一識別碼(UUID)。 |
Schema定義樣本:
{
"schema": {
"fields": [
{"name": "id", "type": "long", "required": true},
{"name": "name", "type": "string", "required": true},
{"name": "created_at", "type": "timestamptz", "required": false}
]
}
}分區規格
分區規格(Partition Spec)定義了Table的資料分區方式。合理的分區規格可以顯著提升查詢效能,減少掃描的資料量。
支援以下分區轉換(Transform)類型:
轉換類型 | 說明 | 適用類型 |
identity | 使用欄位原始值進行分區,不做轉換。 | 所有類型。 |
year | 從日期或時間戳記中提取年份進行分區。 | date、timestamp、timestamptz。 |
month | 從日期或時間戳記中提取月份進行分區。 | date、timestamp、timestamptz。 |
day | 從日期或時間戳記中提取日期進行分區。 | date、timestamp、timestamptz。 |
hour | 從時間戳記中提取小時進行分區。 | timestamp、timestamptz。 |
bucket[N] | 對欄位值計算雜湊後分成N個桶。適用於高基數欄位的均勻分布。 | int、long、string、decimal、date、timestamp、timestamptz、uuid。 |
truncate[N] | 按寬度N截斷欄位值進行分區。對於int/long按數值寬度截斷,對於string按字元長度截斷。 | int、long、string、decimal。 |
控制台
通過OSS控制台建立Table的操作流程如下:
登入OSS控制台,在左側導覽列選擇Table Bucket 列表。
單擊目標Table Bucket名稱,進入詳情頁。
在Table 列表 Tab下,單擊建立 Table。
在建立Table彈窗中配置以下參數:
命名空間:從下拉式功能表選擇已有命名空間,或輸入新的命名空間名稱直接建立。Namespace名稱長度為1~255個字元,只允許小寫字母、數字和底線(_),不能以底線開頭或結尾,名稱在同一個Table Bucket內唯一。
Table 格式:選擇Iceberg。
資料表名稱:輸入表名,長度1~255位,僅允許小寫字母、數字和底線(_),不能以底線開頭或結尾,在同一Namespace下必須唯一。
服務端加密方式:選擇不加密或AES256。
表格式版本:選擇Iceberg表格式版本(如2)。
欄位資訊:添加欄位,每個欄位需配置ID、列名稱、是否必填、資料類型。
分區資訊:添加分區,每個分區需配置sourceId、分區欄位名稱、transform。
排序資訊:添加排序,每個排序需配置sourceId、transform、direction、nullOrder。
單擊確定。
ossutil
使用ossutil建立Table前,如果目標Namespace尚不存在,需要先建立Namespace。建立Table的命令樣本如下。其中,properties中的format-version用於指定Iceberg表格式版本,可選值為2或3;--encryption-configuration用於佈建服務端加密。
# 步驟2:建立Table(指定表格式版本和服務端加密方式)
ossutil tables-api create-table \
--region cn-hangzhou \
--endpoint https://cn-hangzhou.oss-tables.aliyuncs.com \
--table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket \
--namespace mynamespace \
--name mytable \
--format ICEBERG \
--metadata '{"iceberg":{"schema":{"fields":[{"name":"id","type":"long","required":true},{"name":"data","type":"string"}]},"properties":{"format-version":"2"}}}' \
--encryption-configuration '{"sseAlgorithm":"AES256"}'SDK
Python
使用Python SDK建立Table的程式碼範例如下。如果目標Namespace尚不存在,需要先建立Namespace:
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="create table sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--namespace', help='The namespace of the table.', required=True)
parser.add_argument('--name', help='The name of the table.', required=True)
parser.add_argument('--format', help='The format of the table.', required=True)
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
# Create schema fields
schema = oss_tables.models.IcebergSchema(fields=[
oss_tables.models.SchemaField(name="id", type="long", required=True),
oss_tables.models.SchemaField(name="name", type="string", required=False),
oss_tables.models.SchemaField(name="ts", type="timestamptz", required=False),
])
# Create partition spec
partition_spec = oss_tables.models.IcebergPartitionSpec(
spec_id=0,
fields=[
oss_tables.models.IcebergPartitionField(
source_id=2,
field_id=1001,
name="region",
transform="identity",
),
],
)
# Create iceberg metadata(通過 properties 指定 format-version,支援 "2" 或 "3")
iceberg_metadata = oss_tables.models.IcebergMetadata(
schema=schema,
partition_spec=partition_spec,
properties={"format-version": "2"},
)
# Set metadata
metadata = oss_tables.models.TableMetadata(iceberg=iceberg_metadata)
# Add encryption configuration(sse_algorithm 支援 "AES256" 和 "KMS",KMS 需配合 kms_key_arn)
encryption_configuration = oss_tables.models.EncryptionConfiguration(
sse_algorithm="AES256",
)
result = client.create_table(oss_tables.models.CreateTableRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=args.namespace,
name=args.name,
format=args.format,
metadata=metadata,
encryption_configuration=encryption_configuration,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id},'
f' table arn: {result.table_arn},'
f' version token: {result.version_token}')
if __name__ == "__main__":
main()Go
使用Go SDK建立Table的程式碼範例如下。如果目標Namespace尚不存在,需要先建立Namespace:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
namespace string
name string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
flag.StringVar(&namespace, "namespace", "", "The name of the namespace.")
flag.StringVar(&name, "name", "", "The name of the table.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(namespace) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, namespace name required")
}
if len(name) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table name required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
// 通過 map 構建完整 iceberg 中繼資料,properties 中的 format-version 用於指定表格式版本("2" 或 "3")
icebergMap := map[string]any{
"schema": map[string]any{
"fields": []map[string]any{
{"name": "id", "type": "long", "required": true},
{"name": "data", "type": "string"},
},
},
"properties": map[string]any{
"format-version": "2",
},
}
result, err := client.CreateTable(context.TODO(), &tables.CreateTableRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: oss.Ptr(namespace),
Name: oss.Ptr(name),
Format: oss.Ptr("ICEBERG"),
Metadata: &tables.TableMetadata{
Iceberg: &tables.IcebergMetadata{Schema: icebergMap},
},
// 服務端加密:SseAlgorithm
EncryptionConfiguration: &tables.EncryptionConfiguration{
SseAlgorithm: oss.Ptr("AES256"),
},
})
if err != nil {
log.Fatalf("failed to create table %v", err)
}
log.Printf("create table result:%#v\n", result)
}Java
使用Java SDK建立Table的程式碼範例如下。如果目標Namespace尚不存在,需要先建立Namespace。
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
import java.util.ArrayList;
import java.util.List;
public class CreateTableSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
String namespace = "mynamespace";
String name = "mytable";
String format = "iceberg";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
// Create schema fields
List<SchemaField> fields = new ArrayList<>();
fields.add(SchemaField.newBuilder()
.name("id")
.type("long")
.required(true)
.build());
fields.add(SchemaField.newBuilder()
.name("name")
.type("string")
.required(false)
.build());
fields.add(SchemaField.newBuilder()
.name("ts")
.type("timestamptz")
.required(false)
.build());
// Create schema
IcebergSchema icebergSchema = IcebergSchema.newBuilder()
.fields(fields)
.build();
// Create partition spec
IcebergPartitionField partitionField = IcebergPartitionField.newBuilder()
.sourceId(2)
.transform("identity")
.name("region")
.fieldId(1001)
.build();
List<IcebergPartitionField> partitionFields = new ArrayList<>();
partitionFields.add(partitionField);
IcebergPartitionSpec partitionSpec = IcebergPartitionSpec.newBuilder()
.specId(0)
.fields(partitionFields)
.build();
// Create iceberg metadata
IcebergMetadata icebergMetadata = IcebergMetadata.newBuilder()
.schema(icebergSchema)
.partitionSpec(partitionSpec)
.build();
// Set metadata
TableMetadata metadata = TableMetadata.newBuilder()
.iceberg(icebergMetadata)
.build();
// Add encryption configuration
EncryptionConfiguration encryptionConfig = EncryptionConfiguration.newBuilder()
.sseAlgorithm("AES256")
.build();
CreateTableRequest request = CreateTableRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.namespace(namespace)
.name(name)
.format(format)
.metadata(metadata)
.encryptionConfiguration(encryptionConfig)
.build();
CreateTableResult result = client.createTable(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Created table with ARN: %s%n", result.tableARN());
System.out.printf("Version token: %s%n", result.versionToken());
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用CreateTable介面建立Table。
查詢Table
擷取Table的詳細資料,包括名稱、所屬Namespace、tableARN、格式、versionToken、metadataLocation、warehouseLocation等。
控制台
通過OSS控制台查詢Table資訊的操作流程如下:
登入OSS控制台,在左側導覽列選擇 Table Bucket 列表。
單擊目標Table Bucket名稱,進入詳情頁。
在Table 列表 Tab下,單擊目標Table名稱,查看Table詳情。
ossutil
使用ossutil查詢Table資訊的命令樣本如下:
ossutil tables-api get-table \
--table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket \
--namespace mynamespace \
--name mytableSDK
Python
使用Python SDK查詢Table資訊的程式碼範例如下:
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="get table sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.')
parser.add_argument('--namespace', help='The namespace of the table.')
parser.add_argument('--name', help='The name of the table.')
parser.add_argument('--table-arn', help='The ARN of the table.')
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.get_table(oss_tables.models.GetTableRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=args.namespace,
name=args.name,
table_arn=args.table_arn,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id},'
f' name: {result.name},'
f' table arn: {result.table_arn},'
f' format: {result.format},'
f' created by: {result.created_by},'
f' created at: {result.created_at}')
if __name__ == "__main__":
main()Go
使用Go SDK查詢Table資訊的程式碼範例如下:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
namespace string
name string
tableArn string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
flag.StringVar(&namespace, "namespace", "", "The name of the namespace.")
flag.StringVar(&name, "name", "", "The name of the table.")
flag.StringVar(&tableArn, "table-arn", "", "The arn of the table.")
}
func main() {
flag.Parse()
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
// function one
result, err := client.GetTable(context.TODO(), &tables.GetTableRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: oss.Ptr(namespace),
Name: oss.Ptr(name),
})
// function two
//result, err := client.GetTable(context.TODO(), &tables.GetTableRequest{
// TableARN: oss.Ptr(tableArn),
//})
if err != nil {
log.Fatalf("failed to get table %v", err)
}
log.Printf("get table result:%#v\n", result)
}Java
使用Java SDK查詢Table資訊的程式碼範例如下:
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class GetTableSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
String namespace = "mynamespace";
String tableName = "mytable";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
// 方式一:通過 Table Bucket ARN + Namespace + Name 查詢
GetTableRequest request = GetTableRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.namespace(namespace)
.name(tableName)
.build();
// 方式二:通過 Table ARN 查詢
// GetTableRequest request = GetTableRequest.newBuilder()
// .tableARN("acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket/table/table-id")
// .build();
GetTableResult result = client.getTable(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Table name: %s%n", result.name());
System.out.printf("Table ARN: %s%n", result.tableARN());
System.out.printf("Format: %s%n", result.format());
System.out.printf("Created by: %s%n", result.createdBy());
System.out.printf("Created at: %s%n", result.createdAt());
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用GetTable介面查詢Table的詳細資料。
列舉Table
列舉指定Namespace下的所有Table,支援分頁查詢。
控制台
通過OSS控制台列舉Table的操作流程如下:
登入OSS控制台,在左側導覽列選擇Table Bucket 列表。
單擊目標Table Bucket名稱,進入詳情頁。
在Table 列表 Tab下查看當前Table Bucket下所有Namespace中的Table。
ossutil
使用ossutil列舉Table的命令樣本如下:
ossutil tables-api list-tables \
--table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket \
--namespace mynamespaceSDK
Python
使用Python SDK列舉Table的程式碼範例如下:
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="list tables sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--namespace', help='The namespace of the tables.', required=True)
parser.add_argument('--prefix', help='The prefix to filter tables.')
parser.add_argument('--max-tables', type=int, help='The maximum number of tables to return.')
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.list_tables(oss_tables.models.ListTablesRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=args.namespace,
prefix=args.prefix,
max_tables=args.max_tables,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id},'
f' continuation token: {result.continuation_token}')
if result.table_summaries:
for i, table in enumerate(result.table_summaries):
print(f'table {i + 1}:'
f' name: {table.name},'
f' namespace: {table.namespace},'
f' type: {table.type},'
f' table arn: {table.table_arn},'
f' created at: {table.created_at},'
f' modified at: {table.modified_at}')
if __name__ == "__main__":
main()Go
使用Go SDK列舉Table的程式碼範例如下:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
namespace string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
flag.StringVar(&namespace, "namespace", "", "The name of the namespace.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(namespace) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, namespace name required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
p := client.NewListTablesPaginator(&tables.ListTablesRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: oss.Ptr(namespace),
})
var i int
log.Println("Tables:")
for p.HasNext() {
i++
page, err := p.NextPage(context.TODO())
if err != nil {
log.Fatalf("failed to get page %v, %v", i, err)
}
for _, b := range page.Tables {
log.Printf("Table %v,%v,%v,%v,%v,%v\n", oss.ToString(b.CreatedAt), oss.ToString(b.TableARN), oss.ToString(b.ModifiedAt), oss.ToString(b.Type), oss.ToString(b.Name), b.Namespace)
}
}
}Java
使用Java SDK列舉Table的程式碼範例如下:
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class ListTablesSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
String namespace = "mynamespace";
int maxTables = 10;
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
ListTablesRequest request = ListTablesRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.namespace(namespace)
.maxTables(maxTables)
.build();
ListTablesResult result = client.listTables(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Continuation token: %s%n", result.continuationToken());
if (result.tables() != null) {
System.out.printf("Number of tables: %d%n", result.tables().size());
for (int i = 0; i < result.tables().size(); i++) {
TableSummary table = result.tables().get(i);
System.out.printf("Table %d:%n", i + 1);
System.out.printf(" Name: %s%n", table.name());
System.out.printf(" Namespace: %s%n", table.namespace());
System.out.printf(" ARN: %s%n", table.tableARN());
System.out.printf(" Type: %s%n", table.type());
System.out.printf(" Created at: %s%n", table.createdAt());
System.out.printf(" Modified at: %s%n", table.modifiedAt());
}
} else {
System.out.println("No tables found.");
}
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用ListTables介面列舉Table。
重新命名與移動Table
通過RenameTable介面執行以下操作:
重新命名Table:修改Table名稱,保持所屬Namespace不變。
移動Table:將Table移動到同一Table Bucket下的其他Namespace。
該操作支援versionToken實現樂觀鎖控制,防止並行作業導致的資料衝突。
ossutil
使用ossutil重新命名Table的命令樣本如下:
ossutil tables-api rename-table \
--table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket \
--namespace mynamespace \
--name mytable \
--new-namespace mynamespace \
--new-name mytable_renamedSDK
Python
使用Python SDK重新命名Table的程式碼範例如下:
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="rename table sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--namespace', help='The namespace of the table.', required=True)
parser.add_argument('--name', help='The current name of the table.', required=True)
parser.add_argument('--new-namespace-name', help='The new namespace name for the table.')
parser.add_argument('--new-name', help='The new name for the table.')
parser.add_argument('--version-token', help='The version token for the table.')
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.rename_table(oss_tables.models.RenameTableRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=args.namespace,
name=args.name,
new_namespace_name=args.new_namespace_name,
new_name=args.new_name,
version_token=args.version_token,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id}')
print(f'successfully renamed table: {args.namespace}/{args.name}')
if __name__ == "__main__":
main()Go
使用Go SDK重新命名Table的程式碼範例如下:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
namespace string
name string
newName string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
flag.StringVar(&namespace, "namespace", "", "The name of the namespace.")
flag.StringVar(&name, "name", "", "The name of the table.")
flag.StringVar(&newName, "new-name", "", "The new name of the table.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(namespace) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, namespace name required")
}
if len(name) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table name required")
}
if len(newName) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table new name required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.RenameTable(context.TODO(), &tables.RenameTableRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: oss.Ptr(namespace),
Name: oss.Ptr(name),
NewName: oss.Ptr(newName),
})
if err != nil {
log.Fatalf("failed to rename table %v", err)
}
log.Printf("rename table result:%#v\n", result)
}Java
使用Java SDK重新命名Table的程式碼範例如下:
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class RenameTableSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
String namespace = "mynamespace";
String name = "mytable";
String newName = "mytable-renamed";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
RenameTableRequest request = RenameTableRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.namespace(namespace)
.name(name)
.newName(newName)
.build();
RenameTableResult result = client.renameTable(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Successfully renamed table from %s/%s to %s/%s%n",
namespace, name, namespace, newName);
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用RenameTable介面重新命名Table。
刪除Table
刪除指定的Table,刪除後Table中的資料將被清除,且該操作無法復原。請在執行前確認已做好資料備份。
控制台
通過OSS控制台刪除Table的操作流程如下:
登入OSS控制台,在左側導覽列選擇Table Bucket 列表。
單擊目標Table Bucket名稱,進入詳情頁。
在Table 列表 Tab下,找到目標Table,單擊操作列的刪除。
在確認彈窗中單擊確定。
ossutil
使用ossutil刪除Table的命令樣本如下:
ossutil tables-api delete-table \
--table-bucket-arn acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket \
--namespace mynamespace \
--name mytableSDK
Python
使用Python SDK刪除Table的程式碼範例如下:
# 步驟1:刪除Table
# 完整樣本:delete_table.py
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="delete table sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--namespace', help='The namespace of the table.', required=True)
parser.add_argument('--name', help='The name of the table.', required=True)
parser.add_argument('--version-token', help='The version token for optimistic locking.')
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.delete_table(oss_tables.models.DeleteTableRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=args.namespace,
name=args.name,
version_token=args.version_token,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id}')
print(f'successfully deleted table: {args.namespace}/{args.name}')
if __name__ == "__main__":
main()
# China_an_fen_ge_xian China_an_fen_ge_xian China_an_fen_ge_xian
# 步驟2:刪除Namespace(如果需要刪除Namespace,需先刪除該Namespace下的所有Table)
# 完整樣本:delete_namespace.py
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="delete namespace sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--namespace', help='The namespace to delete.', required=True)
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.delete_namespace(oss_tables.models.DeleteNamespaceRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=args.namespace,
))
print(f'status code: {result.status_code},'
f' request id: {result.request_id}')
print(f'successfully deleted namespace: {args.namespace}')
if __name__ == "__main__":
main()Go
使用Go SDK刪除Table的程式碼範例如下:
// 步驟1:刪除Table
// 完整樣本:delete_table.go
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
namespace string
name string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
flag.StringVar(&namespace, "namespace", "", "The name of the namespace.")
flag.StringVar(&name, "name", "", "The name of the table.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(namespace) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, namespace name required")
}
if len(name) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table name required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.DeleteTable(context.TODO(), &tables.DeleteTableRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: oss.Ptr(namespace),
Name: oss.Ptr(name),
})
if err != nil {
log.Fatalf("failed to delete table %v", err)
}
log.Printf("delete table result:%#v\n", result)
}
// ----------------------------------------------------------------
// 步驟2:刪除Namespace(如果需要刪除Namespace,需先刪除該Namespace下的所有Table)
// 完整樣本:delete_namespace.go
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
}
func main() {
flag.Parse()
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.DeleteNamespace(context.TODO(), &tables.DeleteNamespaceRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: oss.Ptr("my_space"),
})
if err != nil {
log.Fatalf("failed to delete namespace %v", err)
}
log.Printf("delete namespace result:%#v\n", result)
}Java
使用Java SDK刪除Table的程式碼範例如下:
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class DeleteTableSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
String namespace = "mynamespace";
String tableName = "mytable";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
DeleteTableRequest request = DeleteTableRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.namespace(namespace)
.name(tableName)
.build();
DeleteTableResult result = client.deleteTable(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Successfully deleted table: %s/%s from bucket: %s%n",
namespace, tableName, tableBucketARN);
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用DeleteTable介面刪除Table。
中繼資料管理
Table的Iceberg中繼資料檔案記錄了表的Schema、快照歷史、分區規格等核心資訊。中繼資料管理涉及以下兩個關鍵屬性:
metadataLocation:中繼資料檔案的儲存路徑。每次Iceberg commit操作後,系統會產生新的中繼資料檔案。路徑格式為
oss://{tableId}--table-oss/metadata/00000-{uuid}.metadata.json。versionToken:版本標識。每次中繼資料更新後versionToken會變化。在執行UpdateTableMetadataLocation或RenameTable操作時,需要攜帶當前的versionToken來實現樂觀鎖控制。
ossutil
查詢中繼資料位置:
ossutil tables-api get-table-metadata-location --table-bucket-arn {ARN} --namespace {ns} --name {table}更新中繼資料位置:
ossutil tables-api update-table-metadata-location --table-bucket-arn {ARN} --namespace {ns} --name {table} --metadata-location {location} --version-token {token}SDK
Python
查詢中繼資料位置:
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="get table metadata location sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--namespace', help='The namespace of the table.', required=True)
parser.add_argument('--name', help='The name of the table.', required=True)
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.get_table_metadata_location(
oss_tables.models.GetTableMetadataLocationRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=args.namespace,
name=args.name,
)
)
print(f'status code: {result.status_code},'
f' request id: {result.request_id},'
f' version token: {result.version_token},'
f' metadata location: {result.metadata_location},'
f' warehouse location: {result.warehouse_location}')
if __name__ == "__main__":
main()更新中繼資料位置:
import argparse
import alibabacloud_oss_v2 as oss
import alibabacloud_oss_v2.tables as oss_tables
parser = argparse.ArgumentParser(description="update table metadata location sample")
parser.add_argument('--region', help='The region in which the table bucket is located.', required=True)
parser.add_argument('--endpoint', help='The domain names that other services can use to access OSS Tables.')
parser.add_argument('--table-bucket-arn', help='The ARN of the table bucket.', required=True)
parser.add_argument('--namespace', help='The namespace of the table.', required=True)
parser.add_argument('--name', help='The name of the table.', required=True)
parser.add_argument('--version-token', help='The version token.', required=True)
parser.add_argument('--metadata-location', help='The new metadata location.', required=True)
def main():
args = parser.parse_args()
credentials_provider = oss.credentials.EnvironmentVariableCredentialsProvider()
cfg = oss.config.load_default()
cfg.credentials_provider = credentials_provider
cfg.region = args.region
if args.endpoint is not None:
cfg.endpoint = args.endpoint
client = oss_tables.Client(cfg)
result = client.update_table_metadata_location(
oss_tables.models.UpdateTableMetadataLocationRequest(
table_bucket_arn=args.table_bucket_arn,
namespace=args.namespace,
name=args.name,
version_token=args.version_token,
metadata_location=args.metadata_location,
)
)
print(f'status code: {result.status_code},'
f' request id: {result.request_id},'
f' name: {result.name},'
f' version token: {result.version_token},'
f' metadata location: {result.metadata_location},'
f' table arn: {result.table_arn}')
if __name__ == "__main__":
main()Go
查詢中繼資料位置:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
namespace string
name string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
flag.StringVar(&namespace, "namespace", "", "The name of the namespace.")
flag.StringVar(&name, "name", "", "The name of the table.")
}
func main() {
flag.Parse()
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(namespace) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, namespace name required")
}
if len(name) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table name required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.GetTableMetadataLocation(context.TODO(), &tables.GetTableMetadataLocationRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: oss.Ptr(namespace),
Name: oss.Ptr(name),
})
if err != nil {
log.Fatalf("failed to get table metadata location %v", err)
}
log.Printf("get table metadata location result:%#v\n", result)
}更新中繼資料位置:
package main
import (
"context"
"flag"
"log"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/tables"
)
var (
region string
tableBucketArn string
namespace string
name string
metadataLocation string
versionToken string
)
func init() {
flag.StringVar(®ion, "region", "", "The region in which the bucket is located.")
flag.StringVar(&tableBucketArn, "table-bucket-arn", "", "The arn of the table bucket.")
flag.StringVar(&namespace, "namespace", "", "The name of the namespace.")
flag.StringVar(&name, "name", "", "The name of the table.")
flag.StringVar(&metadataLocation, "metadata-location", "", "The metadata location of the table.")
flag.StringVar(&versionToken, "version-token", "", "The version token of the table.")
}
func main() {
flag.Parse()
if len(region) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, region required")
}
if len(tableBucketArn) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table bucket arn required")
}
if len(namespace) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, namespace name required")
}
if len(name) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table name required")
}
if len(metadataLocation) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table metadata location required")
}
if len(versionToken) == 0 {
flag.PrintDefaults()
log.Fatalf("invalid parameters, table version token required")
}
cfg := oss.LoadDefaultConfig().
WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
WithRegion(region)
client := tables.NewTablesClient(cfg)
result, err := client.UpdateTableMetadataLocation(context.TODO(), &tables.UpdateTableMetadataLocationRequest{
TableBucketARN: oss.Ptr(tableBucketArn),
Namespace: oss.Ptr(namespace),
Name: oss.Ptr(name),
MetadataLocation: oss.Ptr(metadataLocation),
VersionToken: oss.Ptr(versionToken),
})
if err != nil {
log.Fatalf("failed to update table metadata location %v", err)
}
log.Printf("update table metadata location result:%#v\n", result)
}Java
查詢中繼資料位置:
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class GetTableMetadataLocationSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
String namespace = "mynamespace";
String name = "mytable";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
GetTableMetadataLocationRequest request = GetTableMetadataLocationRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.namespace(namespace)
.name(name)
.build();
GetTableMetadataLocationResult result = client.getTableMetadataLocation(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Metadata location: %s%n", result.metadataLocation());
System.out.printf("Version token: %s%n", result.versionToken());
System.out.printf("Warehouse location: %s%n", result.warehouseLocation());
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}更新中繼資料位置:
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.tables.OSSTablesClient;
import com.aliyun.sdk.service.oss2.tables.models.*;
public class UpdateTableMetadataLocationSample {
public static void main(String[] args) throws Exception {
String region = "cn-hangzhou";
String tableBucketARN = "acs:osstables:cn-hangzhou:1234567890:bucket/mytablebucket";
String namespace = "mynamespace";
String name = "mytable";
String versionToken = "your-version-token";
String metadataLocation = "oss://warehouse/metadata/v1.metadata.json";
try (OSSTablesClient client = OSSTablesClient.newBuilder()
.credentialsProvider(new EnvironmentVariableCredentialsProvider())
.region(region)
.build()) {
UpdateTableMetadataLocationRequest request = UpdateTableMetadataLocationRequest.newBuilder()
.tableBucketARN(tableBucketARN)
.namespace(namespace)
.name(name)
.versionToken(versionToken)
.metadataLocation(metadataLocation)
.build();
UpdateTableMetadataLocationResult result = client.updateTableMetadataLocation(request);
System.out.printf("Status code:%d, request id:%s%n",
result.statusCode(), result.requestId());
System.out.printf("Successfully updated table metadata location for table: %s%n", name);
System.out.printf("New metadata location: %s%n", result.metadataLocation());
System.out.printf("New version token: %s%n", result.versionToken());
} catch (Exception e) {
System.out.println("Error: " + e.getMessage());
}
}
}API
使用GetTableMetadataLocation介面查詢中繼資料檔案位置。
使用UpdateTableMetadataLocation介面更新中繼資料位置。
配額與限制
限制項 | 說明 |
單個Table Bucket內的Table數量 | 最多10,000個,如果您需要更多的表數量,請聯絡支援人員。 |
單個Table Bucket內的Namespace數量 | 最多10,000個。 |
Table Bucket數量 | 每個地區每個帳號最多建立10個Table Bucket。 |
Table Bucket名稱長度 | 3~32個字元。 |
Table名稱長度 | 1~255個字元。 |
Table Bucket名稱字元 | 只允許小寫字母、數字和連字號(-),不能以連字號開頭或結尾。 |
Table Bucket名稱首碼 | 不能以oss、acs、aliyun、alibaba、aws為首碼。 |
Table名稱字元範圍 | 小寫字母、數字和底線(_),不能以底線開頭或結尾。 |
Namespace名稱長度 | 1~255個字元。 |
Namespace名稱字元範圍 | 小寫字母、數字和底線(_),不能以底線開頭或結尾。 |
密碼編譯演算法 | 目前僅支援AES256。 |
表格式 | 僅支援ICEBERG。 |
計費說明
OSS Tables目前處於邀測階段,暫時不收取費用,請關注後續官網商業化通知。