VECTOR data type - MaxCompute - Alibaba Cloud Documentation Center

The vector data type allows you to efficiently encode and process vectors. It supports semantic vector search and retrieval applications, such as RAG-based applications, and common vector operations used in vector processing workloads.

Data type syntax

Use the following syntax to specify the VECTOR data type:

VECTOR( FLOAT <type>, <dimension> )

-- Define a 768-dimension floating-point vector
VECTOR(FLOAT, 768)

type: The data type of the vector elements. Currently, only FLOAT is supported.
dimension: The dimension of the vector. It must be a positive multiple of 32, in the range [32, 4096]. For example, valid values include 32, 64, 96, and 128.

Limitations

A VECTOR column cannot be used as a PRIMARY KEY or CLUSTER KEY.
A VECTOR column cannot be used as a key in ORDER BY, GROUP BY, or JOIN clauses.
The VECTOR data type does not support arithmetic operations (such as +, -, *, /, %), comparison operations (such as =, >, <), or aggregate functions (such as SUM, MIN, MAX, AVG, and MEDIAN).
Only the MaxCompute SQL engine supports the VECTOR data type.
You can work with the VECTOR data type using the local client (odpscmd), Python SDK, and DataWorks.

SQL syntax

SET

To use the VECTOR data type, set the following flags:

SET odps.sql.type.system.odps2=true;
SET odps.sql.type.vector.enable=true;

DDL

You can store vector data directly in MaxCompute tables:

-- Create a table with a VECTOR column
CREATE OR REPLACE TABLE products (
  description_embedding VECTOR(FLOAT, 32)
);

DML

There are two main ways to construct VECTOR data.
- You can cast the ARRAY<FLOAT> type to the VECTOR type. When you convert the data, the system automatically checks if the array dimension matches the target vector dimension.
- You can use AI_EMBEDDING to directly generate VECTOR type data.

Example

-- You can explicitly or implicitly cast an ARRAY. The array's dimension must match the target VECTOR's dimension.
INSERT INTO products select CAST(array(1.1, 2.2, 3.3, 4.4, 5.5, 6.6, 7.7, 8.8, 9.9, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0, 30.0, 31.0, 32.0) as VECTOR(FLOAT, 32));

SELECT * FROM products;
-- Result
+-----------------------+
| description_embedding |
+-----------------------+
| [1.1, 2.2, 3.3, 4.4, 5.5, 6.6, 7.7, 8.8, 9.9, 10.0, 11.0, 12.0, 13.0, 14.0, 15.0, 16.0, 17.0, 18.0, 19.0, 20.0, 21.0, 22.0, 23.0, 24.0, 25.0, 26.0, 27.0, 28.0, 29.0, 30.0, 31.0, 32.0] |
+-----------------------+

VECTOR functions

The following vector functions are currently supported:

VECTOR_SEARCH, L2_DISTANCE, INNER_PRODUCT_DISTANCE, and COSINE_DISTANCE.

VECTOR INDEX

To further improve the efficiency of large-scale vector retrieval, MaxCompute supports creating a vector index on the VECTOR field to enable efficient approximate nearest neighbor (ANN) queries. For more information, see VECTOR INDEX.