The GET_DATA_FROM_OSS function reads all or part of an object's content and returns it as a binary value.
Use cases
MaxCompute provides the Object Table feature. This feature allows the compute engine to access unstructured data and its metadata stored in OSS. For more information, see OBJECT TABLE definition.
An Object Table stores the metadata of OSS objects in a specific path. You can use the GET_DATA_FROM_OSS function to dynamically load the binary content of a specified OSS object.
Syntax
BINARY GET_DATA_FROM_OSS (
STRING <full_object_table_name>,
STRING <key>
[, BIGINT <offset>]
[, BIGINT <length>]
[, STRING <object_not_found_policy>]
)Parameters
Parameter | Required | Data type | Description | Default value |
full_object_table_name | Yes | STRING | The full path to the OBJECT TABLE in the three-tier model, including the Project and Schema names, such as If you use RoleARN for authentication when you create the table, this parameter helps automatically generate a Security Token Service (STS) token to access OSS. | None |
key | Yes | STRING | The name of the accessed object in the Object Table. For more information, see the description of the `key` parameter in the response described in View OBJECT TABLE properties. | None |
offset | No | BIGINT | The starting position from which to read the object content. The value must be greater than or equal to 0. | 0. This means the read operation starts from the beginning of the object. |
length | No | BIGINT | The number of bytes to read. | -1. This means the length is not limited. |
object_not_found_policy | No | STRING | Specifies how MaxCompute returns the result of the function invocation if an object key exists in the cached data but the object no longer exists in OSS. Valid values:
| The default value is OUTPUT_NULL. |
Return value
Returns a value of the BINARY type.
Examples
In the following examples, replace the project_name parameter with your actual MaxCompute project name.
Prepare data
Log on to the OSS console, and upload the test data signedget.txt to the
object-table-test/object_table_folderfolder. For more information, see Upload files.You can use the local client (odpscmd) or create a MaxCompute SQL node in DataWorks to create an Object Table and refresh the metadata cache. For more information, see Create an OBJECT TABLE. In this example, the OBJECT TABLE is named
ot_demo_day.-- The Object Table feature in MaxCompute projects supports schemas. Enable the three-layer model. SET odps.namespace.schema=true; -- Select the MaxCompute project. USE <project_name>; -- Select the schema. USE SCHEMA <schema_name>; -- The Object Table feature in MaxCompute projects supports the 2.0 data type system. SET odps.sql.type.system.odps2=true; -- Create an Object Table. CREATE OBJECT TABLE ot_demo_day WITH serdeproperties ( 'odps.properties.rolearn'='acs:ram::xxxxxx:role/aliyunodpsdefaultrole') LOCATION 'oss://oss-cn-hangzhou-internal.aliyuncs.com/object-table-test/object_table_folder'; -- Refresh the table cache. ALTER TABLE ot_demo_day REFRESH METADATA;
Example 1
Read all the content of the Object Table and return it as a binary value. The full path of the Object Table is <project_name>.default.ot_demo_day. The following code shows different parameter combinations.
-- The complete format.
SELECT GET_DATA_FROM_OSS('<project_name>.default.ot_demo_day', key, 0, -1, 'OUTPUT_NULL') FROM ot_demo_day;
-- The following statements are equivalent to get_data_from_oss('<project_name>.default.ot_demo_day', key, 0, -1, 'OUTPUT_NULL').
SELECT GET_DATA_FROM_OSS('<project_name>.default.ot_demo_day', key) FROM ot_demo_day;
SELECT GET_DATA_FROM_OSS('<project_name>.default.ot_demo_day', key, 0) FROM ot_demo_day;
SELECT GET_DATA_FROM_OSS('<project_name>.default.ot_demo_day', key, 0, -1) FROM ot_demo_day;
SELECT GET_DATA_FROM_OSS('<project_name>.default.ot_demo_day', key, 'OUTPUT_NULL') FROM ot_demo_day;
SELECT GET_DATA_FROM_OSS('<project_name>.default.ot_demo_day', key, 0, 'OUTPUT_NULL') FROM ot_demo_day;The following result is returned:
+------+
| _c0 |
+------+
| test=20maxcompute=20download=20files=20by=20url=20 |
+------+Example 2
To output the result of the GET_DATA_FROM_OSS function as the STRING type, nest it within the `STRING` function.
SELECT STRING(
GET_DATA_FROM_OSS('<project_name>.default.ot_demo_day', key, 0, -1, 'OUTPUT_NULL')
)
FROM ot_demo_day;The following result is returned:
+-----+
| _c0 |
+-----+
| test maxcompute download files by url |
+-----+Related functions
The GET_DATA_FROM_OSS function is used to process unstructured data. For more information about functions that process unstructured data, see Functions for unstructured data processing.