All Products
Search
Document Center

Data types

Last Updated: Nov 22, 2021

This topic describes the mappings of data and value types between Spark, Scala, as well as the search indexes and tables of Tablestore. When you use these data and value types, you must follow the mapping rules for Spark, Scala, and Tablestore.

Basic data types

The following table describes the supported basic data types.

Data type in Spark

Value type in Scala

Data type in search indexes

Data type in tables

ByteType

Byte

Long

Integer

ShortType

Short

Long

Integer

IntegerType

Int

Long

Integer

LongType

Long

Long

Integer

FloatType

Float

Double

Double

DoubleType

Double

Double

Double

StringType

String

Keyword/Text

String

BinaryType

Array[Byte]

Binary

Binary

BooleanType

Boolean

Boolean

Boolean

String JSON (geographical coordinates)

String (JSON)

Geopoint

String (JSON)

Geographical location data types

Search indexes support geo query. When a geo query is pushed down to the compute layer, Spark can query and analyze data based on geographical locations in addition to basic types of data.

Geo query provides the following query types: geo-distance query, geo-bounding box query, and geo-polygon query. Geo query can be used to query information about the locations of IoT devices, locations of food delivery orders, locations of employees when they clock in or out, and locations of express delivery orders. You can use the following methods to perform geo query:

  • Use the search indexes of Tablestore. For more information, see Geo-distance query.

  • Use Spark SQL.

    • Use geo-distance query by specifying the central point and the radius for the query.

      select * from table where  val_geo = '{"centerPoint":"3,0", "distanceInMeter": 100000}' and name like 'ali%'
    • Use geo-bounding box query.

      select * from table where geo = '{"topLeft":"8,0", "bottomRight": "0,10"}' and id in { 123 , 321 }
    • Use geo-polygon query.

      select * from table where geo = '{"points":["5,0", "5,1", "6,1", "6,10"]}'

The following table describes the geographical location data types supported by Spark, Scala, search indexes, and tables.

Data type in Spark

Value type in Scala

Data type in search indexes

Data type in tables

String JSON (coordinates of the central point and the radius of a circular geographical area)

String (JSON)

Geopoint

STRING (JSON)

STRING JSON (coordinates of the vertices in a rectangular geographical area)

String (JSON)

Geopoint

STRING (JSON)

String JSON (coordinates of the vertices in a polygonal geographical area)

String (JSON)

Geopoint

STRING (JSON)