By Wang Tantan (Si Ming)
The core and bottleneck of a geo management system is the database storage performance and query capability. On one hand, storage services require low-latency data storage and reading that can deal with massive amounts of data; on the other hand, storage services also need to provide efficient geographic and multi-dimensional data retrieval. Table Store is a serverless and distributed NoSQL database that can completely meet the requirements of a geo management system.
In this article, we will create a geo management system for 100 million shops based on Table Store.
A certain shop search platform provides information about 100 million shops. Users can search for desired shops based on their own dimensions on the PC application and website of this platform. The platform needs to display the specific locations of shops and shop details on a map, and support redirecting to shop homepages.
Dimension 1: [within 1km] [less than 100 RMB per customer ] [top rating] [milk tea];
Dimension 2: [within Hangzhou city] [top rating] [Shen Jia*] shop;
Implementing fast and multi-dimensional geo queries is the core feature of a geo management solution. The following is a sample:
Note: This sample provides data information about 100 million shops. Sample in the official console: Project sample
The preceding screenshot shows an overview of the Shop Search system page based on Table Store. The sample is integrated in the Table Store console. You can log on to the console to experiment with the system. (If you are a new Table Store user, you need to click Activate Now for a trial of this service. The service activation is free. Order data is stored in public instances. A trial doesn't consume storage, network traffic, or CUs.)
The SearchIndex solution in Table Store allows you to easily set up a search system targeting 100 million shops. The SearchIndex feature allows creating indexes such as geo indexes and word-breaking string indexes. It provides you with the capabilities such as geo retrieval and multi-dimensional combined retrieval. Indexes can be created at any time. Inventory data and incremental data are automatically synchronized.
Table Store is a fully-hosted and distributed NoSQL data storage service from Alibaba Cloud that provides features such as storage of massive amounts of data, automatic sharding of hot data, and multi-dimensional retrieval of massive amounts of data. Table Store can effectively solve the challenge of large volumes of geo data.
You can create and activate indexes as needed. Table Store ensures the consistency of data synchronization, greatly reducing the work required for your solution design, service maintenance, and code development.
If you are satisfied with the shop search system for 100 million shops implemented based on Table Store and want to start to set up your own system, simply follow the following steps:
Activate the Table Store service in the console. Table Store is out-of-the-box (post-paid) and billed on a pay-as-you-go basis. Table Store also provides a free quota that is sufficient for functional tests. For more information, visit Table Store Console and Free quota description.
Create a Table Store instance in the console and select a region that supports SearchIndex. (Currently the SearchIndex feature has not been commercialized and is supported in the following regions:Beijing, Shanghai, Hangzhou, and Shenzhen. This feature will be gradually available in other regions.)
After the instance is created, open a ticket to apply for the SearchIndex beta test invitation. (After becoming commercialized, SearchIndex will be enabled by default. No fees will be incurred if the feature is not used.)
Use SDKs with SearchIndex (see the official website for more details). Currently, new functions are added for Java, Go, and Node.js SDKs.
<dependency> <groupId>com.aliyun.openservices</groupId> <artifactId>tablestore</artifactId> <version>4.8.0</version> </dependency>
$ go get github.com/aliyun/aliyun-tablestore-go-sdk
$ npm install email@example.com
The shop search system sample only uses one shop table that mainly includes fields such as shop type, shop name, shop location, average shop ratings, and per capita consumption. The table is designed as follows:
Table Name: geo_positon
You only need to create a "shop information table" under the instance for which you have successfully applied for the beta test invitation: Create and manage data tables in the console (or directly by using SDKs):
Table Store automatically synchronizes full and incremental index data: Users can create and manage indexes through the console (or, they can also create indexes using the SDK)
Insert test data (100 million entries are inserted into the sample in the console. You can insert a small amount of test data by yourself).
Data reading falls into two types:
1. Primary key reading
The primary key column is obtained based on the native Table Store: getRow, getRange, batchGetRow. Primary key reading is used for index (automatic) reverse lookup. Users can also provide a single query page for the primary key (Order MD5). And the query speed is extremely fast at the scale of 100 million entries of data. Multi-dimensional retrieval is not supported for the single primary key query;
2. Index reading (shop query)
Query based on the new SearchIndex function: the search interface. Users can freely design multi-dimensional combination queries for index fields. By setting and selecting different query parameters, different query criteria and different sorting methods are built. Currently, exact query, range query, prefix query, match query, wildcard query, phrase match query, and word breaking string query are supported, and they are combined by boolean AND and OR.
The following is the criteria for querying milk tea shops in the vicinity within 1km range at "36.76613, 111.41461":
List<Query> mustQueries = new ArrayList<Query>(); TermQuery termQuery = new TermQuery(); termQuery.setFieldName("type"); termQuery.setTerm(ColumnValue.fromString(milk tea)); mustQueries.add(termQuery); GeoDistanceQuery geoDistanceQuery = new GeoDistanceQuery(); geoDistanceQuery.setFieldName("pos"); geoDistanceQuery.setCenterPoint("36.76613,111.41461"); geoDistanceQuery.setDistanceInMeter(1000); mustQueries.add(geoDistanceQuery); BoolQuery boolQuery = new BoolQuery(); boolQuery.setMustQueries(mustQueries);
Alibaba Cloud Storage - May 14, 2019
digoal - December 22, 2020
Alibaba Clouder - March 22, 2019
Alibaba Clouder - February 14, 2020
digoal - December 23, 2020
digoal - January 19, 2021
A fully managed NoSQL cloud database service that enables storage of massive amount of structured and semi-structured dataLearn More
ApsaraDB for HBase is a NoSQL database engine that is highly optimized and 100% compatible with the community edition of HBase.Learn More
Block-level data storage attached to ECS instances to achieve high performance, low latency, and high reliabilityLearn More
Plan and optimize your storage budget with flexible storage servicesLearn More
More Posts by Alibaba Cloud Storage