RDS SQLFlow is an AI solution based on AliSQL. It provides a fully managed service to simplify the implementation of machine learning. With built-in machine learning services and SQL extensions, RDS SQLFlow connects AliSQL to an AI engine, allowing you to describe the data flow and AI constructs behind your applications with minimal SQL code.
Step 1: Set up the rds_sqlflow service
-
Create an RDS Custom instance with the following parameter settings. For detailed instructions, see Create an RDS Custom instance.
-
In the Instance Type section, set Architecture to AI Node.
-
For Image, select SQLFlow under Default Image.
-
-
Use an ECS instance in the same region and VPC as your RDS Custom instance to connect to it using the
sshcommand. For detailed instructions, see Connect to an RDS Custom instance. -
Navigate to the /home/script directory and run the
start_sqlflow.shscript. This script creates a local MySQL process and starts the RDS SQLFlow container.cd /home/script ./start_sqlflow.sh -P 50054 -u aliyun_rds -p aliyun_rdsParameters:
Parameter
Description
-PThe host port that is mapped to the port in the RDS SQLFlow container. The default value is 50054.
-uThe username for the MySQL process. The default value is aliyun_rds.
-pThe password for the MySQL user. If you do not specify this parameter, a random password is generated.
Output:
-
A Transport Layer Security (TLS) certificate. Copy the certificate content and save it as a
ca.crtfile on your client. -
A client connection command. You can use this command to connect to the RDS SQLFlow node and specify the data source for machine learning.
Save the following content as ca.crt on the client: -----BEGIN CERTIFICATE----- xxx -----END CERTIFICATE----- Connect to SQLFlow with the following command: sqlflow -c ca.crt -d'mysql://aliyun_rds:aliyun_rds@tcp(172.xxx.xxx.xxx:3306)/mysql?maxAllowedPacket=0' -s172.xxx.xxx.xxx:xxx:50054 Also, you can change the mysql config to connect with your own mysql.Result: Run the following commands to check the script execution status.
-
Run the
ps -ef | grep mysqlcommand to check the status of the MySQL process.[root@rc-xxx script]# ps -ef | grep mysql mysql 22669 1 0 09:52 ? 00:00:02 /home/u01/mysql80_current/bin/mysqld --defaults-file=/etc/my.cnf root 22811 21833 0 09:57 pts/0 00:00:00 grep --color=auto mysql -
Run the
docker pscommand to check the status of the RDS SQLFlow container.[root@rc-xxx script]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS b8af67093420 rds-sqlflow:latest "bash /start.sh" About a minute ago Up About a minute 50051/tcp, 0.0.0.0:50054->50052/tcp, [::]:50054->50052/tcp
-
Step 2: Connect to the rds_sqlflow service
-
Install the RDS SQLFlow client on an ECS instance in the same region and VPC as the RDS Custom instance, and grant executable permissions to the client.
-
You can obtain the RDS SQLFlow client in one of the following ways:
-
From the RDS Custom instance, copy the RDS SQLFlow client file from the /home directory to the target ECS instance. This tutorial uses the /home directory as an example.
scp /home/sqlflow root@<private_IP_address_of_the_ECS_instance>:/home -
On the target ECS instance, download the client.
wget -O sqlflow "https://forsharelogs.oss-cn-beijing.aliyuncs.com/sqlflow?Expires=1729065153&OSSAccessKeyId=TMP.3KhXUKdnV6VCQmT15btm1k2tLRdWeBh26vq78fg2zJDLLfMAmrXYP2hWJKw64Wq7VWmLQLiMuCPCDiTv5ERnhL7a9e****&Signature=mgbW6Vh5q%2FP8aGxrzz0QtglBUxQ%3D"
-
-
On the target ECS instance, make the
sqlflowfile executable.cd /home chmod +x sqlflow -
On the target ECS instance, create a
ca.crtfile and paste the TLS certificate content that you generated in Step 1.vim ca.crt
-
-
Use the client connection command generated when you ran the
start_sqlflow.shscript in Step 1 to connect to the rds_sqlflow service. For example, if thesqlflowclient is installed in the /home directory, run the following command:/home/sqlflow -c ca.crt -d'mysql://user:password@tcp(mysqlHost:mysqlPort})/mysql?maxAllowedPacket=0' -ssqlflowHost:sqlflowPortParameters:
Parameter
Description
-cThe certificate file generated in Step 1.
-dSpecifies the MySQL data source for RDS SQLFlow. Example:
'mysql://user:password@tcp(mysqlHost:mysqlPort)/mysql?maxAllowedPacket=0'.-
user: The username for the MySQL data source. -
password: The password for the MySQL data source user. -
mysqlHost: The endpoint of the MySQL data source. -
mysqlPort: The port number of the MySQL data source.
NoteBy default, the client connection command generated in Step 1 uses the MySQL service running on the RDS SQLFlow server. You can also specify the endpoint of another MySQL service to use as the data source.
-sSpecifies the endpoint of the RDS SQLFlow server. Example:
sqlflowHost:sqlflowPort.-
sqlflowHost: The endpoint of the RDS SQLFlow server, which is the private IP address of the RDS Custom instance. -
sqlflowPort: The port number of the RDS SQLFlow server.
Example:
/home/sqlflow -c ca.crt -d'mysql://aliyun_rds:aliyun@tcp(47.XXX.XXX.153:3306})/mysql?maxAllowedPacket=0' -s172.XXX.XXX.180:50054The
Welcome to SQLFlowmessage indicates you have successfully connected and entered the SQLFlow CLI. -
Step 3: Prepare data
RDS SQLFlow requires a MySQL data source for machine learning.
The MySQL data source can be the MySQL database running on the RDS SQLFlow server or another MySQL database, such as an ApsaraDB RDS for MySQL instance.
This tutorial uses the MySQL database on the RDS SQLFlow server to build the Iris dataset and train a TensorFlow DNNClassifier algorithm. The DNNClassifier is suitable for classifying large datasets that use one-dimensional data as input and have complex internal patterns that are difficult to interpret.
The Iris dataset in this tutorial contains four features and one label. The four features describe the botanical morphology of each iris flower and are represented as floating-point numbers. The label represents the subspecies of each iris flower as an integer, which can be 0, 1, or 2. In TensorFlow, a feature column acts as a bridge between a column in your data and a feature used to train the model. TensorFlow provides multiple types of feature columns. For more information, see the official TensorFlow documentation. The table below describes the columns in the Iris dataset.
|
Name |
Type |
Description |
|
sepal_length |
float |
A feature. The sepal length. Unit: cm. |
|
sepal_width |
float |
A feature. The sepal width. Unit: cm. |
|
petal_length |
float |
A feature. The petal length. Unit: cm. |
|
petal_width |
float |
A feature. The petal width. Unit: cm. |
|
class |
int |
The label. The subspecies category. Valid values: 0, 1, and 2. |
In the SQLFlow CLI, run the following SQL statements to create the training dataset and evaluation dataset for machine learning.
Step 4: Train the model
In the SQLFlow CLI, run the following SQL statement to train a three-class DNNClassifier model. The model has two hidden layers with 100 nodes each. The learning rate for the optimizer is 0.1, and the training runs for 10 epochs.
CREATE DATABASE IF NOT EXISTS sqlflow_models;
SELECT * FROM iris.train -- Specify iris.train as the training dataset.
TO TRAIN DNNClassifier WITH
model.n_classes = 3, -- Set the number of classes to 3.
model.hidden_units = [100, 100], -- Specify two hidden layers, each with 100 units.
optimizer.learning_rate=0.1, -- Set the learning rate to 0.1.
train.epoch = 10 -- Set the number of training epochs to 10.
COLUMN sepal_length, sepal_width, petal_length, petal_width -- Specify the feature columns.
LABEL class -- Specify the label column as class.
INTO sqlflow_models.my_dnn_model; -- Save the trained model to the specified table.
Step 5: Evaluate the model
-
Use the
iris.testdataset to evaluate thesqlflow_models.my_dnn_modelmodel. The evaluation results are saved to thesqlflow_models.evaluate_result_tabletable.SELECT * FROM iris.test TO EVALUATE sqlflow_models.my_dnn_model WITH validation.metrics = Accuracy LABEL class INTO sqlflow_models.evaluate_result_table; SELECT * FROM sqlflow_models.evaluate_result_table; -
After the evaluation is complete, run the following SQL statement to view the results. You can fine-tune the model parameters based on these results.
SELECT * FROM sqlflow_models.evaluate_result_table;Result:
+----------------------+----------+ | LOSS | ACCURACY | +----------------------+----------+ | 0.035312581807374954 | 1 | +----------------------+----------+
Step 6: Make predictions
-
Use the
sqlflow_models.my_dnn_modelmodel to make predictions. The results are written to theclasscolumn of theiris.predicttable. The prediction statement is as follows:SELECT * FROM iris.test TO PREDICT iris.predict.class USING sqlflow_models.my_dnn_model; -
After the prediction is complete, run the following SQL statement to view the results.
SELECT * FROM iris.predict LIMIT 5;Result:
+--------------+-------------+--------------+-------------+-------+ | SEPAL LENGTH | SEPAL WIDTH | PETAL LENGTH | PETAL WIDTH | CLASS | +--------------+-------------+--------------+-------------+-------+ | 6.3 | 2.7 | 4.9 | 1.8 | 2 | | 5.7 | 2.8 | 4.1 | 1.3 | 1 | | 5 | 3 | 1.6 | 0.2 | 0 | | 6.3 | 3.3 | 6 | 2.5 | 2 | | 5 | 3.5 | 1.6 | 0.6 | 0 | +--------------+-------------+--------------+-------------+-------+