This topic describes how to use TensorFlow to develop an image classification model in the Machine Learning Platform for AI console.
Prerequisites
- An Object Storage Service (OSS) bucket is created. Machine Learning Platform for AI is authorized to access the OSS bucket. For more information, see Create buckets and Grant PAI the permissions to access OSS.
Important When you create an OSS bucket, make sure that versioning is disabled for the bucket. Otherwise, model training may fail.
- GPU resources are enabled.
Note GPU resources that are based on MaxCompute are not supported.
Background information
Due to the development of the Internet, a large amount of data related to images and voice is generated. Unstructured data poses a great challenge to data engineers due to the following reasons:
- Technical expertise is required to use deep learning algorithms.
- Computing resources such as GPU compute engines are pricey.
Machine Learning Designer and Machine Learning Studio provide a built-in image classification template that uses a deep learning framework. You can create an experiment from the template and use the experiment in scenarios such as content moderation and object detection.
Datasets
In the following sample experiment, the CIFAR-10 dataset is used. The dataset contains 60,000 color images of 32 × 32 pixels. The images are classified into 10 categories, including airplane, automobile, bird, cat, deer, dog, frog, horse, ship, and truck, as shown in the following figure. For more information about the download URLs of the dataset and relevant code, see
CIFAR-10 dataset.
In the experiment, the dataset is divided into a training dataset of 50,000 images and a prediction dataset of 10,000 images. The training dataset is further divided into five
training batches. The entire prediction dataset is used as a
test batch. Each batch is a separate file, as shown in the following figure.
Data preparation
Upload the dataset files and relevant code that are used in this experiment to an OSS bucket. For example, you can create a folder named
aohai_test in an OSS bucket and four subfolders in the aohai_test folder, as shown in the following figure.
The four subfolders are used for the following purposes:
- check_point: stores the model that is generated by the experiment.
Note After you create an experiment from the template in Machine Learning Studio, you must set the Checkpoint Output Directory/Model Input Directory parameter of the TensorFlow component to the path of an existing OSS folder. Otherwise, you cannot run the experiment. In this experiment, set the Checkpoint Output Directory/Model Input Directory parameter of the TensorFlow component to the path of the check_point subfolder.
- cifar-10-batches-py: stores the cifar-10-batcher-py file of the training dataset and the bird_mount_bluebird.jpg file of the prediction dataset.
- train_code: stores the cifar_pai.py file, which contains the training code.
- predict_code: stores the cifar_predict_pai.py file, which contains the prediction code.
Use TensorFlow to develop an image classification model
This section uses Machine Learning Studio as an example to demonstrate how to use TensorFlow to classify images.
- Go to the Machine Learning Designer page.
- Log on to the Machine Learning Platform for AI console.
- In the left-side navigation pane, click Workspaces. On the Workspace list page, click the name of the workspace that you want to manage.
- In the left-side navigation pane, choose to go to the Machine Learning Designer page.
- In the upper-right corner of the Visualized Modeling (Machine Learning Designer) page, click Go to Studio (Old Version).
- Create an experiment.
- Run the experiment and view the result.
- In the upper-left corner of the canvas, click Run.
- After the experiment stops running, you can view the prediction result in the OSS path that is specified by Checkpoint Output Directory/Model Input Directory.
Training code
This section describes the key code in the
cifar_pai.py file.
- The following code can be used to train a convolutional neural network (CNN) model for image classification:
network = input_data(shape=[None, 32, 32, 3], data_preprocessing=img_prep, data_augmentation=img_aug) network = conv_2d(network, 32, 3, activation='relu') network = max_pool_2d(network, 2) network = conv_2d(network, 64, 3, activation='relu') network = conv_2d(network, 64, 3, activation='relu') network = max_pool_2d(network, 2) network = fully_connected(network, 512, activation='relu') network = dropout(network, 0.5) network = fully_connected(network, 10, activation='softmax') network = regression(network, optimizer='adam', loss='categorical_crossentropy', learning_rate=0.001)
- The following code can be used to generate a model named model.tfl:
model = tflearn.DNN(network, tensorboard_verbose=0) model.fit(X, Y, n_epoch=100, shuffle=True, validation_set=(X_test, Y_test), show_metric=True, batch_size=96, run_id='cifar10_cnn') model_path = os.path.join(FLAGS.checkpointDir, "model.tfl") print(model_path) model.save(model_path)
Prediction code
This section describes the key code in the
cifar_predict_pai.py file. The system reads the
bird_bullocks_oriole.jpg image file and resizes the image to 32 × 32 pixels. Then, the system passes the image to the
model.predict function. The output of the function is the weights of the 10 categories based on which the image is recognized:
['airplane','automobile','bird','cat','deer','dog','frog','horse','ship','truck']. The system returns the category that has the highest weight as the prediction result.
predict_pic = os.path.join(FLAGS.buckets, "bird_bullocks_oriole.jpg")
img_obj = file_io.read_file_to_string(predict_pic)
file_io.write_string_to_file("bird_bullocks_oriole.jpg", img_obj)
img = scipy.ndimage.imread("bird_bullocks_oriole.jpg", mode="RGB")
# Scale it to 32x32
img = scipy.misc.imresize(img, (32, 32), interp="bicubic").astype(np.float32, casting='unsafe')
# Predict
prediction = model.predict([img])
print (prediction[0])
print (prediction[0])
#print (prediction[0].index(max(prediction[0])))
num=['airplane','automobile','bird','cat','deer','dog','frog','horse','ship','truck']
print ("This is a %s"%(num[prediction[0].index(max(prediction[0]))]))