Platform For AI:FeatureStore best practices

Method 1: Use the console

1. Create a FeatureStore project.

   1. Log on to the PAI console. In the navigation pane on the left, click Data Preparation > FeatureStore.

   2. Select a workspace and click Enter FeatureStore.

   3. Click Create Project. In the dialog box that appears, configure the project parameters.

      Parameter	Suggested value
      Name	Enter a custom name. This topic uses fs_demo as an example.
      Description	Enter a custom description.
      Offline Store	Select the MaxCompute data source you created.
      Online Store	Select the FeatureDB data source you created.

   4. Click Submit to create the FeatureStore project.

2. Create a feature entity.

   1. On the FeatureStore Project List page, click the project name to open the project details page.

   2. On the Feature Entity tab, click Create Feature Entity. In the dialog box that appears, configure the parameters for the user feature entity.

      Parameter	Suggested value
      Feature Entity Name	Enter a custom name. This topic uses user as an example.
      Join Id	user_id

   3. Click Submit.

   4. Click Create Feature Entity. In the dialog box that appears, configure the parameters for the item feature entity.

      Parameter	Suggested value
      Feature Entity Name	Enter a custom name. This topic uses item as an example.
      Join Id	item_id

   5. Click Submit to create the feature entity.

3. Create a feature view.

   1. On the Feature View tab of the project details page, click Create Feature View. In the dialog box that appears, configure the parameters for the user feature view.

      Parameter	Suggested value
      View Name	Enter a custom name. This topic uses user_table_preprocess_all_feature_v1 as an example.
      Type	Offline
      Write Method	Use Offline Table
      Data Source	Select the MaxCompute data source you created.
      Feature Table	Select the user table you prepared: rec_sln_demo_user_table_preprocess_all_feature_v1.
      Feature Fields	Select the user_id primary key.
      Sync To Online Feature Table	Yes
      Feature Entity	user
      Feature Lifecycle (seconds)	Keep the default value.

   2. Click Submit.

   3. Click Create Feature View. In the dialog box that appears, configure the item feature view.

      Parameter	Suggested value
      View Name	Enter a custom name. This topic uses item_table_preprocess_all_feature_v1 as an example.
      Type	Offline
      Write Method	Use Offline Table
      Data Source	Select the MaxCompute data source you created.
      Feature Table	Select the item table you prepared: rec_sln_demo_item_table_preprocess_all_feature_v1.
      Feature Fields	Select the item_id primary key.
      Sync To Online Feature Table	Yes
      Feature Entity	item
      Feature Lifecycle (seconds)	Keep the default value.

   4. After the configuration is complete, click Submit to create the feature view.

4. Create a label table.

   1. On the Label Table tab of the project details page, click Create Label Table. In the dialog box that appears, configure the label table information.

      Parameter	Suggested value
      Data Source	Select the MaxCompute data source you created.
      Table Name	Select the label table you prepared: rec_sln_demo_label_table.

   2. Click Submit.

5. Create a model feature.

   1. On the Model Feature tab of the project details page, click Create Model Feature. In the dialog box that appears, configure the model feature parameters.

      Parameter	Suggested value
      Model Feature Name	Enter a custom name. This topic uses fs_rank_v1 as an example.
      Select Features	Select the user and item feature views you created.
      Label Table Name	Select the label table you created: rec_sln_demo_label_table.

   2. Click Submit to create the model feature.

   3. On the model feature list page, click Details to the right of your model.

   4. In the Model Feature Details dialog box that appears, on the Basic Information tab, you can view the Exported Table Name, which is fs_demo_fs_rank_v1_trainning_set. This table is used for subsequent feature production and model training.

6. Install the FeatureStore Python SDK. For more information, see Use FeatureStore to manage features in a recommendation system.

Method 2: Use the FeatureStore Python SDK

1. Log on to the DataWorks console.

2. In the navigation pane on the left, click Resource Groups.

3. On the Exclusive Resource Groups tab, find the resource group whose Purpose is Data Scheduling. Click the icon next to its schedule resource and select O&M Assistant.

4. Click Create Command. In the dialog box that appears, configure the command parameters.

   Parameter	Suggested value
   Command Name	Enter a custom name. This topic uses install as an example.
   Command Type	Manual Input (pip Command Cannot Be Used To Install Third-party Packages)
   Command Content	/home/tops/bin/pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple https://feature-store-py.oss-cn-beijing.aliyuncs.com/package/feature_store_py-2.0.2-py3-none-any.whl
   Timeout	Set a custom time.

5. Click Create to create the command.

6. Click Run Command. In the dialog box that appears, click Run.

7. You can click Refresh to view the latest execution status. When the status changes to Success, the installation is complete.

For detailed steps on using the SDK, see DSW Gallery.

Python SDK

For more information, see Use the FeatureStore Python SDK to build a recommendation system.

Go SDK

The FeatureStore Go SDK is open source. For more information, see aliyun-pai-featurestore-go-sdk.

1. Installation

   Run the following command to install the FeatureStore Go SDK.

   go get github.com/aliyun/aliyun-pai-featurestore-go-sdk/v2

2. Usage

   1. Run the following command to initialize the client.

      accessId := os.Getenv("AccessId") 
      accessKey := os.Getenv("AccessKey") 
      regionId := "cn-hangzhou" 
      projectName := "fs_test_ots" 
      
      client, err := NewFeatureStoreClient(regionId, accessId, accessKey, projectName)

      Note

      The SDK connects directly to the online data source. Therefore, the client must run in a VPC environment. For example, Hologres requires connections to originate from a specified VPC.

   2. Retrieve feature data from a feature view.

      // get project by name 
      project, err := client.GetProject("fs_test_ots") 
      if err != nil {     
      	// t.Fatal(err) 
      } 
      // get featureview by name 
      user_feature_view := project.GetFeatureView("user_fea") 
      if user_feature_view == nil {     
      	// t.Fatal("feature view not exist") 
      } 
      
      // get online features 
      features, err := user_feature_view.GetOnlineFeatures([]interface{}{"100043186", "100060369"}, []string{"*"}, nil)

      Here, []string{"*"} means to retrieve all features in the feature view. You can also specify the names of the features that you want to retrieve.

      The following is a sample of the returned data:

      [     
      {         
      "city":"Hefei",    "follow_cnt":1,         "gender":"male",         "user_id":"100043186"    
       },     {                     
       "city":"",         "follow_cnt":5,         "gender":"male",         "user_id":"100060369"     
      } 
      ]

   3. Retrieve feature data from a model feature.

      A model feature can be associated with multiple feature entities. You can provide multiple join IDs to retrieve the corresponding features together.

      This example uses three join IDs: join_id, user_id, and item_id. When you fetch features, you must provide a value for each join ID.

      // get project by name 
      project, err := client.GetProject("fs_test_ots") 
      if err != nil {     
      	// t.Fatal(err) 
      }
      
      // get ModelFeature by name 
      model_feature := project.GetModelFeature("rank") 
      if model_feature == nil {     
      	// t.Fatal("model feature not exist") 
      } 
      
      // get online features 
      features, err := model_feature.GetOnlineFeatures(map[string][]interface{}{"user_id": {"100000676", "100004208"}, "item_id":{"238038872", "264025480"}} )

      The following is a sample of the returned data:

      [     
      {         
      "age":26,         "author":100015828,         "category":"14",         "city":"Shenyang",         "duration":63,         "gender":"male",         "item_id":"238038872",         "user_id":"100000676"    
       },     {         
      "age":23,         "author":100015828,         "category":"15",         "city":"Xi'an",         "duration":22,         "gender":"male",         "item_id":"264025480",         "user_id":"100004208"    
       }
       ]
      
      

      You can also specify a feature entity to retrieve all of its corresponding features.

      The following is a sample of the returned data:

      [    
       {         
      "age":26,         "city":"Shenyang",         "gender":"male",         "user_id":"100000676"    
       },     {        
       "age":23,         "city":"Xi'an",         "gender":"male",         "user_id":"100004208"    
       } 
      ]

Java SDK

The FeatureStore Java SDK is open source. For more information, see aliyun-pai-featurestore-java-sdk.

This section uses a Hologres data source as an example.

1. Run the following code to load environment variables and initialize the service.

   public static String accessId = "";
   public static String accessKey = "";
   // Configure the host based on the region
   public static String host = "";
   // Get the accessId and accessKey from the configured local environment variables
   static {
       accessId = System.getenv("ACCESS_KEY_ID");
       accessKey = System.getenv("ACCESS_KEY_SECRET");
   }

2. Create a configuration class that configures the regionId, accessId, accessKey, and project name.

   Configuration cf = new Configuration("cn-hangzhou",Constants.accessId,Constants.accessKey,"ele28");
   cf.setDomain(Constants.host);// Note: The default environment is a VPC environment

3. Initialize the client.

   ApiClient apiClient = new ApiClient(cf);
   // FS client
   FeatureStoreClient featureStoreClient = new FeatureStoreClient(apiClient);

4. Retrieve the project. This example uses a project named ele28.

   Project project=featureStoreClient.getProject("ele28");
   if(project==null){
       throw new RuntimeException("Project not found");
   }

5. Retrieve the feature view of the project. This example uses a feature view named mc_test.

   FeatureView featureView=project.getFeatureView("mc_test");
   if (featureView == null) {
       throw  new RuntimeException("FeatureView not found");
   }

6. Retrieve online feature data based on the feature view.

   Map<String,String> m1=new HashMap<>();
   m1.put("gender","gender1"); // Create an alias                                                                                                                                                                                                 user_id='100027781'(FS_INT64) age='28'(FS_INT64) city='null'(FS_STRING) item_cnt='0'(FS_INT64) follow_cnt='0'(FS_INT64) follower_cnt='2'(FS_INT64) register_time='1697641608'(FS_INT64) tags='0'(FS_STRING) gender1='female'(FS_STRING) ---------------

   To retrieve online features, String[]{"*"} retrieves all properties. You can also specify certain properties to retrieve only a subset of the information.

   FeatureResult featureResult1=featureView.getOnlineFeatures(new String[]{"100017768","100027781","100072534"},new String[]{"*"},m1);

   Output the feature information.

   while(featureResult1.next()){
       System.out.println("---------------");
       // Feature name
       for(String m:featureResult1.getFeatureFields()){
           System.out.print(String.format("%s=%s(%s) ",m,featureResult1.getObject(m),featureResult1.getType(m)));
       }
       System.out.println("---------------");
   }   

   The following data is returned.

   ---------------
   user_id='100017768'(FS_INT64)  age='28'(FS_INT64)  city='Dongguan'(FS_STRING) item_cnt='1'(FS_INT64)  follow_cnt='1'(FS_INT64)  follower_cnt='0'(FS_INT64) register_time='1697202320'(FS_INT64)  tags='1,2'(FS_STRING)  gender1='female'(FS_STRING) 
   ---------------

7. Retrieve the model.

   Model model=project.getModelFeature("model_t1");
   if(model==null){
       throw new RuntimeException("Model not found");
   }

   1. Retrieve data from the model feature.

      This model feature uses two join IDs: user_id and item_id. The number of values that you provide must match the number of join IDs. This example provides one value for each join ID.

      Map<String, List<String>> m2=new HashMap<>(); 
      m2.put("user_id",Arrays.asList("101683057"));
      m2.put("item_id",Arrays.asList("203665415"));

   2. Retrieve all feature data related to the user feature entity from the model feature.

      FeatureResult featureResult2 = model.getOnlineFeaturesWithEntity(m2,"user");

      The following data is returned.

      ---------------
      user_id='101683057' age='28' city='Shenzhen' follower_cnt='234' follow_cnt='0' gender='male' item_cnt='0' register_time='1696407642' tags='2' item_id='203665415' author='132920407' category='14' click_count='0' duration='18.0' praise_count='10' pub_time='1698218997' title='#Idiom Story' 
      ---------------

Cpp SDK

The FeatureStore C++ SDK is currently integrated into the EasyRec Processor. It is specifically optimized for feature extraction, cache management, and read operations to provide a high-performance, low-latency solution for large-scale recommendation scenarios.

• Memory usage: When dealing with complex and large-scale feature data, memory consumption is significantly reduced. The memory savings are more significant under high feature loads.

• Feature pull time: Instead of pulling features from online stores (such as FeatureDB and Hologres), the SDK pulls them directly from MaxCompute into the Elastic Algorithm Service (EAS) cache. This significantly shortens feature loading time. Additionally, MaxCompute provides better stability and extensibility, which reduces the load on online stores during scale-out operations.

• Model scoring time: Using this SDK improves the TP100 performance metric for model scoring, stabilizes response times, and significantly reduces timeout requests. These improvements enhance the overall reliability and user experience of the recommendation service.