Platform For AI:Data source configuration

All data source configurations use a Map[string]object structure. The key, such as holo_info or mysql_info in the examples, is a custom name that uniquely identifies the data source and can be referenced in other configurations.

Hologres

Hologres configurations correspond to HologresConfs in the main configuration.

Configuration example:

 "HologresConfs": {
      "holo_info": {
        "DSN": "postgres://<UserName>:<Password>@<HOST>/<DATABASENAME>?sslmode=disable&connect_timeout=1"
      }
  }

Parameter	Type	Required	Description
DSN	string	Yes	The connection string, containing the username, password, host, and database name.

Connection pool parameters

The Hologres connection pool defaults to a maximum of 50 idle connections and 100 open connections. You can adjust these limits with the following environment variables:

• HOLOGRES_MAX_OPEN_CONNS: Maximum number of open connections. Default: 100. Must be a positive integer; invalid values fall back to 100.

• HOLOGRES_MAX_IDLE_CONNS: Maximum number of idle connections. Default: 50. Must be a positive integer; invalid values fall back to 50.

If your engine service is deployed on PAI-EAS, you can add environment variables on the service update page, or add the following to the env array within the containers field of your service configuration JSON:

    {
        "name": "HOLOGRES_MAX_IDLE_CONNS",
        "value": "100"
    },
    {
        "name": "HOLOGRES_MAX_OPEN_CONNS",
        "value": "200"
    }

Kafka

Kafka configurations correspond to KafkaConfs in the main configuration.

Configuration example:

"KafkaConfs":{
  "kafka_info": {
    "BootstrapServers":"",
    "Topic":""
  }
}

Parameter	Type	Required	Description
BootstrapServers	string	Yes	The connection information for the Kafka server.
Topic	string	Yes	The name of the topic.

Redis

Redis configurations correspond to RedisConfs in the main configuration.

Configuration example:

"RedisConfs":{
  "redis_info":{
    "Host":"",
    "Port":6379,
    "Password":"",
    "DbNum":0,
    "MaxIdle":0,
    "ConnectTimeout":30,
    "ReadTimeout":50,
    "WriteTimeout":50
  }
}

Parameter	Type	Required	Description
Host	string	Yes	The host address.
Port	int	Yes	The port number.
Password	string	Yes	The login password.
DbNum	int	No	The Redis database index. The default is 0.
MaxIdle	int	No	The maximum number of idle connections.
ConnectTimeout	int	No	The timeout for establishing a connection.
ReadTimeout	int	No	The timeout for read operations.
WriteTimeout	int	No	The timeout for write operations.

MySQL

MySQL configurations correspond to MysqlConfs in the main configuration.

Configuration example:

 "MysqlConfs": {
      "mysql_info": {
        "DSN": "<USERNAME>:<PASSWORD>@tcp(<HOST>)/<DATABASENAME>?parseTime=true&loc=Asia%2FShanghai"
      }
  }

Parameter	Type	Required	Description
DSN	string	Yes	The connection string, containing the username, password, host, database name, and other parameters.

Graph

Graph configurations correspond to GraphConfs in the main configuration.

Configuration example:

 "GraphConfs": {
      "graph_info": {
         "Host":"",
    	 "UserName":"",
    	 "Passwd":""
      }
  }

Parameter	Type	Required	Description
Host	string	Yes	The host address.
UserName	string	Yes	The username.
Passwd	string	Yes	The password.

PAI-FeatureStore

PAI-FeatureStore configurations correspond to FeatureStoreConfs in the main configuration. For more information about PAI-FeatureStore, see FeatureStore Overview.

Configuration example:

 "FeatureStoreConfs": {
      "fs_info": {
         "RegionId":"cn-hangzhou",
    	 "ProjectName":"",
    	 "FeatureDBUsername": "",
         "FeatureDBPassword": "",
         "HologresUsername": "",
         "HologresPassword": ""
      }
  }

Parameter	Type	Required	Description
RegionId	string	Yes	The region ID, such as cn-hangzhou or cn-beijing.
ProjectName	string	Yes	The name of the project created in PAI-FeatureStore.
FeatureDBUsername	string	No	Username for FeatureDB. Required if FeatureDB is the online store for PAI-FeatureStore.
FeatureDBPassword	string	No	Password for FeatureDB. Required if FeatureDB is the online store for PAI-FeatureStore.
HologresUsername	string	No	Username for Hologres. Required if Hologres is the online store for PAI-FeatureStore.
HologresPassword	string	No	Password for Hologres. Required if Hologres is the online store for PAI-FeatureStore.

SLS

Simple Log Service (SLS) configurations correspond to SlsConfs in the main configuration.

Configuration example:

 "SlsConfs": {
      "sls_info": {
    	  "EndPoint":"",
    	  "ProjectName":"",
    	  "LogstoreName":""
      }
  }

Parameter	Type	Required	Description
EndPoint	string	Yes	The endpoint of the SLS service. For more information, see SLS endpoints.
ProjectName	string	Yes	The project name.
LogstoreName	string	Yes	The name of the Logstore.

DataHub

DataHub configurations correspond to DatahubConfs in the main configuration.

Configuration example:

 "DatahubConfs": {
     "datahub_info": {
         "Endpoint":"",
         "ProjectName":"",
         "TopicName":"",
    	 "Schemas":[]DatahubTopicSchema
      }
  }

Parameter	Type	Required	Description
Endpoint	string	Yes	The endpoint of the DataHub service. For more information, see DataHub endpoints.
ProjectName	string	Yes	The project name.
TopicName	string	Yes	The name of the topic.
Schemas	[]DatahubTopicSchema	Yes	The schema of the topic.

DatahubTopicSchema

Parameter	Type	Required	Description
Field	string	Yes	The name of the schema field.
Type	string	Yes	The data type of the schema field. Valid values: string, integer, bigint, double, float, timestamp.

Tablestore

Tablestore configurations correspond to TableStoreConfs in the main configuration.

Configuration example:

 "TableStoreConfs": {
        "tablestore_info": {
            "EndPoint": "",
            "InstanceName": ""
        }
  }

Parameter	Type	Required	Description
EndPoint	string	Yes	The endpoint of the Tablestore service. For more information, see Tablestore endpoints.
InstanceName	string	Yes	The instance name.

OpenSearch

OpenSearch configurations correspond to OpenSearchConfs in the main configuration.

Configuration example:

"OpenSearchConfs": {
    "opensearch_info":{
        "EndPoint": "opensearch-cn-beijing.aliyuncs.com"
    }
}

Parameter	Type	Required	Description
EndPoint	string	Yes	The endpoint of the OpenSearch service. For more information, see OpenSearch endpoints.