Submit a data import task using an API operation | Data Import | Lindorm - Lindorm

This topic describes how to submit a data import task using an API operation.

Submit a task

API (POST): http://{BDSMaster}:12311/pro/proc/bulkload/create. Replace `{BDSMaster}` with the master hostname of the Lindorm instance. You can log on to the Lindorm console and find the hostname in the Basic Information section of the Cluster Information page.

Parameters:

Parameter	Description
src	The name of the source data source.
dst	The name of the destination data source.
readerConfig	The configuration of the read plugin. The file must be in JSON format. For a configuration example, see General-purpose batch import service.
writerConfig	The configuration of the write plugin. The file must be in JSON format. For a configuration example, see General-purpose batch import service.
driverSpec	The specification of the driver. Valid values: small, medium, large, and xlarge. Set this parameter to large.
instances	The number of executors.
fileType	If the source data source is HDFS, specify the file type. Valid values: CSV and Parquet.
sparkAdditionalParams	The extension parameters. This parameter is optional.

Example:

curl -d "src=hdfs&dst=ld&readerConfig={\"filePath\":\"parquet/\",\"column\":[\"id\",\"intcol\",\"doublecol\",\"stringcol\",\"string1col\",\"decimalcol\"]}&writerConfig={\"columns\":[\"ROW||String\",\"f:intcol||Int\",\"f:doublecol||Double\",\"f:stringcol||String\",\"f:string1col||String\",\"f:decimalcol||Decimal\"],\"namespace\":\"default\",\"lindormTable\":\"bulkload_test\",\"compression\":\"zstd\"}&driverSpec=large&instances=5&fileType=Parquet" -H "Content-Type: application/x-www-form-urlencoded" -X POST http://{LTSMaster}:12311/pro/proc/bulkload/create

The following response is returned. The value of the `message` parameter is the task ID.

{"success":"true","message":"proc-91-ff383c616e5242888b398e51359c****"}

Get task information

API operation (GET): http://{LTSMaster}:12311/pro/proc/{procId}/info. Replace {LTSMaster} with the master hostname of the Lindorm instance. You can obtain the hostname from the Basic Information section on the Cluster Information page of LTS.
Parameter: `procId` specifies the task ID.

Example:

curl http://{LTSMaster}:12311/pro/proc/proc-91-ff383c616e5242888b398e51359c****/info

The following response is returned:

{
    "data":{
        "checkJobs":Array,
        "procId":"proc-91-ff383c616e5242888b398e51359c****",  // Task ID
        "incrJobs":Array,
        "procConfig":Object,
        "stage":"WAIT_FOR_SUCCESS",
        "fullJobs":Array,
        "mergeJobs":Array,
        "srcDS":"hdfs",    // Source data source
        "sinkDS":"ld-uf6el41jkba96****",  // Destination data source
        "state":"RUNNING",   // Task status
        "schemaJob":Object,   
        "procType":"SPARK_BULKLOAD"   // Task type
    },
    "success":"true"
}

Stop a task

API operation (GET): http://{LTSMaster}:12311/pro/proc/{procId}/abort. Replace {LTSMaster} with the master hostname of the Lindorm instance. You can obtain the hostname from the Basic Information section on the Cluster Information page of LTS.
Parameter: `procId` specifies the task ID.

Example:

curl http://{LTSMaster}:12311/pro/proc/proc-91-ff383c616e5242888b398e51359c****/abort

The following response is returned:

{"success":"true","message":"ok"}

Retry a task

API operation (GET): http://{LTSMaster}:12311/pro/proc/{procId}/retry. Replace {LTSMaster} with the master hostname of the Lindorm instance. You can obtain the hostname from the Basic Information section on the Cluster Information page of LTS.
Parameter: `procId` specifies the task ID.

Example:

curl http://{LTSMaster}:12311/pro/proc/proc-91-ff383c616e5242888b398e51359c****/retry

The following response is returned:

{"success":"true","message":"ok"}

Delete a task

API operation (GET): http://{LTSMaster}:12311/pro/proc/{procId}/delete. Replace {LTSMaster} with the master hostname of the Lindorm instance. You can obtain the hostname from the Basic Information section on the Cluster Information page of LTS.
Parameter: `procId` specifies the task ID.

Example:

curl http://{LTSMaster}:12311/pro/proc/proc-91-ff383c616e5242888b398e51359c****/delete

The following response is returned:

{"success":"true","message":"ok"}