DataWorks:Use large language models

1. Call a large language model in a large language model node

Data Studio (new version) in DataWorks lets you use a large language model node to process data. In the large language model node, you can configure a large language model service and call the model.

2. Call a large language model in a Shell node

This example shows how to call a large language model in a Shell node to answer a specific question.

1. Create a Shell node and add the following sample command to the node:

   Note

   Replace the large language model service invocation address that starts with http and the token that starts with DW with the actual values that you obtained in the Preparations section.

   curl -X POST http://ms-xxx.cn-beijing.dataworks-model.aliyuncs.com/v1/completions -H "Authorization: DW-ms-xxx" -H "Content-Type: application/json" -d '{"prompt":"What are the differences and connections between AI, machine learning, and deep learning?", "stream":"false", "max_tokens": 1024}' -v    

2. After you edit the node content, in the Run Configuration section on the right side of the node editor, select the resource group whose network connectivity configuration to the model service is complete.

3. Click Run to call the deployed model service and execute the command.

   Note

   • Node scheduling configuration: If you want to run the node that calls the model service periodically, go to the Scheduling section on the right. Select the resource group where you have completed the network connectivity configuration with the model service. Then, configure the scheduling properties in the Scheduling Policies section.

   • Node publishing: If the task must run in the production environment, click the icon to publish the task. A node runs periodically only after it is published to the production environment.

3. Call a large language model in a Python node

This example shows how to call a large language model in a Python node to follow an instruction to write a poem.

1. This example depends on the Python requests library. Use the following parameters to create a custom image based on an official DataWorks image and install the dependency.

   Custom image parameter	Configuration description
   Image Name/ID	You can select an image suitable for Python nodes from the DataWorks image list.
   Supported Task Type	Select Python.
   Package	Python 3: requests. Script: /home/tops/bin/pip3 install 'urllib3<2.0'.

2. Create a Python node and add the following sample code to the node:

   Note

   Replace the large language model service invocation address that starts with http and the token that starts with DW with the actual values that you obtained in the Preparations section.

   import requests
   import json
   import time
   import sys
   
   def stream_print_response():
       httpUrl = "http://ms-xxxx.cn-beijing.dataworks-model.aliyuncs.com"
       apikey = "DW-ms-xxxx"
       url = httpUrl + "/v1/completions"
       headers = {
           "Authorization": apikey,
           "Content-Type": "application/json"
       }
       data = {
           "prompt": "Please write a poem about spring",
           "stream": True,
           "max_tokens": 512
       }
   
       try:
           response = requests.post(url, headers=headers, json=data, stream=True)
           response.raise_for_status()
   
           full_text = ""  # Accumulate the full response to prevent loss.
           buffer = ""     # Used to handle incomplete JSON lines (optional).
   
           for line in response.iter_lines():
               if not line:
                   continue  # Skip empty lines.
   
               line_str = line.decode('utf-8').strip()
               # print(f"[DEBUG] Received line: {line_str}")  # For debugging.
   
               if line_str.startswith("data:"):
                   data_str = line_str[5:].strip()  # Remove "data: ".
   
                   if data_str == "[DONE]":
                       print("\n[Stream response ended]")
                       break
   
                   # Try to parse JSON.
                   try:
                       parsed = json.loads(data_str)
                       choices = parsed.get("choices", [])
                       if choices:
                           delta_text = choices[0].get("text", "")
                           if delta_text:
                               # Accumulate to the full text.
                               full_text += delta_text
   
                               # Print the new characters one by one.
                               for char in delta_text:
                                   print(char, end='', flush=True)
                                   sys.stdout.flush()
                                   time.sleep(0.03)  # Typewriter effect.
   
                   except json.JSONDecodeError as e:
                       # print(f"[WARNING] JSON parsing failed: {e}, original text: {data_str}")
                       continue
   
           print(f"\n\n[Full response length: {len(full_text)} characters]")
           print(f"[ Full content]:\n{full_text}")
   
       except requests.exceptions.RequestException as e:
           print(f" Request failed: {e}")
       except Exception as e:
           print(f" Other error: {e}")
   
   if __name__ == "__main__":
       stream_print_response()
   

3. After you edit the node content, in the Run Configuration section on the right side of the node editor, select the resource group for which you have completed the network connectivity configuration and the custom Image from Step 1 that has the requests library installed.

4. Click Run to call the deployed model service to execute the command.

   Note

   • Node scheduling configuration: To run the node that calls the model service periodically, go to the Scheduling Configuration section on the right. Select the resource group for which you have completed the network connectivity configuration and the custom Image from Step 1 with the requests library installed. Then, configure the scheduling properties in the Scheduling Policy section.

   • Node publishing: If the task must run in the production environment, click the icon to publish the task. A node runs periodically only after it is published to the production environment.

1. Call a vector model service in a large language model node

Data Studio (new version) lets you use a large language model node to process data. In the large language model node, you can configure a large language model service and call a vector model.

2. Call a vector model in a Shell node

This example shows how to call a vector model in a Shell node to convert text into a vector.

1. Create a Shell node and add the following sample command to the node:

   Note

   Replace the large language model service invocation address that starts with http and the token that starts with DW with the actual values that you obtained in the Preparations section.

   curl -X POST "http://ms-xxx.cn-beijing.dataworks-model.aliyuncs.com/v1/embeddings" \
     -H "Authorization: DW-ms-xxx" \
     -H "Content-Type: application/json" \
     -d '{
       "input": "This is a piece of text that needs to be converted into a vector",
       "model": "bge-m3"
         }'

2. After you edit the node content, in the Run Configuration section on the right side of the node editor, select the resource group where you have completed the network connectivity configuration for the model service.

3. Click Run to execute the command and call the deployed model service.

   Note

   • Node scheduling configuration: To run the node that calls the model service periodically, go to the Scheduling section on the right. Select the resource group where you have completed the network connectivity configuration for the model service. Then, configure the scheduling properties in the Scheduling Policies section.

   • Node publishing: If the task must run in the production environment, click the icon to publish the task. A node runs periodically only after it is published to the production environment.

3. Call a vector model in a Python node

This example shows how to call a vector model in a Python node to convert text into a vector.

1. This example depends on the Python requests library. Use the following parameters to create a custom image based on an official DataWorks image and install the dependency.

   Custom image parameter	Configuration description
   Image Name/ID	You can select an image suitable for Python nodes from the DataWorks image list.
   Supported Task Type	Select Python.
   Package	Python 3: requests. Script: /home/tops/bin/pip3 install 'urllib3<2.0'.

2. Create a Python node and add the following sample code to the node:

   Note

   Replace the large language model service invocation address that starts with http and the token that starts with DW with the actual values that you obtained in the Preparations section.

   import requests
   import json
   import sys
   
   # Replace with your API address and token.
   api_url = "http://ms-xxx.cn-beijing.dataworks-model.aliyuncs.com" + "/v1/embeddings"
   token = "DW-ms-xxx"
   
   print("api_url:"+api_url)
   print("token:"+token)
   
   headers = {
       "Authorization": f"{token}",
       "Content-Type": "application/json"
   }
   
   payload = {
       "input": "Test text",
       "model": "bge-m3"
   }
   
   try:
       response = requests.post(api_url, headers=headers, data=json.dumps(payload))
       print("Response status code:", response.status_code)
       print("Response content:", response.text)  # View detailed error messages.
   except Exception as e:
       print("Request exception:", e)

3. After you edit the node content, in the Run Configuration section on the right side of the node editor, select the resource group where you have completed the network connectivity configuration and the custom Image from Step 1 that has the requests library installed.

4. Click Run to execute the command to call the deployed model service.

   Note

   • Node scheduling configuration: To run a node that calls a model service periodically, go to the Scheduling Configurations section on the right. Select the resource group for which you have completed the network connectivity configuration to the model service. Then, configure the scheduling properties in the Scheduling Policies section.

   • Node publishing: If the task must run in the production environment, click the icon to publish the task. A node runs periodically only after it is published to the production environment.