PolarDB:Submit jobs to a Ray application

Comparison of methods

A Ray application offers multiple methods for job submission. Choose the one that best suits your use case and preferences.

Preparations

Before submitting a job, obtain from your Ray application the connection address and configuration information of the application, and add the private or public IP address of your development environment to the application whitelist.

Submit a job using the JupyterLab Terminal

This method directly uses the preset environment provided by the Ray application to run the ray job submit command. No local dependencies need to be installed, making it the preferred choice for quickly testing and running scripts.

1. Log on to JupyterLab:

   1. Open the Jupyter public URL in your browser.

   2. In the Password or token field, enter the key for secret.jupyterlab.password to access the JupyterLab interface.

2. Prepare a Python script:

   1. In the JupyterLab file browser, create a working directory (for example, src), double-click the working directory to open it, and create a Python script file (for example, script.py). In the toolbar at the top of the file browser on the left side of JupyterLab, click the upload button on the right of the + button to upload your local project files (such as the src directory and requirements.txt) to the working directory. On the Launcher page of JupyterLab, click Python File in the Other section to create a Python file.

   2. Example src/script.py content:

      import ray
      import time
      @ray.remote
      def retrieve_task(item, db):
          time.sleep(item / 10.)
          return item, db[item]
      if __name__ == "__main__":
          database = [
              "Learning", "Ray", "Flexible", "Distributed", "Python", "for",
              "Machine", "Learning"
          ]
          ray.init()
          db_object_ref = ray.put(database)
          retrieve_refs = [
              retrieve_task.remote(item, db_object_ref) for item in [0, 2, 4, 6]
          ]
          result = [print(data) for data in ray.get(retrieve_refs)]

3. Submit the job and view the results:

   1. In JupyterLab, use File > New Launcher to open a new terminal.

   2. In the terminal, run the following commands to submit the job and view its logs.

      # 1. Return to the parent directory to demonstrate using the --working-dir parameter.
      cd ..
      # 2. Set the environment variable. Replace the placeholder with your actual key.
      # Anonymous key for job submission: secret.jwt.anonKey
      ANON_KEY="<YOUR_ANON_KEY>"
      # 3. Submit the job.
      # Submit the job and wait for the result.
      ray job submit --headers "{\"Authorization\": \"Bearer $ANON_KEY\"}" --working-dir ./src -- python script.py
      # Submit the job without waiting for the result.
      ray job submit --headers "{\"Authorization\": \"Bearer $ANON_KEY\"}" --no-wait --working-dir ./src -- python script.py

      Example output:

      requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http://xxx/api/version
      (base) jovyan@pa-xxx:~/work$ ANON_KEY="exxx"
      Yxxx
      (base) jovyan@pa-2xxx:~/work$ ray job submit --headers "{\"Authorization\": \"Bearer $ANON_KEY\"}" --no-wait --working-dir ./src -- python script.py
      Job submission server address: http://1xxx:8265
      2025-12-18 03:47:06,460 INFO dashboard_sdk.py:338 -- Uploading package gcs://_ray_pkg_xxx.zip.
      2025-12-18 03:47:06,461 INFO packaging.py:576 -- Creating a file package for local module './src'.
      -------------------------------------------------------
      Job 'raysubmit_Jxxx' submitted successfully
      -------------------------------------------------------
      Next steps
        Query the logs of the job:
          ray job logs raysubmit_Jxxx
        Query the status of the job:
          ray job status raysubmit_xxx
        Request the job to be stopped:
          ray job stop raysubmit_Jxxx
      (base) jovyan@pa-xxx:~/work$

Run interactive jobs in Jupyter Notebook

Jupyter Notebook is ideal for scenarios that require line-by-line code execution, exploratory analysis, and debugging.

1. Log on and create a notebook:

   1. Open the Jupyter public URL in your browser.

   2. In the Password or token field, enter the key corresponding to secret.jupyterlab.password to access the JupyterLab interface.

   3. Use File > New Launcher to open a new Notebook, Console, or Python File. The Launcher page provides three Python development options: Python 3 (ipykernel) in the Notebook section is used to create an interactive notebook, Python 3 (ipykernel) in the Console section is used to open an interactive console, and Python File in the Other section is used to create a Python script file.

2. Run the code: Taking Notebook as an example, enter and run the following code in a code cell.

   import ray
   import time
   @ray.remote
   def retrieve_task(item, db):
       time.sleep(item / 10.)
       return item, db[item]
   if __name__ == "__main__":
       database = [
           "Learning", "Ray", "Flexible", "Distributed", "Python", "for",
           "Machine", "Learning"
       ]
       ray.init()
       db_object_ref = ray.put(database)
       retrieve_refs = [
           retrieve_task.remote(item, db_object_ref) for item in [0, 2, 4, 6]
       ]
       result = [print(data) for data in ray.get(retrieve_refs)]

   The output is as follows:

   (0, 'Learning')
   (2, 'Flexible')
   (4, 'Python')
   (6, 'Machine')

Submit a job using the Ray Python SDK

When you need to programmatically submit jobs from your local development environment, an existing Python application, or a CI/CD pipeline, the Ray Python SDK is recommended.

1. Prepare the environment

   Note

   First, install a Python 3 environment on your local machine.

   Install the Ray SDK:

   pip install ray    

2. Prepare the code files
   You need to create two Ray application scripts in your working directory. One is used for business processing and the other is used for submitting the job. An example of the working directory structure is as follows:
   

   - working-dir/
    - script.py # Your Ray business logic script
    - ray_job_submit.py # Script for submitting the job

   script.py

   import ray
   import time
   @ray.remote
   def retrieve_task(item, db):
       time.sleep(item / 10.)
       return item, db[item]
   if __name__ == "__main__":
       database = [
           "Learning", "Ray", "Flexible", "Distributed", "Python", "for",
           "Machine", "Learning"
       ]
       ray.init()
       db_object_ref = ray.put(database)
       retrieve_refs = [
           retrieve_task.remote(item, db_object_ref) for item in [0, 2, 4, 6]
       ]
       result = [print(data) for data in ray.get(retrieve_refs)]
               

   ray_job_submit.py

   import os
   import json
   import base64
   import hmac
   import hashlib
   import requests
   from datetime import timezone, timedelta, datetime
   from requests.auth import HTTPBasicAuth
   from ray.job_submission import JobSubmissionClient
   # Your Ray Dashboard address
   address = os.getenv("RAY_DASHBOARD_ADDRESS")
   # Authenticate directly using secret.jwt.anonKey.
   token = os.getenv("RAY_ANON_TOKEN")
   # Alternatively, generate a JWT using secret.jwt.secret.
   secret = os.getenv("RAY_JWT_SECRET")
   header = {"alg": "HS256", "typ": "JWT"}
   now = datetime.now(tz=timezone.utc)
   payload = {
     "sub": "anon",
     "iat": int(now.timestamp()),
     "exp": int((now + timedelta(hours=12)).timestamp()),
     "iss": "ray-dashboard",
     "aud": "ray-dashboard-client"
   }
   header_encoded = base64.urlsafe_b64encode(
     json.dumps(header,
                separators=(',', ':')).encode()).decode('utf-8').rstrip('=')
   payload_encoded = base64.urlsafe_b64encode(
     json.dumps(payload,
                separators=(',', ':')).encode()).decode('utf-8').rstrip('=')
   message = f"{header_encoded}.{payload_encoded}"
   signature = hmac.new(secret.encode(), message.encode(),
                        hashlib.sha256).digest()
   signature_encoded = base64.urlsafe_b64encode(signature).decode('utf-8').rstrip(
     '=')
   token = f"{message}.{signature_encoded}"
   # You can also obtain a JWT by sending a POST request to /api/auth/token. You need to provide a username and password.
   username = os.getenv('RAY_USERNAME')
   password = os.getenv('RAY_PASSWORD')
   response = requests.post(f'{address}/api/auth/token',
                            auth=HTTPBasicAuth(username, password),
                            timeout=5)
   if response.status_code == 200:
     token_data = response.json()
     token = token_data['access_token']
   else:
     print(f"Failed to get JWT: {response.status_code}")
   client = JobSubmissionClient(address,
                                headers={"Authorization": f"Bearer {token}"})
   job_id = client.submit_job(
     # Entrypoint shell command to execute
     entrypoint="python script.py",
     # Path to the local directory that contains the script.py file
     runtime_env={"working_dir": "./"})
   print(job_id)

3. Configure and run

   1. View the following information on the Ray Application Configuration Information page, and set it as environment variables or replace the values in the submit_job.py script.

      Variable	Configuration item	Example
      RAY_DASHBOARD_ADDRESS	The Ray application's connection address. Choose the public or private address based on your actual environment.	http://123.xxx.xxx.xxx:8265
      RAY_ANON_TOKEN	secret.jwt.anonKey	eyJhbGciOi...
      RAY_JWT_SECRET	secret.jwt.secret	tNhVxysSRD...
      RAY_USERNAME	secret.dashboard.username	admin
      RAY_PASSWORD	secret.dashboard.password	UTLof$rMVM...

   2. In your working directory, run the submission script:

      python ray_job_submit.py

   3. After the execution succeeds, you can log on to the Ray Dashboard and view the running status of the submitted job on the Jobs page. The job record shows Entrypoint as python script.py, Status as SUCCEEDED, and Status message as Job finished successfully, which indicates that the task has been successfully completed.

Submit a job from your local machine or an ECS instance by using the Ray CLI

This method is similar to using the JupyterLab Terminal but is executed from your own environment. It is suitable for developers who prefer the command line.

1. Ensure that the prerequisites are met

   • Network: Ensure that the IP address of your development environment is in the whitelist or security group of the application and can access the Ray Dashboard address.

   • To install Ray, run pip install ray.

2. Submit the job: In your local terminal or ECS Terminal, use the ray job submit command to submit a job. All commands must use the --address parameter to explicitly specify the Ray Dashboard address.

   # Set environment variables. Replace the placeholders with your actual values.
   # The Ray Dashboard address for the application. It must be prefixed with http://.
   RAY_ADDRESS="<YOUR_DASHBOARD_URL>"
   # Anonymous key for job submission: secret.jwt.anonKey
   ANON_KEY="<YOUR_ANON_KEY>"
   # Submit the job
   ray job submit --address "$RAY_ADDRESS" --headers "{\"Authorization\": \"Bearer $ANON_KEY\"}"  --working-dir <path_to_your_working_dir> -- python script.py

Submit a job using the REST API

This method is intended only for running simple, single-line commands. It cannot automatically upload local Python scripts or code directories. For most use cases, we recommend the more powerful and convenient Ray Python SDK or Ray CLI.

1. Prepare request information: Obtain the Ray Dashboard and the anonymous key secret.jwt.anonKey.

2. Send a POST request: Use curl or an HTTP client in any programming language to send the request. The following example uses Python:

   import requests
   import json
   # Your Ray Dashboard address. It must be prefixed with http://.
   RAY_ADDRESS = "<YOUR_DASHBOARD_URL>"
   # Your anonymous key.
   RAY_ANON_KEY = "<YOUR_ANON_KEY>"
   headers = {
       'Content-Type': 'application/json',
       "Authorization": f"Bearer {RAY_ANON_KEY}"
   }
   # The entrypoint can only be a simple, single-line command.
   payload = {
       "entrypoint": "python -c \"import ray; ray.init(); print(ray.nodes())\"",
   }
   # The API endpoint must end with a forward slash ("/").
   response = requests.post(
       f"{RAY_ADDRESS}/api/jobs/",
       json=payload,
       headers=headers
   )
   response.raise_for_status()
   job_info = response.json()
   print(f"Job submitted successfully: {job_info}")

   The output is as follows:

   Job submitted successfully: {'job_id': 'raysubmit_xxx', 'submission_id': 'raysubmit_xxx'}