This topic describes how to use Open Computing Language (OpenCL) on an f3 instance to create an image and download the image to a Field Programmable Gate Array (FPGA).

Prerequisites

  • An Alibaba Cloud account is created. To create an Alibaba Cloud account, go to the Alibaba Cloud official website.
  • An f3 instance is created and assigned a public IP address.

    For more information about how to create an f3 instance, see Create an f3 instance.

    Note Only images shared by Alibaba Cloud can be used on f3 instances.
  • A rule is added to the security group to which the f3 instance belongs to allow access over SSH port 22.
  • The ID of the f3 instance is obtained on the Instances page of the ECS console.
  • An Object Storage Service (OSS) bucket dedicated to FPGA as a Service (FaaS) is created.

    The bucket and the f3 instance belong to the same account and reside in the same region. For more information, see Create buckets.

  • You must complete the following operations before you can manage FPGA-based instance as a RAM user:
    • Create a RAM user and grant permissions to the RAM user. For more information, see Create a RAM user and Grant permissions to a RAM user.

      The permissions you must grant to the RAM users include AliyunECSReadOnlyAccess, AliyunOSSFullAccess, and AliyunRAMFullAccess.

    • Click here to grant permissions to FaaS.
    • Obtain the AccessKey ID and AccessKey secret of the RAM user.

Background information

Before you perform the operations, take note of the following items:
  • All operations described in this topic must be performed by one account in the same region.
  • We recommend that you manage an f3 instance as a RAM user. You must create a role for the RAM user and grant the role temporary permissions to access the specified OSS bucket.
  • The operations and commands described in this topic are based on the SDAccel development environment 2018.2. If you use the SDAccel development environment of other versions, the operations and commands may vary.

Procedure

Step 1. Set up the environment

  1. Connect to an f3 instance.
    Note The subsequent compilation process may take a few hours. We recommend that you log on using Screen or nohup to avoid forced logout due to an SSH timeout.
  2. Run the following command to install Screen:
    yum install screen -y                        
  3. Run the following command to start Screen:
    screen -S f3opencl                        
  4. Run the following command to set up the environment:
    source /root/xbinst_oem/F3_env_setup.sh xocl  #Run the command each time you open a new terminal window.
    Note
    • The process to set up the environment involves installing the xocl driver, setting the vivado environment variable, checking the vivado license, detecting the aliyun-f3 sdaccel platform, configuring 2018.2 runtime, and checking the faascmd version.
    • If you want to run an emulation of SDAccel, do not run the preceding commands to set up the environment. You only need to configure the environment variable for vivado separately.
    • We recommend that you use Makefile for emulation.

Step 2. Compile a binary file

Perform the following operations to compile the vadd and kernel_global_bandwidth binary files:

  • Example 1: vadd
    1. Copy the example directory.
      cp -rf /opt/Xilinx/SDx/2018.2/examples . /
    2. Go to the vadd directory.
      cd examples/vadd/
    3. Run the cat sdaccel.mk | grep "XDEVICE=" command to check the value of XDEVICE. Make sure that the value is xilinx_aliyun-f3_dynamic_5_0.
    4. Perform the following operations to modify the common.mk file:
      1. Run the vim ../common/common.mk command to open the file.
      2. At the end of code line 61, add the compilation parameter --xp param:compiler.acceleratorBinaryContent=dcp. The parameter may be added in code line 60, 61, or 62. The modified code is as follows:
        CLCC_OPT += $(CLCC_OPT_LEVEL) ${DEVICE_REPO_OPT} --platform ${XDEVICE} ${KERNEL_DEFS} ${KERNEL_INCS} --xp param:compiler.acceleratorBinaryContent=dcp
        Note You must add the compilation parameter --xp param:compiler.acceleratorBinaryContent=dcp so that Xilinx® OpenCL™ Compiler (xocc) generates a DCP file after the placement and routing is completed. The file cannot be a bit file. Then, you can submit the DCP file to the compilation server.
    5. Run the following command to compile the program:
      make -f sdaccel.mk xbin_hw
      If the following similar information is displayed, the compilation of the binary file is started. The compilation process may take a few hours.f3_opencl_1
  • Example 2: kernel_global_bandwidth
    1. Run the following commands in sequence to clone xilinx 2018.2 example:
      git clone https://github.com/Xilinx/SDAccel_Examples.git
      cd SDAccel_Examples/
      git checkout 2018.2
      Note The Git branch version must be 2018.2.
    2. Run the cd getting_started/kernel_to_gmem/kernel_global_bandwidth/ command to access the directory.
    3. Perform the following operations to modify the Makefile file:
      1. Run the vim Makefile command to open the file.
      2. Set DEVICES to xilinx_aliyun-f3_dynamic_5_0.
      3. In code line 33, add the compilation parameter --xp param:compiler.acceleratorBinaryContent=dcp. The modified code is as follows:
        CLFLAGS +=--xp "param:compiler.acceleratorBinaryContent=dcp" --xp "param:compiler.preserveHlsOutput=1" --xp "param:compiler.generateExtraRunData=true" --max_memory_ports bandwidth -DNDDR_BANKS=$(ddr_banks)
    4. Run the following command to compile the program:
      make TARGET=hw

      If the following similar information is displayed, the compilation of the binary file is started. The compilation process may take a few hours.

      f3_opencl_2

Step 3. Check the packaging script

Run the following command to check whether the packaging script exists:
file /root/xbinst_oem/sdaccel_package.sh
If cannot open (No such file or directory) is displayed in the command output, the script does not exist. You must run the following command to manually download the packaging script:
wget http://fpga-tools.oss-cn-shanghai.aliyuncs.com/sdaccel_package.sh

Step 4. Create an image

  1. Initialize the faascmd tool and set up the OSS environment.
    1. Run the following command to configure the AccessKey pair of the RAM user:
      #Replace <hereIsYourSecretId> with the AccessKey ID, and <hereIsYourSecretKey> with the AccessKey secret of the RAM user.
      faascmd config --id=<hereIsYourSecretId> --key=<hereIsYourSecretKey>  
    2. Run the following command to configure the OSS bucket dedicated to FaaS:
    #Replace <hereIsYourBucket> with the name of the OSS bucket that you created.
    faascmd auth --bucket=<hereIsYourBucket>  
  2. Run the ls command to obtain the file suffixed by .xclbin.
    f3_opencl_3
  3. Run the following command to package the binary file:
    /root/xbinst_oem/sdaccel_package.sh -xclbin=/opt/Xilinx/SDx/2018.2/examples/vadd/bin_vadd_hw.xclbin
    After the binary file is packaged, you can find the packaged file in the same directory, as shown in the following figure.f3_opencl_4

Step 5. Download the image

You can use a scripted or step-by-step process to upload the netlist files and download the FPGA image.

  • Scripted process: This process applies only to f3 instances with a single FPGA.
    1. Run the following command to upload and generate the image:
      sh /root/xbinst_oem/tool/faas_upload_and_create_image.sh <bit.tar.gz - name of the package to be uploaded>
      f3_opencl_5
    2. Run the following command to download the image:
      sh /root/xbinst_oem/tool/faas_download_image.sh <bit.tar.gz - package name> <0/1>  #The last number <0/1> stands for the FPGA serial number in the instance.
      Note 0 indicates the first FPGA of the f3 instance. For single-FPGA instances, the FPGA serial number is always 0. For instances with multiple FPGAs, such as an instance with four FPGAs, the serial numbers are 0, 1, 2, and 3.
      If you want to download the same image to multiple FPGAs, add the serial number to the end of each command line. For example, run the following commands to download the same image to four FPGAs:
      sh /root/xbinst_oem/tool/faas_download_image.sh <bit.tar.gz - package name> 0
      sh /root/xbinst_oem/tool/faas_download_image.sh <bit.tar.gz - package name> 1
      sh /root/xbinst_oem/tool/faas_download_image.sh <bit.tar.gz - package name> 2
      sh /root/xbinst_oem/tool/faas_download_image.sh <bit.tar.gz - package name> 3
  • Step-by-step process: Use the faascmd tool to perform operations. For more information, see Use faascmd.
    1. Run the following commands in sequence to upload the package to your OSS bucket, and then upload the GBS file from your OSS bucket to the OSS bucket in the FaaS administrative unit:
      faascmd upload_object --object=bit.tar.gz --file=bit.tar.gz
      faascmd create_image --object=bit.tar.gz --fpgatype=xilinx --name=<hereIsFPGAImageName> --tags=<hereIsFPGAImageTag> --encrypted=false --shell=<hereIsShellVersionOfFPGA>
      f3_opencl_6f3_opencl_7
    2. Run the following command to check whether the FPGA image is downloadable:
      faascmd list_images
      The command output is as follows:
      • If "State":"compiling" is displayed, the FPGA image is being compiled.
      • If "State":"success" is displayed, the FPGA image is downloaded. You must find and record the value of FpgaImageUUID.f3_opencl_8
    3. Run the following command. Find and record the value of FpgaUUID in the command output.
      faascmd list_instances --instanceId=<hereIsYourInstanceId> #Replace <hereIsYourInstanceId> with the ID of the f3 instance.
    4. Run the following command to download the FPGA image:
      #Replace <hereIsYourInstanceId> with the ID the f3 instance. Replace <hereIsFpgaUUID> with the value of FpgaUUID that you recorded. Replace <hereIsImageUUID> with the value of FpgaImageUUID that you recorded.
      faascmd download_image --instanceId=<hereIsYourInstanceId> --fpgauuid=<hereIsFpgaUUID> --fpgatype=xilinx --imageuuid=<hereIsImageUUID> --imagetype=afu --shell=<hereIsShellVersionOfFpga>
      f3_9
    5. Run the following command to check whether the image is downloaded:
      faascmd fpga_status --fpgauuid=<hereIsFpgaUUID> --instanceId=<hereIsYourInstanceId> #Replace <hereIsFpgaUUID> with the value of FpgaUUID that you recorded earlier. Replace <hereIsYourInstanceId> with the ID of the f3 instance.
      The following figure shows the command output. If the value of FpgaImageUUID in the command output is the same as that of FpgaImageUUID that you recorded, and "TaskStatus":"valid" is displayed, the image is downloaded.f3_10

Step 6. Run the host program

  1. Run the following command to set up the environment:
    source /root/xbinst_oem/F3_env_setup.sh xocl  #Run the command each time you open a new terminal window.
  2. Configure the sdaccel.ini file.
    In the directory where the Host binary file is located, run the vim sdaccel.ini command to create the sdaccel.ini file and enter the following content:
    [Debug]
    profile=true
    [Runtime]
    runtime_log = "run.log"
    hal_log = hal.log
    ert=false
    kds=false
  3. Run the host program.
    • For vadd, run the following commands:
      make -f sdaccel.mk host
      ./vadd bin_vadd_hw.xclbin
    • For kernel_global_bandwidth, run the following command:
      ./kernel_global
    If Test Passed is displayed in the command output, the test is passed.

Other common commands

The following table describes some common commands for f3 instances.

Task Command
View the help documentation make -f ./sdaccel.mk help
Run software emulation make -f ./sdaccel.mk run_cpu_em
Run hardware emulation make -f ./sdaccel.mk run_hw_em
Compile only the host code make -f ./sdaccel.mk host
Compile and generate downloadable files make -f sdaccel.mk xbin_hw
Clean a work directory make -f sdaccel.mk clean
Forcibly clean a work directory make -f sdaccel.mk cleanall
Note
  • During emulation, follow the Xilinx emulation process. You do not need to set up the F3_env_setup environment.
  • The SDAccel runtime and SDAccel development platform are available in the official f3 images provided by Alibaba Cloud. You can also click SDAccel runtime and SDAccel development platform to download them.