Deploy a PyTorch deep learning model on a security-enhanced instance - Instance| Alibaba Cloud Documentation Center

This topic describes how to deploy a PyTorch deep learning model on an Intel^® SGX-based security-enhanced instance and how to use the PyTorch model.

Background information

Artificial intelligence (AI) models are built upon large amounts of training data and computing power and are an extremely valuable form of intellectual property. PyTorch is widely recognized by AI developers for its flexible and dynamic programming environment, dynamic graph mechanism, and flexible networking architecture. Typically, PyTorch models are deployed on cloud servers provided by cloud service providers, such as Alibaba Cloud Elastic Compute Service (ECS) instances. All PyTorch model users and cloud service providers need to make sure that PyTorch models deployed on the public cloud are available but invisible and cannot be stolen by others.

Some security-enhanced ECS instances provide encrypted computing capabilities based on Intel^® Software Guard Extension (SGX) to create a hardware-level trusted confidential environment that offers a high degree of security. This ensures the confidentiality and integrity of essential code and data and protects them from malware attacks.

You can deploy PyTorch deep learning models in the trusted confidential environment of security-enhanced ECS instances. This ensures the security of data transmission and data usage and the integrity of PyTorch deep learning applications.

Architecture

The parameters of the SGX-based PyTorch end-to-end security model are shown in the Figure 1 figure. The model is stored in ciphertext at the deployment phase. Related operations are performed within the SGX enclave. The model parameters are decrypted only within the SGX enclave, and the keys are transmitted by using the secure remote attestation channel.

This practice involves three roles: dkeyserver, dkeycache, and PyTorch with SGX. Figure 2 shows these roles.

dkeyserver: the key server, which is deployed on the on-premises computer of the PyTorch model user. The PyTorch model user first encrypts the PyTorch model parameters by using the tools provided by PyTorch with SGX and then builds the on-premises key server dkeyserver. The encrypted model is then deployed on the Alibaba Cloud ECS SGX instance. The key server manages all model keys and model IDs and receives key requests from the key distribution service of the ECS SGX instance.
dkeycache: the key distribution service, which is deployed on the ECS SGX instance. The key distribution service of the ECS SGX instance first requests all model keys from the key server. After the key server completes the SGX remote attestation, it sends the keys to the SGX enclave for the key distribution service of the ECS SGX instance by using the secure remote attestation channel. This operation is automatically completed after the key distribution server is started.
PyTorch with SGX: the ECS SGX instance that runs PyTorch, which is deployed on the same server as dkeycache. When an Alibaba Cloud PyTorch instance uses models to make predictions or perform classification tasks for model inferences, it automatically sends a request for a model key to the key distribution service. The key is encrypted and sent to the SGX enclave of the PyTorch instance by using the SGX secure channel. The enclave started by PyTorch with SGX uses the key to decrypt the model parameters and perform model prediction operations. Model parameters are protected by SGX-based hardware throughout the process and are available but invisible, which ensures the security of data transmission and data usage.

Procedure

In this practice, dkeyserver, dkeycache, and PyTorch with SGX are deployed on the same security-enhanced instance for easy verification.

Create a security-enhanced instance and install the packages required to run PyTorch.
1. Create a security-enhanced instance.
  For more information about how to create a security-enhanced instance, see Create security-enhanced instances.
  Take note of the following parameters:
  - Instance Type: Select an instance type with at least 4 vCPUs and 16 GiB of memory. In this topic, ecs.g7t.4xlarge is used.
  - Image: Select Alibaba Cloud Linux 2.1903 LTS 64-bit (UEFI).
  - Public IP Address: Select Assign Public IPv4 Address.
2. Build an SGX encrypted computing environment.
  For more information about how to create a security-enhanced instance, see Build an SGX encrypted computing environment.
3. Install the packages required to run PyTorch.
  PyTorch has requirements on the version of software such as Python and GCC. Run the following commands to install the specified version of software:
```
yum update --skip-broken
sudo yum install -y teesdk git gcc-c++ scl-utils alinux-release-experimentals  python36-devel
sudo yum install -y devtoolset-7-gcc devtoolset-7-gdb devtoolset-7-binutils devtoolset-7-make devtoolset-7-gcc devtoolset-7-gcc-c++
scl -l devtoolset-7
ln -sf /opt/rh/devtoolset-7/root/bin/g++ /usr/bin/g++
ln -sf /opt/rh/devtoolset-7/root/bin/gcc /usr/bin/gcc
ln -sf /opt/rh/devtoolset-7/root/bin/c++ /usr/bin/c++
ln -sf /usr/bin/python3 /usr/bin/python
```
  If the page shown in the following figure appears, the packages are installed.
4. Install the PyTorch dependency library, encryption and decryption dependency library, and CMake.
```
pip3 install --upgrade pip
pip3 install astunparse numpy ninja pyyaml mkl mkl-include setuptools cmake cffi typing_extensions future six requests dataclasses setuptools_rust pycryptodomex pycryptodome torchvision
ln -sf /usr/local/bin/cmake /usr/bin/cmake
ln -sf /usr/local/bin/cmake /bin/cmake
```
Log on to the ECS instance.
For more information, see Connection methods.
Switch to the working directory such as /home/test and obtain the PyTorch sample code.
The sample code contains code of dkeyserver, dkeycache, and PyTorch with SGX.
```
cd /home/PyTorch
git clone https://github.com/intel/sgx-pytorch -b sgx pytorch
cd /home/test/pytorch
git submodule sync && git submodule update --init --recursive
```

Compile PyTorch with SGX in the ECS SGX instance.

Compile oneAPI Deep Neural Network Library (oneDNN).
oneDNN is an open-source cross-platform performance library of basic building blocks for deep learning applications. This library is optimized for Intel Architecture Processors, Intel Processor Graphics, and Xe Architecture graphics. oneDNN is intended for developers of deep learning applications and models who are interested in improving application performance on Intel CPUs and GPUs.
```
source /opt/alibaba/teesdk/intel/sgxsdk/environment
cd /home/test/pytorch/third_party/sgx/linux-sgx
git am ../0001*
cd external/dnnl
make
sudo cp sgx_dnnl/lib/libsgx_dnnl.a /opt/alibaba/teesdk/intel/sgxsdk/lib64/libsgx_dnnl2.a
sudo cp sgx_dnnl/include/*  /opt/alibaba/teesdk/intel/sgxsdk/include/
```
Compile the PyTorch enclave.
The enclave of PyTorch with SGX performs model parameter decryption and model prediction operations.
```
source /opt/alibaba/teesdk/intel/sgxsdk/environment
cd /home/test/pytorch/enclave_ops/ideep-enclave
make
```

Compile PyTorch.

cd /home/test/pytorch
pip3 uninstall torch    #Uninstall the installed PyTorch. Then, install the self-compiled PyTorch.
source /opt/alibaba/teesdk/intel/sgxsdk/environment
python setup.py develop --cmake-only
sudo python setup.py develop && python -c "import torch"

Compile the secure PyTorch computing operator.

source /opt/alibaba/teesdk/intel/sgxsdk/environment
cd /home/test/pytorch/enclave_ops/secure_op && mkdir build && cd build
cmake -DCMAKE_PREFIX_PATH="$(python -c 'import torch.utils; print(torch.utils.cmake_prefix_path)')" ..
make

Compile and generate the dkeyserver executable file on the key server and the dkeycache executable file on the ECS SGX instance.
```
cd /home/test/pytorch/enclave_ops/deployment
make
```
Start the key service on the key server.
```
cd /home/test/pytorch/enclave_ops/deployment/bin/dkeyserver
sudo ./dkeyserver
```
The key server starts and waits for key requests from the dkeycache service deployed on the ECS SGX instance.
Compile dkeycache on the ECS SGX instance and start the key distribution service.
```
cd /home/test/pytorch/enclave_ops/deployment/bin/dkeycache
sudo ./dkeycache
```
After startup, dkeycache requests all model keys from dkeyserver. After dkeyserver completes the SGX remote attestation, dkeyserver sends the keys to the SGX enclave of dkeyserver by using the secure remote attestation channel.
Run ResNet-based test cases on the ECS SGX instance.
```
cd /home/test/pytorch/enclave_ops/test
sudo python whole_resnet.py
```
The ciphertext parameters of the PyTorch model are decrypted in the SGX enclave. The keys are obtained from dkeycache and then encrypted and transmitted to the enclave.