PolarDB:Confidential Database Performance Test Report

Test environment

Test tool

oltpbench is an open-source benchmark tool for Online Transactional Processing (OLTP) that tests the performance and scalability of database management systems. It supports multiple benchmark programs, including TPC-C, TPC-E, and YCSB. This report uses the TPC-C benchmark to evaluate Confidential Database performance in OLTP scenarios.

Metric

TPS (Transactions Per Second): the number of transactions a database executes per second, measured by the number of successful COMMIT operations.

Test steps

Confidential Database encrypts data based on the column encryption rules you configure. This test adjusts the set of encrypted columns in the TPC-C benchmark to measure performance across different encryption coverage levels.

1. Set up oltpbench on your ECS instance. For details, see Configure oltpbench.

2. Integrate EncJDBC with oltpbench. For details, see Integrate EncJDBC.

3. Create an oltpbench configuration file:

   <?xml version="1.0"?>
   <parameters>
   
       <!-- Connection details -->
       <dbtype>mysql</dbtype>
       <driver>com.aliyun.encdb.mysql.jdbc.EncDriver</driver>
       <DBUrl>jdbc:mysql:encdb://xxxx.rwlb.rds.aliyuncs.com:3306/xxxx</DBUrl>
       <DBName>xxxx</DBName>
       <username>xxxx</username>
       <password>xxxxx</password>
       <isolation>TRANSACTION_SERIALIZABLE</isolation>
   
       <scalefactor>1</scalefactor>
   
       <!-- The workload -->
       <terminals>8</terminals>
       <works>
           <work>
               <time>180</time>
               <rate>unlimited</rate>
               <weights>45,43,4,4,4</weights>
           </work>
       </works>
   
   
       <!-- TPCC specific -->
           <transactiontypes>
           <transactiontype>
                   <name>NewOrder</name>
           </transactiontype>
           <transactiontype>
                   <name>Payment</name>
           </transactiontype>
           <transactiontype>
                   <name>OrderStatus</name>
           </transactiontype>
           <transactiontype>
                   <name>Delivery</name>
           </transactiontype>
           <transactiontype>
                   <name>StockLevel</name>
           </transactiontype>
           </transactiontypes>
   </parameters>

4. After compiling the oltpbench project, import data into the database:

   ./oltpbenchmark -b tpcc -c tpcc_encmysql.xml --load=true --create=true -s 1 -o tpcc

5. Adjust the encryption settings and run the benchmark to measure TPS:

   ./oltpbenchmark -b tpcc -c tpcc_encmysql.xml --execute=true -s 1 -o tpcc

Test scenarios and results

Each result is the average TPS from three test runs, each lasting one minute. Results include the overhead of decrypting encrypted fields returned to the client. In this TPC-C scenario, Queries Per Second (QPS) is a fixed multiple of TPS, at approximately 9 to 10 times the TPS value, so only TPS is reported. The terminals parameter represents the number of concurrent clients, which corresponds to the number of parallel test threads.