Use o TPC-H para avaliar o desempenho de consultas OLAP, consultas pontuais de chave/valor e atualizações de dados no Hologres.
TPC-H
A seguir, um trecho da especificação do TPC Benchmark™ H (TPC-H):
O TPC-H é um benchmark de suporte a decisões. Compõe-se de um conjunto de consultas ad-hoc orientadas a negócios e de modificações concorrentes de dados. As consultas e os dados que populam o banco de dados foram escolhidos por sua ampla relevância no setor. Este benchmark representa sistemas de suporte a decisões que examinam grandes volumes de dados, executam consultas com alto grau de complexidade e fornecem respostas a questões críticas de negócio.
Baixe a especificação completa do TPCH.
A implementação do TPC-H descrita aqui é baseada no benchmark TPC-H, mas não está totalmente em conformidade com ele. Portanto, estes resultados de teste não são comparáveis aos resultados publicados do benchmark TPC-H.
Dataset
O TPC-H é um benchmark de suporte a decisões desenvolvido pelo Transaction Processing Performance Council (TPC), amplamente utilizado para avaliar o desempenho de consultas analíticas.
O TPC-H modela um data warehouse para um sistema de vendas baseado em um ambiente de produção real. Inclui 8 tabelas, e seu volume de dados varia de 1 GB a 3 TB. O benchmark contém 22 consultas. A métrica principal é o tempo de resposta das consultas, conforme definido no benchmark TPC-H.
Casos de uso
Este plano de testes abrange os seguintes cenários:
OLAP query scenario test: Executa as 22 consultas do benchmark TPC-H em tabelas orientadas a colunas.
Key-value point query scenario test: Realiza consultas pontuais filtradas por chave primária na tabela ORDERS orientada a linhas.
Data update scenario: Avalia o desempenho de atualização de dados do mecanismo OLAP em tabelas com chave primária.
A ferramenta de geração de dados do TPC-H usa o fator de escala (SF) para controlar o tamanho do dataset. Um SF igual a 1 produz 1 GB de dados.
O raw data volume indicado acima não inclui índices. Reserve espaço em disco adicional conforme necessário.
Observações
Para garantir resultados de teste consistentes, crie uma nova instância para cada teste. Não utilize uma instância redimensionada.
Testar cenários de consulta OLAP
-
Prepare o ambiente.
Create a Hologres instance. Este teste utiliza uma instância dedicada de pagamento conforme o uso com 96 núcleos e 384 GB de memória. Selecione os recursos de computação de acordo com suas necessidades.
-
Create an ECS instance com as seguintes especificações:
Parâmetro
Especificação
Specification
ecs.g6.4xlarge
Image
Alibaba Cloud Linux 3.2104 LTS 64-bit
Data disk
Um ESSD. A capacidade depende do volume dos dados de teste.
-
Baixe e configure o toolkit Hologres Benchmark.
Faça login na instância ECS. Connect to an instance.
-
Instale o client psql.
yum update -y yum install postgresql-server -y yum install postgresql-contrib -y -
Baixe e extraia o toolkit Hologres Benchmark.
wget https://oss-tpch.oss-cn-hangzhou.aliyuncs.com/hologres_benchmark.tar.gz tar xvf hologres_benchmark.tar.gz -
Acesse o diretório hologres_benchmark.
cd hologres_benchmark -
Execute o comando
vim group_vars/allpara configurar os parâmetros do benchmark.# db config login_host: "" login_user: "" login_password: "" login_port: "" # benchmark run cluster: hologres cluster: "hologres" RUN_MODE: "HOTRUN" # benchmark config scale_factor: 1 work_dir_root: /your/working_dir/benchmark/workdirs dataset_generate_root_path: /your/working_dir/benchmark/datasetsDescrição dos parâmetros:
Tipo
Parâmetro
Descrição
Parâmetros de conexão do service Hologres
login_host
O endpoint VPC da instância Hologres.
Para encontrar esse valor, faça login no console do Hologres e acesse a página de detalhes da instância. O endpoint VPC está na coluna Domain Name da seção Network Information.
NotaEste endpoint não inclui o número da porta. Exemplo:
hgpostcn-cn-nwy364b5v009-cn-shanghai-vpc-st.hologres.aliyuncs.comlogin_port
A porta do endpoint VPC da instância Hologres.
Para encontrar esse valor, faça login no console do Hologres e acesse a página de detalhes da instância. A porta está na coluna Domain Name da seção Network Information.
login_user
Seu AccessKey ID.
Acesse o gerenciamento de AccessKey para obter seu AccessKey ID.
login_password
Seu AccessKey secret.
Parâmetros de configuração do benchmark
scale_factor
O fator de escala do dataset, que controla o tamanho dos dados gerados em GB. O valor padrão é 1.
work_dir_root
O diretório de trabalho raiz. Esse diretório armazena os dados do TPC-H, como instruções de criação de tabelas e instruções SQL executadas. O valor padrão é
/your/working_dir/benchmark/workdirs.dataset_generate_root_path
O caminho de armazenamento do dataset de teste gerado. O valor padrão é
/your/working_dir/benchmark/datasets.
-
Execute o seguinte comando para iniciar um teste TPC-H automatizado de ponta a ponta.
Este teste gera os dados, cria um banco de dados de teste (por exemplo, tpc_h_sf1000), cria as tabelas e importa os dados.
bin/run_tpch.shComo alternativa, execute o comando a seguir para rodar apenas o teste de consultas TPC-H.
bin/run_tpch.sh query -
Visualize os resultados do teste.
-
Visão geral dos resultados do teste
O comando bin/run_tpch.sh exibe resultados semelhantes ao seguinte:
TASK [tpc_h : debug] ************************************************************************************************** skipping: [worker-1] ok: [master] => { "command_output.stdout_lines": [ "[info] 2024-06-28 14:46:09.768 | Run sql queries started.", "[info] 2024-06-28 14:46:09.947 | Run q10.sql started.", "[info] 2024-06-28 14:46:10.088 | Run q10.sql finished. Time taken: 0:00:00, 138 ms", "[info] 2024-06-28 14:46:10.239 | Run q11.sql started.", "[info] 2024-06-28 14:46:10.396 | Run q11.sql finished. Time taken: 0:00:00, 154 ms", "[info] 2024-06-28 14:46:10.505 | Run q12.sql started.", "[info] 2024-06-28 14:46:10.592 | Run q12.sql finished. Time taken: 0:00:00, 85 ms", "[info] 2024-06-28 14:46:10.703 | Run q13.sql started.", "[info] 2024-06-28 14:46:10.793 | Run q13.sql finished. Time taken: 0:00:00, 88 ms", "[info] 2024-06-28 14:46:10.883 | Run q14.sql started.", "[info] 2024-06-28 14:46:10.981 | Run q14.sql finished. Time taken: 0:00:00, 95 ms", "[info] 2024-06-28 14:46:11.132 | Run q15.sql started.", "[info] 2024-06-28 14:46:11.266 | Run q15.sql finished. Time taken: 0:00:00, 131 ms", "[info] 2024-06-28 14:46:11.441 | Run q16.sql started.", "[info] 2024-06-28 14:46:11.609 | Run q16.sql finished. Time taken: 0:00:00, 165 ms", "[info] 2024-06-28 14:46:11.728 | Run q17.sql started.", "[info] 2024-06-28 14:46:11.818 | Run q17.sql finished. Time taken: 0:00:00, 88 ms", "[info] 2024-06-28 14:46:12.017 | Run q18.sql started.", "[info] 2024-06-28 14:46:12.184 | Run q18.sql finished. Time taken: 0:00:00, 164 ms", "[info] 2024-06-28 14:46:12.287 | Run q19.sql started.", "[info] 2024-06-28 14:46:12.388 | Run q19.sql finished. Time taken: 0:00:00, 98 ms", "[info] 2024-06-28 14:46:12.503 | Run q1.sql started.", "[info] 2024-06-28 14:46:12.597 | Run q1.sql finished. Time taken: 0:00:00, 93 ms", "[info] 2024-06-28 14:46:12.732 | Run q20.sql started.", "[info] 2024-06-28 14:46:12.888 | Run q20.sql finished. Time taken: 0:00:00, 154 ms", "[info] 2024-06-28 14:46:13.184 | Run q21.sql started.", "[info] 2024-06-28 14:46:13.456 | Run q21.sql finished. Time taken: 0:00:00, 269 ms", "[info] 2024-06-28 14:46:13.558 | Run q22.sql started.", "[info] 2024-06-28 14:46:13.657 | Run q22.sql finished. Time taken: 0:00:00, 97 ms", "[info] 2024-06-28 14:46:13.796 | Run q2.sql started.", "[info] 2024-06-28 14:46:13.935 | Run q2.sql finished. Time taken: 0:00:00, 136 ms", "[info] 2024-06-28 14:46:14.051 | Run q3.sql started.", "[info] 2024-06-28 14:46:14.155 | Run q3.sql finished. Time taken: 0:00:00, 101 ms", "[info] 2024-06-28 14:46:14.255 | Run q4.sql started.", "[info] 2024-06-28 14:46:14.341 | Run q4.sql finished. Time taken: 0:00:00, 83 ms", "[info] 2024-06-28 14:46:14.567 | Run q5.sql started.", "[info] 2024-06-28 14:46:14.799 | Run q5.sql finished. Time taken: 0:00:00, 230 ms", "[info] 2024-06-28 14:46:14.881 | Run q6.sql started.", "[info] 2024-06-28 14:46:14.950 | Run q6.sql finished. Time taken: 0:00:00, 67 ms", "[info] 2024-06-28 14:46:15.138 | Run q7.sql started.", "[info] 2024-06-28 14:46:15.320 | Run q7.sql finished. Time taken: 0:00:00, 180 ms", "[info] 2024-06-28 14:46:15.572 | Run q8.sql started.", "[info] 2024-06-28 14:46:15.831 | Run q8.sql finished. Time taken: 0:00:00, 256 ms", "[info] 2024-06-28 14:46:16.081 | Run q9.sql started.", "[info] 2024-06-28 14:46:16.322 | Run q9.sql finished. Time taken: 0:00:00, 238 ms", "[info] 2024-06-28 14:46:16.325 | ----------- HOT RUN finished. Time taken: 3255 mill_sec -----------------" ] } skipping: [worker-2] skipping: [worker-3] skipping: [worker-4] TASK [tpc_h : clear Env] ********************************************************************************************** skipping: [worker-1] skipping: [worker-2] skipping: [worker-3] skipping: [worker-4] ok: [master] TASK [tpc_h : debug] ************************************************************************************************** ok: [master] => { "work_dir": "/your/working_dir/benchmark/workdirs/tpc_h/sf1" } skipping: [worker-1] skipping: [worker-2] skipping: [worker-3] skipping: [worker-4] -
Detalhes dos resultados do teste
O comando
bin/run_tpch.shcria um diretório de trabalho e exibe seu caminho como <work_dir>. Acesse esse diretório para visualizar as instruções de consulta, as instruções de criação de tabelas e os logs de execução.TASK [tpc_h : debug] ************************************************************ ok: [master] => { "work_dir": "/your/working_dir/benchmark/workdirs/tpc_h/sf1" } skipping: [worker-1] skipping: [worker-2] skipping: [worker-3] skipping: [worker-4]Execute
cd <work_dir>/logspara acessar o diretório de logs e visualizar os resultados do teste e a saída detalhada do SQL.A estrutura de diretórios de <work_dir> é a seguinte:
working_dir/ `-- benchmark |-- datasets | `-- tpc_h | `-- sf1 | |-- worker-1 | | |-- customer.tbl | | `-- lineitem.tbl | |-- worker-2 | | |-- orders.tbl | | `-- supplier.tbl | |-- worker-3 | | |-- nation.tbl | | `-- partsupp.tbl | `-- worker-4 | |-- part.tbl | `-- region.tbl `-- workdirs `-- tpc_h `-- sf1 |-- config |-- hologres | |-- logs | | |-- q10.sql.err | | |-- q10.sql.out | | |-- q11.sql.err | | |-- q11.sql.out | | |-- q12.sql.err | | |-- q12.sql.out | | |-- q13.sql.err | | |-- q13.sql.out | | |-- q14.sql.err | | |-- q14.sql.out | | |-- q15.sql.err | | |-- q15.sql.out | | |-- q16.sql.err | | |-- q16.sql.out | | |-- q17.sql.err | | |-- q17.sql.out | | |-- q18.sql.err | | |-- q18.sql.out | | |-- q19.sql.err | | |-- q19.sql.out | | |-- q1.sql.err | | |-- q1.sql.out | | |-- q20.sql.err | | |-- q20.sql.out | | |-- q21.sql.err | | |-- q21.sql.out | | |-- q22.sql.err | | |-- q22.sql.out | | |-- q2.sql.err | | |-- q2.sql.out | | |-- q3.sql.err | | |-- q3.sql.out | | |-- q4.sql.err | | |-- q4.sql.out | | |-- q5.sql.err | | |-- q5.sql.out | | |-- q6.sql.err | | |-- q6.sql.out | | |-- q7.sql.err | | |-- q7.sql.out | | |-- q8.sql.err | | |-- q8.sql.out | | |-- q9.sql.err | | |-- q9.sql.out | | `-- run.log | `-- logs-20240628144609 | |-- q10.sql.err | |-- q10.sql.out | |-- q11.sql.err | |-- q11.sql.out | |-- q12.sql.err | |-- q12.sql.out | |-- q13.sql.err | |-- q13.sql.out | |-- q14.sql.err | |-- q14.sql.out | |-- q15.sql.err | |-- q15.sql.out | |-- q16.sql.err | |-- q16.sql.out | |-- q17.sql.err | |-- q17.sql.out | |-- q18.sql.err | |-- q18.sql.out | |-- q19.sql.err | |-- q19.sql.out | |-- q1.sql.err | |-- q1.sql.out | |-- q20.sql.err | |-- q20.sql.out | |-- q21.sql.err | |-- q21.sql.out | |-- q22.sql.err | |-- q22.sql.out | |-- q2.sql.err | |-- q2.sql.out | |-- q3.sql.err | |-- q3.sql.out | |-- q4.sql.err | |-- q4.sql.out | |-- q5.sql.err | |-- q5.sql.out | |-- q6.sql.err | |-- q6.sql.out | |-- q7.sql.err | |-- q7.sql.out | |-- q8.sql.err | |-- q8.sql.out | |-- q9.sql.err | |-- q9.sql.out | `-- run.log |-- queries | |-- ddl | | |-- hologres_analyze_tables.sql | | `-- hologres_create_tables.sql | |-- q10.sql | |-- q11.sql | |-- q12.sql | |-- q13.sql | |-- q14.sql | |-- q15.sql | |-- q16.sql | |-- q17.sql | |-- q18.sql | |-- q19.sql | |-- q1.sql | |-- q20.sql | |-- q21.sql | |-- q22.sql | |-- q2.sql | |-- q3.sql | |-- q4.sql | |-- q5.sql | |-- q6.sql | |-- q7.sql | |-- q8.sql | `-- q9.sql |-- run_hologres.sh |-- run_mysql.sh |-- run.sh `-- tpch_tools |-- dbgen |-- qgen `-- resouces |-- dists.dss `-- queries |-- 10.sql |-- 11.sql |-- 12.sql |-- 13.sql |-- 14.sql |-- 15.sql |-- 16.sql |-- 17.sql |-- 18.sql |-- 19.sql |-- 1.sql |-- 20.sql |-- 21.sql |-- 22.sql |-- 2.sql |-- 3.sql |-- 4.sql |-- 5.sql |-- 6.sql |-- 7.sql |-- 8.sql `-- 9.sql
-
Teste de consulta pontual por chave/valor
Este teste utiliza o banco de dados hologres_tpch e a tabela orders do OLAP query test.
-
Create a table
Consultas pontuais por chave/valor exigem uma tabela orientada a linhas. Conecte-se ao Hologres com um cliente psql e execute as seguintes instruções para criar a tabela
orders_row.NotaPara se conectar com um cliente psql, siga as etapas em Connect to Hologres for Development.
DROP TABLE IF EXISTS public.orders_row; BEGIN; CREATE TABLE public.orders_row( O_ORDERKEY BIGINT NOT NULL PRIMARY KEY ,O_CUSTKEY INT NOT NULL ,O_ORDERSTATUS TEXT NOT NULL ,O_TOTALPRICE DECIMAL(15,2) NOT NULL ,O_ORDERDATE TIMESTAMPTZ NOT NULL ,O_ORDERPRIORITY TEXT NOT NULL ,O_CLERK TEXT NOT NULL ,O_SHIPPRIORITY INT NOT NULL ,O_COMMENT TEXT NOT NULL ); CALL SET_TABLE_PROPERTY('public.orders_row', 'orientation', 'row'); CALL SET_TABLE_PROPERTY('public.orders_row', 'clustering_key', 'o_orderkey'); CALL SET_TABLE_PROPERTY('public.orders_row', 'distribution_key', 'o_orderkey'); COMMIT; -
Import data
Use a instrução INSERT INTO a seguir para importar dados da tabela
ordersdo dataset TPC-H para a tabelaorders_row.NotaO Hologres V2.1.17 e versões posteriores oferecem suporte ao Serverless Computing. Para cenários como importação offline em larga escala, grandes jobs de ETL e consultas de alto volume em tabelas externas, utilize o Serverless Computing para executar essas tarefas. Esse recurso emprega recursos serverless adicionais em vez dos recursos da sua instância, o que aumenta a estabilidade da instância e reduz a probabilidade de erros de out-of-memory (OOM). Não é necessário reservar recursos de computação extras para sua instância, e você paga apenas pelas tarefas executadas. Para mais informações sobre o Serverless Computing, consulte Serverless Computing. Para instruções sobre como usar o Serverless Computing, consulte Use Serverless Computing.
-- (Optional) It's recommended to use Serverless Computing for bulk offline imports and ETL jobs. SET hg_computing_resource = 'serverless'; INSERT INTO public.orders_row SELECT * FROM public.orders; -- Reset the configuration to ensure that subsequent SQL statements do not use serverless resources. RESET hg_computing_resource; -
Execute queries
-
Gere as instruções de consulta.
As consultas pontuais por chave/valor se dividem em dois tipos:
Tipo
Instrução de consulta
Descrição
consulta pontual de valor único
SELECT column_a ,column_b ,... ,column_x FROM table_x WHERE pk = value_x ;Essa consulta filtra por um único valor na cláusula
WHERE.consulta pontual de múltiplos valores
SELECT column_a ,column_b ,... ,column_x FROM table_x WHERE pk IN ( value_a, value_b,..., value_x ) ;Essa consulta filtra por múltiplos valores na cláusula
WHERE.Execute o seguinte script para gerar as instruções SQL necessárias.
rm -rf kv_query mkdir kv_query cd kv_query echo " \set column_values random(1,99999999) select O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT from public.orders_row WHERE o_orderkey =:column_values; " >> kv_query_single.sql echo " \set column_values1 random(1,99999999) \set column_values2 random(1,99999999) \set column_values3 random(1,99999999) \set column_values4 random(1,99999999) \set column_values5 random(1,99999999) \set column_values6 random(1,99999999) \set column_values7 random(1,99999999) \set column_values8 random(1,99999999) \set column_values9 random(1,99999999) select O_ORDERKEY,O_CUSTKEY,O_ORDERSTATUS,O_TOTALPRICE,O_ORDERDATE,O_ORDERPRIORITY,O_CLERK,O_SHIPPRIORITY,O_COMMENT from public.orders_row WHERE o_orderkey in(:column_values1,:column_values2,:column_values3,:column_values4,:column_values5,:column_values6,:column_values7,:column_values8,:column_values9); " >> kv_query_in.sqlO script gera dois arquivos SQL:
kv_query_single.sql: Contém o SQL para a consulta pontual de valor único.kv_query_in.sql: Contém o SQL para a consulta pontual de múltiplos valores. Esse script gera uma instrução SQL que filtra por nove valores aleatórios.
-
Para coletar estatísticas de consulta, utilize a ferramenta pgbench. Instale-a executando o seguinte comando.
yum install postgresql-contrib -yInstale o pgbench 13 ou versão posterior. Se o pgbench já estiver instalado, verifique se a versão é 9.6 ou posterior. Para verificar a versão:
pgbench --version -
Execute as instruções de teste.
NotaExecute os comandos a seguir no diretório onde as instruções de consulta foram geradas.
-
Para o cenário de consulta pontual de valor único, use o pgbench para executar um teste de estresse.
PGUSER=<AccessKey ID> PGPASSWORD=<AccessKey Secret> PGDATABASE=<database> pgbench -h <endpoint> -p <port> -c <Client_Num> -T <Query_Seconds> -M prepared -n -f kv_query_single.sql -
Para o cenário de consulta pontual de múltiplos valores, use o pgbench para executar um teste de estresse.
PGUSER=<AccessKey ID> PGPASSWORD=<AccessKey Secret> PGDATABASE=<database> pgbench -h <endpoint> -p <port> -c <Client_Num> -T <Query_Seconds> -M prepared -n -f kv_query_in.sql
Descrição dos parâmetros:
Parâmetro
Descrição
AccessKey ID
O AccessKey ID da sua conta da Alibaba Cloud.
Acesse a página AccessKey Management para obter o AccessKey ID.
AccessKey Secret
O AccessKey Secret da sua conta da Alibaba Cloud.
Acesse a página AccessKey Management para obter o AccessKey Secret.
database
-
O nome do banco de dados do Hologres.
-
Ao criar uma instância do Hologres, o sistema cria automaticamente um banco de dados chamado
postgres. -
É possível usar o banco de dados padrão
postgres, mas ele possui recursos limitados. Para cargas de trabalho em produção, Create a database.
endpoint
O endpoint da instância do Hologres.
Acesse a página de detalhes da instância no Hologres console e obtenha o endpoint na seção Network Information.
port
A porta da instância do Hologres.
Acesse a página Instance Details no Hologres console para obter a porta.
Client_Num
O número de clientes simultâneos.
Como este teste mede o desempenho de consulta e não a concorrência, defina este parâmetro como 1.
Query_Seconds
A duração total do teste de estresse, em segundos. Por exemplo, defina este parâmetro como 300.
-
-
Atualização de dados
Este cenário testa o desempenho do mecanismo OLAP em atualizações completas de linha disparadas por conflito de chave primária.
-
Gere o arquivo de consulta.
echo " \set O_ORDERKEY random(1,99999999) INSERT INTO public.orders_row(o_orderkey,o_custkey,o_orderstatus,o_totalprice,o_orderdate,o_orderpriority,o_clerk,o_shippriority,o_comment) VALUES (:O_ORDERKEY,1,'demo',1.1,'2021-01-01','demo','demo',1,'demo') on conflict(o_orderkey) do update set (o_orderkey,o_custkey,o_orderstatus,o_totalprice,o_orderdate,o_orderpriority,o_clerk,o_shippriority,o_comment)= ROW(excluded.*); " > /root/insert_on_conflict.sql -
Insira e atualize os dados. Parameters.
PGUSER=<AccessKey_ID> PGPASSWORD=<AccessKey_Secret> PGDATABASE=<Database> pgbench -h <Endpoint> -p <Port> -c <Client_Num> -T <Query_Seconds> -M prepared -n -f /root/insert_on_conflict.sql -
Exemplo de resultado:
transaction type: Custom query scaling factor: 1 query mode: prepared number of clients: 249 number of threads: 1 duration: 60 s number of transactions actually processed: 1923038 tps = 32005.850214 (including connections establishing) tps = 36403.145722 (excluding connections establishing)
Gravações em tempo real com o Flink
Este caso de uso avalia o desempenho de gravação de dados em tempo real.
-
DDL do Hologres
A tabela do Hologres utilizada neste caso possui 10 colunas, e a coluna
keyé a chave primária.DROP TABLE IF EXISTS flink_insert; BEGIN ; CREATE TABLE IF NOT EXISTS flink_insert( key INT PRIMARY KEY ,value1 TEXT ,value2 TEXT ,value3 TEXT ,value4 TEXT ,value5 TEXT ,value6 TEXT ,value7 TEXT ,value8 TEXT ,value9 TEXT ); CALL SET_TABLE_PROPERTY('flink_insert', 'orientation', 'row'); CALL SET_TABLE_PROPERTY('flink_insert', 'clustering_key', 'key'); CALL SET_TABLE_PROPERTY('flink_insert', 'distribution_key', 'key'); COMMIT; -
Script do job Flink
Este script usa o gerador de dados aleatórios integrado do Realtime Compute for Apache Flink para gravar dados no Hologres. Ele atualiza a linha inteira em caso de conflito de chave primária, e cada linha ultrapassa 512 B de tamanho.
CREATE TEMPORARY TABLE flink_case_1_source ( key INT, value1 VARCHAR, value2 VARCHAR, value3 VARCHAR, value4 VARCHAR, value5 VARCHAR, value6 VARCHAR, value7 VARCHAR, value8 VARCHAR, value9 VARCHAR ) WITH ( 'connector' = 'datagen', -- optional options -- 'rows-per-second' = '1000000000', 'fields.key.min'='1', 'fields.key.max'='2147483647', 'fields.value1.length' = '57', 'fields.value2.length' = '57', 'fields.value3.length' = '57', 'fields.value4.length' = '57', 'fields.value5.length' = '57', 'fields.value6.length' = '57', 'fields.value7.length' = '57', 'fields.value8.length' = '57', 'fields.value9.length' = '57' ); -- Create a Hologres sink table. CREATE TEMPORARY TABLE flink_case_1_sink ( key INT, value1 VARCHAR, value2 VARCHAR, value3 VARCHAR, value4 VARCHAR, value5 VARCHAR, value6 VARCHAR, value7 VARCHAR, value8 VARCHAR, value9 VARCHAR ) WITH ( 'connector' = 'hologres', 'dbname'='<yourDbname>', -- The name of the Hologres database. 'tablename'='<yourTablename>', -- The name of the Hologres table that receives the data. 'username'='<yourUsername>', -- Your AccessKey ID. 'password'='<yourPassword>', -- Your AccessKey Secret. 'endpoint'='<yourEndpoint>', -- The VPC endpoint of your Hologres instance. 'connectionSize' = '10', -- The default value is 3. 'jdbcWriteBatchSize' = '1024', -- The default value is 256. 'jdbcWriteBatchByteSize' = '2147483647', -- The default value is 20971520. 'mutatetype'='insertorreplace' -- Inserts a new row or replaces an existing row on conflict. ); -- Write data from the source to the sink. insert into flink_case_1_sink select key, value1, value2, value3, value4, value5, value6, value7, value8, value9 from flink_case_1_source ; -
Exemplo de resultado
No Hologres console, acesse a página Monitoring Information. No gráfico Query Latency (ms), a métrica select_avg atinge um pico de aproximadamente 400 ms. No gráfico Real-time Import RPS (records/second), a métrica sdk oscila entre 200.000 e 400.000 registros/segundo.
22 instruções de consulta TPC-H
Clique em uma consulta na tabela para visualize a instrução SQL.
|
Nome |
Consulta |
|||
|
As 22 instruções de consulta TPC-H |
||||
|
- |
- |
|||
-
Q1
select l_returnflag, l_linestatus, sum(l_quantity) as sum_qty, sum(l_extendedprice) as sum_base_price, sum(l_extendedprice * (1 - l_discount)) as sum_disc_price, sum(l_extendedprice * (1 - l_discount) * (1 + l_tax)) as sum_charge, avg(l_quantity) as avg_qty, avg(l_extendedprice) as avg_price, avg(l_discount) as avg_disc, count(*) as count_order from lineitem where l_shipdate <= date '1998-12-01' - interval '120' day group by l_returnflag, l_linestatus order by l_returnflag, l_linestatus; -
Q2
select s_acctbal, s_name, n_name, p_partkey, p_mfgr, s_address, s_phone, s_comment from part, supplier, partsupp, nation, region where p_partkey = ps_partkey and s_suppkey = ps_suppkey and p_size = 48 and p_type like '%STEEL' and s_nationkey = n_nationkey and n_regionkey = r_regionkey and r_name = 'EUROPE' and ps_supplycost = ( select min(ps_supplycost) from partsupp, supplier, nation, region where p_partkey = ps_partkey and s_suppkey = ps_suppkey and s_nationkey = n_nationkey and n_regionkey = r_regionkey and r_name = 'EUROPE' ) order by s_acctbal desc, n_name, s_name, p_partkey limit 100; -
Q3
select l_orderkey, sum(l_extendedprice * (1 - l_discount)) as revenue, o_orderdate, o_shippriority from customer, orders, lineitem where c_mktsegment = 'MACHINERY' and c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate < date '1995-03-23' and l_shipdate > date '1995-03-23' group by l_orderkey, o_orderdate, o_shippriority order by revenue desc, o_orderdate limit 10; -
Q4
select o_orderpriority, count(*) as order_count from orders where o_orderdate >= date '1996-07-01' and o_orderdate < date '1996-07-01' + interval '3' month and exists ( select * from lineitem where l_orderkey = o_orderkey and l_commitdate < l_receiptdate ) group by o_orderpriority order by o_orderpriority; -
Q5
select n_name, sum(l_extendedprice * (1 - l_discount)) as revenue from customer, orders, lineitem, supplier, nation, region where c_custkey = o_custkey and l_orderkey = o_orderkey and l_suppkey = s_suppkey and c_nationkey = s_nationkey and s_nationkey = n_nationkey and n_regionkey = r_regionkey and r_name = 'EUROPE' and o_orderdate >= date '1996-01-01' and o_orderdate < date '1996-01-01' + interval '1' year group by n_name order by revenue desc; -
Q6
select sum(l_extendedprice * l_discount) as revenue from lineitem where l_shipdate >= date '1996-01-01' and l_shipdate < date '1996-01-01' + interval '1' year and l_discount between 0.02 - 0.01 and 0.02 + 0.01 and l_quantity < 24; -
Q7
select supp_nation, cust_nation, l_year, sum(volume) as revenue from ( select n1.n_name as supp_nation, n2.n_name as cust_nation, extract(year from l_shipdate) as l_year, l_extendedprice * (1 - l_discount) as volume from supplier, lineitem, orders, customer, nation n1, nation n2 where s_suppkey = l_suppkey and o_orderkey = l_orderkey and c_custkey = o_custkey and s_nationkey = n1.n_nationkey and c_nationkey = n2.n_nationkey and ( (n1.n_name = 'CANADA' and n2.n_name = 'BRAZIL') or (n1.n_name = 'BRAZIL' and n2.n_name = 'CANADA') ) and l_shipdate between date '1995-01-01' and date '1996-12-31' ) as shipping group by supp_nation, cust_nation, l_year order by supp_nation, cust_nation, l_year; -
Q8
select o_year, sum(case when nation = 'BRAZIL' then volume else 0 end) / sum(volume) as mkt_share from ( select extract(year from o_orderdate) as o_year, l_extendedprice * (1 - l_discount) as volume, n2.n_name as nation from part, supplier, lineitem, orders, customer, nation n1, nation n2, region where p_partkey = l_partkey and s_suppkey = l_suppkey and l_orderkey = o_orderkey and o_custkey = c_custkey and c_nationkey = n1.n_nationkey and n1.n_regionkey = r_regionkey and r_name = 'AMERICA' and s_nationkey = n2.n_nationkey and o_orderdate between date '1995-01-01' and date '1996-12-31' and p_type = 'LARGE ANODIZED COPPER' ) as all_nations group by o_year order by o_year; -
Q9
select nation, o_year, sum(amount) as sum_profit from ( select n_name as nation, extract(year from o_orderdate) as o_year, l_extendedprice * (1 - l_discount) - ps_supplycost * l_quantity as amount from part, supplier, lineitem, partsupp, orders, nation where s_suppkey = l_suppkey and ps_suppkey = l_suppkey and ps_partkey = l_partkey and p_partkey = l_partkey and o_orderkey = l_orderkey and s_nationkey = n_nationkey and p_name like '%maroon%' ) as profit group by nation, o_year order by nation, o_year desc; -
Q10
select c_custkey, c_name, sum(l_extendedprice * (1 - l_discount)) as revenue, c_acctbal, n_name, c_address, c_phone, c_comment from customer, orders, lineitem, nation where c_custkey = o_custkey and l_orderkey = o_orderkey and o_orderdate >= date '1993-02-01' and o_orderdate < date '1993-02-01' + interval '3' month and l_returnflag = 'R' and c_nationkey = n_nationkey group by c_custkey, c_name, c_acctbal, c_phone, n_name, c_address, c_comment order by revenue desc limit 20; -
Q11
select ps_partkey, sum(ps_supplycost * ps_availqty) as value from partsupp, supplier, nation where ps_suppkey = s_suppkey and s_nationkey = n_nationkey and n_name = 'EGYPT' group by ps_partkey having sum(ps_supplycost * ps_availqty) > ( select sum(ps_supplycost * ps_availqty) * 0.0001000000 from partsupp, supplier, nation where ps_suppkey = s_suppkey and s_nationkey = n_nationkey and n_name = 'EGYPT' ) order by value desc; -
Q12
select l_shipmode, sum(case when o_orderpriority = '1-URGENT' or o_orderpriority = '2-HIGH' then 1 else 0 end) as high_line_count, sum(case when o_orderpriority <> '1-URGENT' and o_orderpriority <> '2-HIGH' then 1 else 0 end) as low_line_count from orders, lineitem where o_orderkey = l_orderkey and l_shipmode in ('FOB', 'AIR') and l_commitdate < l_receiptdate and l_shipdate < l_commitdate and l_receiptdate >= date '1997-01-01' and l_receiptdate < date '1997-01-01' + interval '1' year group by l_shipmode order by l_shipmode; -
Q13
select c_count, count(*) as custdist from ( select c_custkey, count(o_orderkey) as c_count from customer left outer join orders on c_custkey = o_custkey and o_comment not like '%special%deposits%' group by c_custkey ) c_orders group by c_count order by custdist desc, c_count desc; -
Q14
select 100.00 * sum(case when p_type like 'PROMO%' then l_extendedprice * (1 - l_discount) else 0 end) / sum(l_extendedprice * (1 - l_discount)) as promo_revenue from lineitem, part where l_partkey = p_partkey and l_shipdate >= date '1997-06-01' and l_shipdate < date '1997-06-01' + interval '1' month; -
Q15
with revenue0(SUPPLIER_NO, TOTAL_REVENUE) as ( select l_suppkey, sum(l_extendedprice * (1 - l_discount)) from lineitem where l_shipdate >= date '1995-02-01' and l_shipdate < date '1995-02-01' + interval '3' month group by l_suppkey ) select s_suppkey, s_name, s_address, s_phone, total_revenue from supplier, revenue0 where s_suppkey = supplier_no and total_revenue = ( select max(total_revenue) from revenue0 ) order by s_suppkey; -
Q16
select p_brand, p_type, p_size, count(distinct ps_suppkey) as supplier_cnt from partsupp, part where p_partkey = ps_partkey and p_brand <> 'Brand#45' and p_type not like 'SMALL ANODIZED%' and p_size in (47, 15, 37, 30, 46, 16, 18, 6) and ps_suppkey not in ( select s_suppkey from supplier where s_comment like '%Customer%Complaints%' ) group by p_brand, p_type, p_size order by supplier_cnt desc, p_brand, p_type, p_size; -
Q17
select sum(l_extendedprice) / 7.0 as avg_yearly from lineitem, part where p_partkey = l_partkey and p_brand = 'Brand#51' and p_container = 'WRAP PACK' and l_quantity < ( select 0.2 * avg(l_quantity) from lineitem where l_partkey = p_partkey ); -
Q18
select c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice, sum(l_quantity) from customer, orders, lineitem where o_orderkey in ( select l_orderkey from lineitem group by l_orderkey having sum(l_quantity) > 312 ) and c_custkey = o_custkey and o_orderkey = l_orderkey group by c_name, c_custkey, o_orderkey, o_orderdate, o_totalprice order by o_totalprice desc, o_orderdate limit 100; -
Q19
select sum(l_extendedprice* (1 - l_discount)) as revenue from lineitem, part where ( p_partkey = l_partkey and p_brand = 'Brand#52' and p_container in ('SM CASE', 'SM BOX', 'SM PACK', 'SM PKG') and l_quantity >= 3 and l_quantity <= 3 + 10 and p_size between 1 and 5 and l_shipmode in ('AIR', 'AIR REG') and l_shipinstruct = 'DELIVER IN PERSON' ) or ( p_partkey = l_partkey and p_brand = 'Brand#43' and p_container in ('MED BAG', 'MED BOX', 'MED PKG', 'MED PACK') and l_quantity >= 12 and l_quantity <= 12 + 10 and p_size between 1 and 10 and l_shipmode in ('AIR', 'AIR REG') and l_shipinstruct = 'DELIVER IN PERSON' ) or ( p_partkey = l_partkey and p_brand = 'Brand#52' and p_container in ('LG CASE', 'LG BOX', 'LG PACK', 'LG PKG') and l_quantity >= 21 and l_quantity <= 21 + 10 and p_size between 1 and 15 and l_shipmode in ('AIR', 'AIR REG') and l_shipinstruct = 'DELIVER IN PERSON' ); -
Q20
select s_name, s_address from supplier, nation where s_suppkey in ( select ps_suppkey from partsupp where ps_partkey in ( select p_partkey from part where p_name like 'drab%' ) and ps_availqty > ( select 0.5 * sum(l_quantity) from lineitem where l_partkey = ps_partkey and l_suppkey = ps_suppkey and l_shipdate >= date '1996-01-01' and l_shipdate < date '1996-01-01' + interval '1' year ) ) and s_nationkey = n_nationkey and n_name = 'KENYA' order by s_name; -
Q21
select s_name, count(*) as numwait from supplier, lineitem l1, orders, nation where s_suppkey = l1.l_suppkey and o_orderkey = l1.l_orderkey and o_orderstatus = 'F' and l1.l_receiptdate > l1.l_commitdate and exists ( select * from lineitem l2 where l2.l_orderkey = l1.l_orderkey and l2.l_suppkey <> l1.l_suppkey ) and not exists ( select * from lineitem l3 where l3.l_orderkey = l1.l_orderkey and l3.l_suppkey <> l1.l_suppkey and l3.l_receiptdate > l3.l_commitdate ) and s_nationkey = n_nationkey and n_name = 'PERU' group by s_name order by numwait desc, s_name limit 100; -
Q22
select cntrycode, count(*) as numcust, sum(c_acctbal) as totacctbal from ( select substring(c_phone from 1 for 2) as cntrycode, c_acctbal from customer where substring(c_phone from 1 for 2) in ('24', '32', '17', '18', '12', '14', '22') and c_acctbal > ( select avg(c_acctbal) from customer where c_acctbal > 0.00 and substring(c_phone from 1 for 2) in ('24', '32', '17', '18', '12', '14', '22') ) and not exists ( select * from orders where o_custkey = c_custkey ) ) as custsale group by cntrycode order by cntrycode;