Na era orientada por dados, os dados não estruturados (como texto, imagens, áudio, vídeo e logs), combinados com dados estruturados e semiestruturados (como JSON), formam os principais ativos de dados de uma empresa. Em suas formas brutas e diversas, esses dados contêm diversos insights de negócios, incluindo feedback de usuários, termos contratuais e imagens de defeitos de produtos. Este guia utiliza um cenário financeiro simulado para demonstrar como recuperar e analisar informações de documentos PDF, como prospectos e contratos, para auxiliar na tomada de decisões operacionais fundamentadas.
Capacidades principais
Esta prática recomendada demonstra como processar e recuperar dados não estruturados de arquivos PDF usando as seguintes capacidades principais do Hologres:
Object Table: Permite ler dados não estruturados, como arquivos PDF, imagens e apresentações, diretamente do OSS em formato tabular.
-
AI function: Possibilita o uso de SQL padrão para chamar funções de IA baseadas em grandes modelos de linguagem (LLMs) integrados e criar serviços de IA.
Transformação de dados: Oferece operadores de embedding e chunking para converter dados não estruturados em um formato estruturado e armazenável. Crie embeddings automaticamente sem depender de algoritmos externos.
Recuperação e análise de dados: Disponibiliza operadores como
ai_geneai_summarizepara executar inferência, resumo e tradução de dados via SQL.
Dynamic Table: Suporta atualização incremental para processar dados não estruturados automaticamente. Essa abordagem processa apenas dados incrementais, reduzindo computações redundantes e o consumo de recursos.
Busca vetorial: Permite realizar buscas por similaridade e reconhecimento de cenas em dados não estruturados usando SQL padrão. Combine predicados vetoriais e escalares na mesma consulta.
Busca de texto completo: Utiliza mecanismos como índice invertido e tokenização para recuperar dados não estruturados com eficiência. Suporta diversos métodos flexíveis de pesquisa, incluindo correspondência de palavras-chave e busca por frases.
Benefícios
Com essas capacidades principais, o Hologres oferece os seguintes benefícios para recuperação e análise de IA multimodal:
Pipeline de dados de IA ponta a ponta: Cobre todo o fluxo de trabalho, desde o embedding e chunking de dados até a transformação incremental, recuperação e análise. Isso permite que desenvolvedores criem aplicações de IA com a mesma facilidade de um sistema típico de big data.
Processamento e análise de dados não estruturados com SQL padrão: Extraia e transforme dados não estruturados usando apenas SQL, eliminando a necessidade de linguagens de programação especializadas ou sistemas externos. Essa abordagem simplifica o processamento de dados e reduz a curva de aprendizado para desenvolvedores.
Recuperação precisa, flexível e inteligente: Construa pipelines de busca híbrida que combinam pesquisas por palavra-chave, semântica e multimodal para cobrir uma ampla gama de cenários, desde consultas precisas até a compreensão de intenção. Integre funções de IA para entender profundamente a intenção do usuário, associações semânticas e inferência contextual, permitindo uma recuperação mais inteligente.
Análise segura e local: Analise os dados onde eles residem (análise in-place) sem exportá-los para sistemas externos. Essa abordagem se integra perfeitamente aos recursos de segurança do Hologres para proteger seus dados de forma eficaz.
Este guia demonstra como usar essas capacidades principais no Hologres para processar e recuperar dados não estruturados, ajudando você a construir uma plataforma de dados de IA multimodal de nível empresarial, eliminar silos de dados e liberar todo o valor das suas informações.
Fluxo de trabalho
O fluxo de trabalho é o seguinte:
-
Prepare o conjunto de dados.
Faça upload dos arquivos PDF do conjunto de dados financeiros para o OSS.
-
Transforme os dados do PDF.
Use uma Object Table para ler os metadados dos arquivos PDF. Em seguida, crie uma Dynamic Table com atualização incremental para realizar embedding e chunking nos dados. Crie um índice vetorial e um índice de texto completo na Dynamic Table para acelerar as buscas subsequentes.
Utilize o operador
ai_embedpara criar um embedding de uma pergunta em linguagem natural. Depois, use a busca híbrida para recuperar e classificar resultados de buscas de texto completo e vetoriais. Por fim, um LLM usa sua capacidade de inferência para gerar a resposta mais relevante.
Antes de começar
-
Preparação de dados
Este tópico utiliza 80 documentos de prospectos corporativos da pasta PDF no conjunto de dados financeiros público no ModelScope.
-
Preparação do ambiente
Crie uma instância do Hologres V4.0 ou posterior e crie um banco de dados.
-
Este tópico usa como exemplo um único nó large-96core-512GB-384GB.
-
Implante modelos. Os seguintes modelos são implantados com as alocações de recursos especificadas:
Parâmetro
Categoria
Descrição
vCPUs
Memória
GPU
Réplicas
to_doc
ds4sd/docling-models
Converte arquivos PDF em documentos.
20
100 GB
1 card (48 GB)
1
chunk
recursive-character-text-splitter
Divide arquivos PDF grandes em chunks.
15
30 GB
0 cards (0 GB)
1
pdf_embed
BAAI/bge-base-zh-v1.5
Cria embeddings de documentos.
7
30 GB
1 card (96 GB)
1
llm
Qwen/Qwen3-32B
Usa um LLM para realizar inferência no conteúdo do documento recuperado com base em um prompt.
7
30 GB
1 card (96 GB)
1
NotaOs recursos para esses modelos são alocados por padrão.
Procedimento
-
Baixe os arquivos PDF e faça upload deles para o OSS.
Baixe os 80 documentos de prospecto (PDFs) do Bosera-JM 14B Challenge Dataset for Finance.
Faça login no console do OSS, crie um bucket e envie os arquivos PDF baixados para o bucket. Para mais informações sobre a operação de upload, consulte Upload simples.
-
Conceda permissões às contas.
-
Faça login no console do Resource Access Management (RAM). Crie uma função RAM do Alibaba Cloud e conceda a ela permissões de OSS.
Recomendamos conceder a permissão AliyunOSSReadOnlyAccess.
-
Conceda permissões de login e acesso ao Hologres para a função RAM.
-
Conta Alibaba Cloud (conta raiz)
Atualize a política de confiança da função RAM. Atualize estes parâmetros:
Action: Defina como
sts:AssumeRole.Service: Defina como
hologres.aliyuncs.com.
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "RAM": [ "acs:ram::1866xxxx:root" ], "Service": [ "hologres.aliyuncs.com" ] } } ], "Version": "1" } -
Usuário RAM (subconta)
-
Conceda permissões ao usuário RAM.
-
Na página , clique em Create Policy e selecione o modo Script Editor para criar uma política. Para mais informações, consulte Criar uma política personalizada.
O Hologres usa essa política para verificar se o usuário RAM tem permissão para criar a função RAM. Conteúdo da política:
{ "Version": "1", "Statement": [ { "Effect": "Allow", "Action": "hologram:GrantAssumeRole", "Resource": "<RoleARN>" } ] } Na página , clique em Add Permissions na coluna Actions
-
-
Conceda permissões à função RAM.
Atualize a política de confiança da função RAM. Atualize estes parâmetros:
Action: Defina como
sts:AssumeRole.Service: Defina como
hologres.aliyuncs.com.
{ "Statement": [ { "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "RAM": [ "acs:ram::1866xxxx:root" ], "Service": [ "hologres.aliyuncs.com" ] } } ], "Version": "1" }
-
-
-
-
Execute embedding e chunking nos arquivos PDF.
Crie uma Object Table e uma Dynamic Table para ler e processar os metadados do PDF. Para simplificar esse processo de múltiplas etapas, o Hologres fornece um procedimento armazenado com os seguintes recursos:
Cria uma Object Table para recuperar os metadados do PDF.
Cria uma Dynamic Table de destino no modo de atualização incremental para armazenar os dados processados. Um índice vetorial e um índice de texto completo são definidos nesta tabela. A atualização automática fica desativada, exigindo atualização manual da tabela.
Durante a atualização da Dynamic Table, os dados passam por embedding e chunking usando os operadores
ai_embedeai_chunk.
O código a seguir representa o procedimento armazenado:
CALL create_rag_corpus_from_oss( oss_path => 'oss://xxxx/bs_challenge_financial_14b_dataset/pdf', oss_endpoint => 'oss-cn-hangzhou-internal.aliyuncs.com', oss_role_arn => 'acs:ram::186xxxx:role/xxxx', corpus_table => 'public.dt_bs_challenge_financial' ); -
Atualize a tabela de resultados.
Atualize manualmente a Object Table e a Dynamic Table criadas pelo procedimento armazenado para concluir a transformação dos dados. Esta etapa está encapsulada no procedimento armazenado para processamento de PDF, que executa as seguintes ações:
Atualiza a Object Table para obter os metadados do PDF.
Atualiza a Dynamic Table para realizar o embedding e chunking do PDF.
Use o seguinte código para chamar o procedimento armazenado:
CALL refresh_rag_corpus_table( corpus_table => 'public.dt_bs_challenge_financial' ); -
Recupere dados do PDF.
Após o processamento dos dados, recupere-os usando métodos como busca vetorial e busca de texto completo, conforme seu cenário de negócios. Por exemplo, consulte a tendência de desempenho de uma empresa em seu prospecto para determinar se a perspectiva futura é pessimista ou otimista, o que pode fundamentar decisões de investimento subsequentes.
Busca vetorial
Para simplificar a busca vetorial, o Hologres encapsula processos como embedding de perguntas, construção de prompts e geração de respostas baseadas em LLM em uma função de busca vetorial. Chame essa função diretamente para realizar a recuperação vetorial.
-- Vector-only retrieval + AI reranking SELECT qa_vector_search_retrieval( question => 'By what percentage did the operating income and net profit of Goke Microelectronics increase year-over-year in 2014, 2015, and 2016 during the reporting period?', corpus_table => 'dt_bs_challenge_financial', prompt => 'Please analyze whether the following performance trend is pessimistic or optimistic and provide reasons: ${question}\n\n Reference information:\n\n ${context}' )O seguinte resultado é retornado:
qa_retrieval --------- "Based on the information provided, the analysis of the performance trend of Goke Microelectronics leads to the following conclusions: ### I. Performance Trend Analysis: Pessimistic #### 1. **Sluggish revenue growth** - Revenue in 2014 increased by **15.13%** year-over-year, but in 2015, it **decreased by 5.21%**. Data for 2016 is not provided, but it is clear that the revenue growth trend saw a significant decline in 2015. - The compound annual growth rate (CAGR) of revenue from 2012 to 2014 was only **4.47%**, indicating slow business expansion. #### 2. **Continuous decline in net profit growth** - Net profit grew by **5.43%** in 2014 but **decreased by 3.29%** in 2015. - After non-recurring gains and losses are deducted, the net profit attributable to parent company shareholders decreased by **3.14%** in 2014 and further reduced by **5.60%** in 2015. This indicates that the profitability of the company's main business is continuously deteriorating. - The CAGR of net profit after non-recurring gains and losses are deducted from 2012 to 2014 was **-4.38%**. This is much lower than the revenue growth, indicating that the main business is not profitable and growth relies on non-recurring gains and losses. #### 3. **High proportion of non-recurring gains and losses** - During the reporting period, non-recurring gains and losses accounted for a high proportion of net profit: **17.54%**, **10.25%**, and **8.06%** in 2014, 2013, and 2012, respectively. This shows that a portion of the company's profit comes from non-recurring factors such as policy support and government subsidies, rather than from the sustained growth of its core business. - Relying on non-recurring gains and losses to maintain profit growth is not conducive to the company's long-term stable development. #### 4. **Decline in return on equity (ROE)** - The weighted average ROE increased from **18.10%** in 2014 to **24.82%** in 2015, and then to **28.23%** in 2016. Although the data seems to show growth, this indicator is calculated based on net profit after non-recurring gains and losses are deducted, and the net profit itself is declining. Therefore, this growth may be related to changes in the capital structure rather than a substantial improvement in profitability. ### II. Summary 1. **Sluggish main business growth**: Both revenue and net profit growth show a downward trend, especially the decline in net profit, which indicates weakening profitability. 2. **High reliance on non-recurring gains and losses**: Non-recurring gains and losses account for a high proportion of the company's profit, which indicates that the main business is not sufficiently profitable and the sustainability of the company's performance is questionable. 3. **Intense market competition**: The market for industrial PCs, displays, and power supplies that the company purchases is highly competitive, with stable prices and squeezed profit margins. 4. **Industry environment impact**: Fluctuations in stainless steel market prices and raw material prices may have a certain impact on the company's operating performance. Although the company has taken measures to reduce the impact, this still requires attention in the long run. ### III. Conclusion Overall, the performance trend of Goke Microelectronics is **pessimistic**. The company's main business growth is sluggish, net profit is continuously declining, and it relies heavily on non-recurring gains and losses. The sustainability of its future profitability is questionable. The company needs to strengthen the competitiveness of its core business, optimize its cost structure, and improve the profitability of its main business to achieve long-term stable development."Busca de texto completo
Para facilitar o uso na busca de texto completo, o Hologres encapsula processos como embedding de perguntas, construção de prompts e geração de respostas por um LLM em uma função de busca de texto completo. Chame essa função diretamente para realizar a recuperação de texto completo:
-- Full-text search retrieval SELECT qa_text_search_retrieval( question => 'By what percentage did the operating income and net profit of Goke Microelectronics increase year-over-year in 2014, 2015, and 2016 during the reporting period?', corpus_table => 'dt_bs_challenge_financial', prompt => 'Please analyze whether the following performance trend is pessimistic or optimistic and provide reasons: ${question}\n\n Reference information:\n\n ${context}' );O seguinte resultado é retornado:
qa_text_search_retrieval ---------------- "Based on the information provided, the overall performance trend of Goke Microelectronics in 2014, 2015, and 2016 is **pessimistic**. The specific reasons are as follows: ### 1. **Sluggish revenue growth** - The revenue growth rate in 2014 was **15.13%**, but in 2015, it turned to **-5.21%**, which indicates negative growth. - The compound annual growth rate (CAGR) of revenue from 2012 to 2014 was only **4.47%**, which indicates that the company's revenue growth was slow and its business development was not strong. - The revenue forecast for the first half of 2015 was nearly the same as that for the same period in 2014, but the net profit for the first half of 2015 **slightly decreased** compared with the same period of the previous year, which indicates a decline in profitability. ### 2. **Poor growth in net profit and net profit excluding non-recurring items** - The net profit growth rate in 2014 was **5.43%**, but it decreased to **-3.29%** in 2015, which means net profit declined. - The growth rate of net profit after non-recurring gains and losses are deducted was **-3.14%** in 2014 and further decreased to **-5.60%** in 2015, which indicates that the profitability of the company's main business continued to decline. - The CAGR of net profit excluding non-recurring items from 2012 to 2014 was **-4.38%**, which is significantly lower than the CAGR of revenue. This indicates that the company's profit quality is not high and its main business profitability is weak. ### 3. **Fluctuations in cash flow from operating activities** - In 2014, the proportion of cash received from the sale of goods and provision of services to operating income decreased compared with the previous two years. This was mainly due to the **cross-period collection** of some revenue-recognized items, which indicates problems in the company's cash flow management. - In 2013, the proportion of cash paid for the purchase of goods and services to operating costs was relatively high. This was mainly because **raw materials were purchased and production was completed** in that year, but some costs were not carried over until 2014, which resulted in a lower proportion in 2014. This reflects that the company's procurement and production pace was not stable. ### 4. **Investment and profitability indicators** - The weighted average return on equity (ROE) was **18.10%** in 2014, rose to **24.82%** in 2015, and further increased to **28.23%** in 2016. Although there was an improvement, the increase in ROE may have been mainly attributed to **financial leverage** rather than an improvement in core business profitability. - Considering the continuous decline in net profit and net profit excluding non-recurring items, the increase in ROE does not fully reflect an improvement in the company's operational quality. ### 5. **Performance forecast for the first half of 2015** - The estimated operating revenue for the first half of 2015 is **CNY 85.05 million to CNY 103.95 million**, which is similar to the **CNY 101.2735 million** for the same period in 2014. However, the estimated net profit is **CNY 23.40 million to CNY 28.60 million**, which is lower than the **CNY 29.1266 million** for the same period in 2014. This indicates a decline in the company's profitability. ### Summary In summary, the performance trend of Goke Microelectronics from 2014 to 2016 is **pessimistic**. Although ROE has improved, the sluggish revenue growth, continuous decline in net profit and net profit excluding non-recurring items, and large fluctuations in cash flow from operational activities indicate that the company's main business profitability is weak and its operational quality needs to be improved."Busca híbrida
Para cenários de busca híbrida que combinam busca vetorial e de texto completo com classificação, o Hologres fornece uma função de busca híbrida com classificação. Esta função possui os seguintes recursos:
Recupera as 20 principais respostas com base em uma busca vetorial para a pergunta.
Recupera as 20 principais respostas com base em uma busca de texto completo para a pergunta.
Usa o operador
ai_rankpara reclassificar os resultados combinados de ambas as buscas.Utiliza o operador
ai_gene um LLM para gerar uma resposta final com base em um prompt e nos resultados mais bem classificados.
-- Hybrid search (full-text and vector) + AI reranking SELECT qa_hybrid_retrieval( question => 'By what percentage did the operating income and net profit of Goke Microelectronics increase year-over-year in 2014, 2015, and 2016 during the reporting period?', corpus_table => 'dt_bs_challenge_financial', prompt => 'Please analyze whether the following performance trend is pessimistic or optimistic and provide reasons: ${question}\n\n Reference information:\n\n ${context}' );O seguinte resultado é retornado:
qa_hybrid_retrieval --- "Based on the information provided, we can analyze the performance trend of Goke Microelectronics and determine whether it is pessimistic or optimistic as follows: --- ### I. **Operating Revenue Trend Analysis** 1. **2012-2014 Compound Annual Growth Rate (CAGR)**: - The CAGR of operating income was **4.47%**, which indicates that the company's revenue growth was relatively stable. - In 2014, operating income was **CNY 181.5406 million**, an increase of **15.13%** from 2013. - In 2015, operating income **decreased by 5.21%** year-over-year, which indicates negative growth. 2. **Conclusion**: - Revenue growth recovered in 2014 but saw a significant decline in 2015, which indicates that the company's business expansion has encountered some resistance. --- ### II. **Net Profit Trend Analysis** 1. **2012-2014 CAGR**: - The CAGR of net profit after non-recurring gains and losses are deducted was **-4.38%**, which is lower than the CAGR of operating income. This indicates a decline in the company's profitability. - In 2014, the net profit excluding non-recurring items was **CNY 42,731,071.18**, a decrease of **3.14%** from 2013. - In 2015, the net profit excluding non-recurring items further **decreased by 5.60%** year-over-year. 2. **Impact of Non-recurring Gains and Losses**: - The proportion of non-recurring gains and losses to net profit was **17.54%**, **10.25%**, and **8.06%** in 2014, 2013, and 2012, respectively, which indicates an upward trend. - The increase in non-recurring gains and losses mainly came from government subsidies and income from wealth management products, rather than sustained growth from the main business. 3. **Conclusion**: - The continuous two-year decline in net profit excluding non-recurring items indicates that the profitability of the company's main business is weakening, and performance growth relies on non-recurring gains and losses. This is a worrying signal. --- ### III. **Cash Flow and Operational Stability** 1. **Cash Flow from Operating Activities**: - In 2014, operating income was **CNY 181.5406 million**, but the cash received from the sale of goods and provision of services was not clearly stated, which makes it impossible to determine whether the cash flow is healthy. - During the reporting period, the company's bank deposits were **CNY 130.6338 million**, **CNY 41.5254 million**, and **CNY 98.6461 million**, respectively. The liquidity fluctuated significantly, but the main customers, suppliers, and business model remained stable. 2. **Conclusion**: - Although the company's cash flow fluctuates, its stable customer and supplier base and business model provide a certain guarantee for future development. --- ### IV. **Performance Forecast for the First Half of 2015** - From January to June 2015, the estimated operating income is **CNY 85.05 million to CNY 103.95 million**, a significant increase from the **CNY 46.4119 million** for the same period in 2014. - However, from January to March 2015, the net profit decreased by **48.26%** year-over-year, mainly because the projects with recognized revenue had lower gross profit margins. --- ### V. **Comprehensive Analysis and Judgment** 1. **Optimistic Factors**: - Operating income grew rapidly in 2014, reaching **15.13%**. - Operating income is expected to grow significantly in the first half of 2015, which indicates that the company may be gradually recovering. - The stability of major customers, suppliers, and the business model provides a good operational foundation for the company. 2. **Pessimistic Factors**: - In 2015, operating revenue **decreased by 5.21%** year-over-year, and net profit also declined. - The continuous two-year decline in net profit excluding non-recurring items indicates insufficient profitability of the main business. - The proportion of non-recurring gains and losses is increasing, and performance growth relies on government subsidies and income from wealth management products, which indicates a lack of internal growth momentum. - The net profit from January to March 2015 dropped sharply by **48.26%**, which indicates significant short-term performance fluctuations. --- ### **Final Conclusion: Overall Trend is Pessimistic** - Although the company's operating income recovered in 2014 and is expected to grow in the first half of 2015, the **continuous decline in net profit excluding non-recurring items**, **reliance on non-recurring gains and losses for net profit growth**, and **large short-term performance fluctuations** indicate that the company's current performance growth lacks sustainability and stability. - Therefore, from a long-term perspective, the company's performance trend is **pessimistic**. You need to pay attention to the improvement of its main business profitability and its reliance on non-recurring gains and losses. --- ### **Recommendations** 1. Pay attention to whether the profitability of the company's main business can be improved in the future. 2. Reduce reliance on non-recurring gains and losses and enhance internal growth momentum. 3. Stabilize customer and supplier relationships, optimize the business structure, and increase gross profit margins."Busca híbrida com RRF
Após recuperar resultados usando busca vetorial e de texto completo, classifique-os usando Reciprocal Rank Fusion (RRF). Para facilitar o uso, o Hologres fornece uma função de busca híbrida com classificação RRF (consulte o apêndice abaixo para a definição detalhada). Esta função executa as seguintes ações:
Recupera as 20 principais respostas com base em uma busca vetorial para a pergunta.
Recupera as 20 principais respostas com base em uma busca de texto completo para a pergunta.
Calcula pontuações RRF para os resultados de ambas as buscas e retorna os N principais resultados.
Utiliza o operador
ai_gene um LLM para gerar uma resposta final com base em um prompt e nos resultados mais bem classificados.
-- Hybrid search (full-text and vector) + RRF reranking SELECT qa_hybrid_retrieval_rrf( question => 'By what percentage did the operating income and net profit of Goke Microelectronics increase year-over-year in 2014, 2015, and 2016 during the reporting period?', corpus_table => 'dt_bs_challenge_financial', prompt => 'Please analyze whether the following performance trend is pessimistic or optimistic and provide reasons: ${question}\n\n Reference information:\n\n ${context}' );O seguinte resultado é retornado:
qa_hybrid_retrieval_rrf ------------------ "Based on the information provided, the analysis of the performance trend of Goke Microelectronics leads to the following conclusions: ### **Performance Trend Judgment: Pessimistic** #### **The reasons are as follows:** 1. **Net profit growth is lower than revenue growth:** - The provided information indicates that the company's **compound annual growth rate (CAGR) of operating income from 2012 to 2014 was 4.47%**, which suggests that the overall business growth was relatively stable. - However, the **CAGR of net profit attributable to parent company shareholders after non-recurring gains and losses are deducted was -4.38%**, which is significantly lower than the revenue growth rate. This indicates that while revenue was growing, the company's profitability did not improve in sync and even declined. This could be due to factors such as rising costs, declining gross profit margins, or a reduction in non-recurring gains and losses. 2. **Significant fluctuations in net profit:** - The net profit from January to March 2015 decreased by 48.26% year-over-year. The main reason was the lower gross profit margin of the projects with recognized revenue (for example, the Wuxi Metro Line 1 project was mainly based on module outsourcing). This indicates that the company's short-term performance is susceptible to changes in its business structure, which shows a certain degree of instability. 3. **Decline in gross profit margin and profitability:** - It was mentioned that the "contribution of main business profit to the company's net profit" slightly decreased in 2014 compared with 2013, and 2013 was lower than 2012. This suggests that the profitability of the company's core business may be weakening, possibly due to a combination of increased market competition, rising costs, or changes in the product structure. 4. **Projected profit decline in the first half of 2015:** - The estimated operating revenue for the first half of 2015 is from CNY 85.05 million to CNY 103.95 million, which is similar to that in the same period of 2014. However, the estimated net profit is from CNY 23.40 million to CNY 28.60 million, which is lower than the CNY 29.1266 million in the same period of 2014. This indicates that the company's profitability is further declining and it may be facing certain operational pressures. 5. **Sluggish business growth:** - Although revenue growth is stable, the decline in net profit indicates that the quality of the company's business growth is not high and has not been effectively converted into profit. This may affect investors' confidence in the company's future development. ### **Summary:** The overall performance trend of Goke Microelectronics is **pessimistic**. Although operating revenue has maintained stable growth, the growth of net profit has significantly lagged or even shown negative growth. This indicates that the company's profitability is declining, the quality of its business development is not high, and there is a risk of short-term performance fluctuations. If the company cannot effectively increase its gross profit margin, control costs, or optimize its product structure, its future performance may continue to be under pressure."
Apêndice: Definições de procedimentos
As definições dos procedimentos armazenados usados neste tópico são fornecidas para referência.
Os seguintes procedimentos armazenados e funções são fornecidos como exemplos. Talvez seja necessário modificá-los para seu ambiente específico.
Procedimento de processamento de PDF
-
Crie uma Object Table e uma Dynamic Table
CREATE OR REPLACE PROCEDURE create_rag_corpus_from_oss( oss_path TEXT, oss_endpoint TEXT, oss_role_arn TEXT, corpus_table TEXT, embedding_model TEXT DEFAULT NULL, parse_document_model TEXT DEFAULT NULL, chunk_model TEXT DEFAULT NULL, chunk_size INT DEFAULT 300, chunk_overlap INT DEFAULT 50, overwrite BOOLEAN DEFAULT FALSE ) AS $$ DECLARE corpus_schema TEXT; corpus_name TEXT; obj_table_name TEXT; full_corpus_ident TEXT; full_obj_ident TEXT; embed_expr TEXT; chunk_expr TEXT; parse_expr TEXT; embedding_dims INT; BEGIN -- 1. Split the schema and table names. IF position('.' in corpus_table) > 0 THEN corpus_schema := split_part(corpus_table, '.', 1); corpus_name := split_part(corpus_table, '.', 2); ELSE corpus_schema := 'public'; corpus_name := corpus_table; END IF; obj_table_name := corpus_name || '_obj_table'; full_corpus_ident := format('%I.%I', corpus_schema, corpus_name); full_obj_ident := format('%I.%I', corpus_schema, obj_table_name); -- 2. If overwrite is set to true, drop the existing tables and indexes first. IF overwrite THEN DECLARE dyn_table_exists BOOLEAN; rec RECORD; BEGIN -- Check whether the dynamic table exists. SELECT EXISTS ( SELECT 1 FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace WHERE c.relname = corpus_name AND n.nspname = corpus_schema ) INTO dyn_table_exists; IF dyn_table_exists THEN -- 2.1 Disable auto-refresh for the dynamic table. -- RAISE NOTICE 'Disabling auto refresh for %', full_corpus_ident; -- EXECUTE format('ALTER TABLE IF EXISTS %s SET (auto_refresh_enable=false)', full_corpus_ident); -- 2.2 Find and cancel RUNNING refresh jobs. FOR rec IN EXECUTE format( $f$ SELECT query_job_id FROM hologres.hg_dynamic_table_refresh_log(%L) WHERE status = 'RUNNING'; $f$, corpus_table ) LOOP RAISE NOTICE 'Found running refresh job: %', rec.query_job_id; IF hologres.hg_internal_cancel_query_job(rec.query_job_id::bigint) THEN RAISE NOTICE 'Job % canceled successfully.', rec.query_job_id; ELSE RAISE NOTICE 'Failed to cancel job %.', rec.query_job_id; END IF; END LOOP; -- 2.3 Drop the Dynamic Table. EXECUTE format('DROP TABLE IF EXISTS %s;', full_corpus_ident); ELSE RAISE NOTICE 'Dynamic table % does not exist, skip cancel job and drop.', full_corpus_ident; END IF; -- 2.4 Drop the Object Table regardless. EXECUTE format('DROP OBJECT TABLE IF EXISTS %s;', full_obj_ident); END; END IF; -- 3. Create an Object Table. RAISE NOTICE 'Creating object table: %', obj_table_name; EXECUTE format( $f$ CREATE OBJECT TABLE %s WITH ( path = %L, oss_endpoint = %L, role_arn = %L ); $f$, full_obj_ident, oss_path, oss_endpoint, oss_role_arn ); COMMIT; -- 4. Refresh the Object Table. RAISE NOTICE 'Refreshing object table: %', obj_table_name; EXECUTE format('REFRESH OBJECT TABLE %s;', full_obj_ident); COMMIT; -- 5. Select a document parsing model. IF parse_document_model IS NULL OR length(trim(parse_document_model)) = 0 THEN parse_expr := 'ai_parse_document(file, ''auto'', ''markdown'')'; ELSE parse_expr := format( 'ai_parse_document(%L, file, ''auto'', ''markdown'')', parse_document_model ); END IF; -- 6. Select a chunking model. IF chunk_model IS NULL OR length(trim(chunk_model)) = 0 THEN chunk_expr := format('ai_chunk(doc, %s, %s)', chunk_size, chunk_overlap); ELSE chunk_expr := format( 'ai_chunk(%L, doc, %s, %s)', chunk_model, chunk_size, chunk_overlap ); END IF; -- 7. Select an embedding model. IF embedding_model IS NULL OR length(trim(embedding_model)) = 0 THEN embed_expr := 'ai_embed(chunk)'; EXECUTE 'SELECT array_length(ai_embed(''dummy''), 1)' INTO embedding_dims; ELSE embed_expr := format('ai_embed(%L, chunk)', embedding_model); EXECUTE format( 'SELECT array_length(ai_embed(%L, ''dummy''), 1)', embedding_model ) INTO embedding_dims; END IF; RAISE NOTICE 'Embedding dimension: %', embedding_dims; -- 8. Create a Dynamic Table for RAG output. RAISE NOTICE 'Creating dynamic table: %', corpus_name; EXECUTE format( $f$ CREATE DYNAMIC TABLE %s( CHECK(array_ndims(embedding_vector) = 1 AND array_length(embedding_vector, 1) = %s) ) WITH ( vectors = '{ "embedding_vector": { "algorithm": "HGraph", "distance_method": "Cosine", "builder_params": { "base_quantization_type": "sq8_uniform", "max_degree": 64, "ef_construction": 400, "precise_quantization_type": "fp32", "use_reorder": true } } }', auto_refresh_mode = 'incremental', freshness = '5 minutes', auto_refresh_enable = 'false' ) AS WITH parsed_doc AS ( SELECT object_uri, etag, %s AS doc FROM %s ), chunked_doc AS ( SELECT object_uri, etag, unnest(%s) AS chunk FROM parsed_doc ) SELECT object_uri, etag, chunk, %s AS embedding_vector FROM chunked_doc; $f$, full_corpus_ident, embedding_dims, parse_expr, full_obj_ident, chunk_expr, embed_expr ); COMMIT; -- 9. Create a full-text index. The index name is table_name || '_fulltext_idx'. EXECUTE format( 'CREATE INDEX %I ON %s USING FULLTEXT (chunk);', corpus_name || '_fulltext_idx', full_corpus_ident ); RAISE NOTICE ''; RAISE NOTICE 'RAG corpus created successfully for table: %', corpus_table; RAISE NOTICE ' Vector index: %.embedding_vector', corpus_table; RAISE NOTICE ' Full-text index: %.chunk', corpus_table; END; $$ LANGUAGE plpgsql; -
Procedimento armazenado para atualizar a Object Table e a Dynamic Table
CREATE OR REPLACE PROCEDURE refresh_rag_corpus_table( corpus_table TEXT ) AS $$ DECLARE corpus_schema TEXT; corpus_name TEXT; obj_table_name TEXT; full_corpus_ident TEXT; full_obj_ident TEXT; BEGIN -- 1. Parse the schema and table names. IF position('.' in corpus_table) > 0 THEN corpus_schema := split_part(corpus_table, '.', 1); corpus_name := split_part(corpus_table, '.', 2); ELSE corpus_schema := 'public'; corpus_name := corpus_table; END IF; obj_table_name := corpus_name || '_obj_table'; full_corpus_ident := format('%I.%I', corpus_schema, corpus_name); full_obj_ident := format('%I.%I', corpus_schema, obj_table_name); -- 2. Refresh the Object Table. RAISE NOTICE 'Refreshing Object Table: %', obj_table_name; EXECUTE format('REFRESH OBJECT TABLE %s;', full_obj_ident); -- 3. Refresh the Dynamic Table. RAISE NOTICE 'Refreshing Dynamic Table: %', corpus_name; EXECUTE format('REFRESH TABLE %s;', full_corpus_ident); RAISE NOTICE 'Refresh complete for corpus table %', corpus_table; END; $$ LANGUAGE plpgsql; -
Procedimento armazenado para excluir a Object Table e a Dynamic Table
CREATE OR REPLACE PROCEDURE drop_rag_corpus_table( corpus_table TEXT ) AS $$ DECLARE corpus_schema TEXT; corpus_name TEXT; obj_table_name TEXT; full_corpus_ident TEXT; full_obj_ident TEXT; rec RECORD; BEGIN -- 1. Parse the schema and table names. IF position('.' in corpus_table) > 0 THEN corpus_schema := split_part(corpus_table, '.', 1); corpus_name := split_part(corpus_table, '.', 2); ELSE corpus_schema := 'public'; corpus_name := corpus_table; END IF; obj_table_name := corpus_name || '_obj_table'; full_corpus_ident := format('%I.%I', corpus_schema, corpus_name); full_obj_ident := format('%I.%I', corpus_schema, obj_table_name); -- 2. Drop the tables. -- 2.1 Disable auto-refresh for the dynamic table. -- RAISE NOTICE 'Disabling auto refresh for %', full_corpus_ident; -- EXECUTE format('ALTER TABLE IF EXISTS %s SET (auto_refresh_enable=false)', full_corpus_ident); -- 2.2 Find and cancel RUNNING refresh jobs. FOR rec IN EXECUTE format( $f$ SELECT query_job_id FROM hologres.hg_dynamic_table_refresh_log(%L) WHERE status = 'RUNNING'; $f$, corpus_table ) LOOP RAISE NOTICE 'Found running refresh job: %', rec.query_job_id; IF hologres.hg_internal_cancel_query_job(rec.query_job_id::bigint) THEN RAISE NOTICE 'Job % canceled successfully.', rec.query_job_id; ELSE RAISE NOTICE 'Failed to cancel job %.', rec.query_job_id; END IF; END LOOP; -- 2.3 Drop the Dynamic Table. RAISE NOTICE 'Dropping Dynamic Table: %', corpus_name; EXECUTE format('DROP TABLE IF EXISTS %s;', full_corpus_ident); -- 2.4 Drop the Object Table. RAISE NOTICE 'Dropping Object Table: %', obj_table_name; EXECUTE format('DROP OBJECT TABLE IF EXISTS %s;', full_obj_ident); RAISE NOTICE 'Drop complete for corpus: %', corpus_table; END; $$ LANGUAGE plpgsql;
Função de busca vetorial
-- RAG Q&A with vector-only retrieval
CREATE OR REPLACE FUNCTION qa_vector_search_retrieval(
question TEXT,
corpus_table TEXT,
embedding_model TEXT DEFAULT NULL,
llm_model TEXT DEFAULT NULL,
ranking_model TEXT DEFAULT NULL,
prompt TEXT DEFAULT 'Please answer the following question in ${language} based on the reference information.\n\n Question: ${question}\n\n Reference information:\n\n ${context}',
language TEXT DEFAULT 'Chinese',
vector_recall_count INT DEFAULT 20,
rerank_recall_count INT DEFAULT 5,
vector_col TEXT DEFAULT 'embedding_vector'
)
RETURNS TEXT AS
$$
DECLARE
final_answer TEXT;
sql TEXT;
embedding_expr TEXT;
ai_rank_expr TEXT;
ai_gen_expr TEXT;
embedding_model_valid BOOLEAN;
llm_model_valid BOOLEAN;
ranking_model_valid BOOLEAN;
BEGIN
embedding_model_valid := (embedding_model IS NOT NULL AND trim(embedding_model) != '');
llm_model_valid := (llm_model IS NOT NULL AND trim(llm_model) != '');
ranking_model_valid := (ranking_model IS NOT NULL AND trim(ranking_model) != '');
IF embedding_model_valid THEN
embedding_expr := 'ai_embed(' || quote_literal(embedding_model) || ', ' || quote_literal(question) || ')';
ELSE
embedding_expr := 'ai_embed(' || quote_literal(question) || ')';
END IF;
IF ranking_model_valid THEN
ai_rank_expr := 'ai_rank(' || quote_literal(ranking_model) || ', ' || quote_literal(question) || ', chunk)';
ELSE
ai_rank_expr := 'ai_rank(' || quote_literal(question) || ', chunk)';
END IF;
IF llm_model_valid THEN
ai_gen_expr := 'ai_gen(' || quote_literal(llm_model) ||
', replace(replace(replace(' || quote_literal(prompt) ||
', ''${question}'', ' || quote_literal(question) || '), ''${context}'', merged_chunks), ''${language}'', ' || quote_literal(language) || ') )';
ELSE
ai_gen_expr := 'ai_gen(replace(replace(replace(' || quote_literal(prompt) ||
', ''${question}'', ' || quote_literal(question) || '), ''${context}'', merged_chunks), ''${language}'', ' || quote_literal(language) || '))';
END IF;
sql := '
WITH
embedding_recall AS (
SELECT
chunk,
approx_cosine_distance(' || vector_col || ', ' || embedding_expr || ') AS distance
FROM
' || corpus_table || '
ORDER BY
distance DESC
LIMIT ' || vector_recall_count || '
),
rerank AS (
SELECT
chunk,
' || ai_rank_expr || ' AS score
FROM
embedding_recall
ORDER BY
score DESC
LIMIT ' || rerank_recall_count || '
),
concat_top_chunks AS (
SELECT string_agg(chunk, E''\n\n----\n\n'') AS merged_chunks FROM rerank
)
SELECT ' || ai_gen_expr || '
FROM concat_top_chunks;
';
EXECUTE sql INTO final_answer;
RETURN final_answer;
END;
$$ LANGUAGE plpgsql;
Função de busca de texto completo
CREATE OR REPLACE FUNCTION qa_text_search_retrieval(
question TEXT,
corpus_table TEXT,
llm_model TEXT DEFAULT NULL,
ranking_model TEXT DEFAULT NULL,
prompt TEXT DEFAULT 'Please answer the following question in ${language} based on the reference information.\n\n Question: ${question}\n\n Reference information:\n\n ${context}',
language TEXT DEFAULT 'Chinese',
text_search_recall_count INT DEFAULT 20,
rerank_recall_count INT DEFAULT 5,
text_search_col TEXT DEFAULT 'chunk'
)
RETURNS TEXT AS
$$
DECLARE
final_answer TEXT;
sql TEXT;
ai_rank_expr TEXT;
ai_gen_expr TEXT;
llm_model_valid BOOLEAN;
ranking_model_valid BOOLEAN;
BEGIN
llm_model_valid := (llm_model IS NOT NULL AND trim(llm_model) != '');
ranking_model_valid := (ranking_model IS NOT NULL AND trim(ranking_model) != '');
IF ranking_model_valid THEN
ai_rank_expr := 'ai_rank(' || quote_literal(ranking_model) || ', ' || quote_literal(question) || ', chunk)';
ELSE
ai_rank_expr := 'ai_rank(' || quote_literal(question) || ', chunk)';
END IF;
IF llm_model_valid THEN
ai_gen_expr := 'ai_gen(' || quote_literal(llm_model) ||
', replace(replace(replace(' || quote_literal(prompt) ||
', ''${question}'', ' || quote_literal(question) ||
'), ''${context}'', merged_chunks), ''${language}'', ' || quote_literal(language) || ') )';
ELSE
ai_gen_expr := 'ai_gen(replace(replace(replace(' || quote_literal(prompt) ||
', ''${question}'', ' || quote_literal(question) ||
'), ''${context}'', merged_chunks), ''${language}'', ' || quote_literal(language) || '))';
END IF;
sql := '
WITH
text_search_recall AS (
SELECT
chunk
FROM
' || corpus_table || '
ORDER BY
text_search(' || text_search_col || ', ' || quote_literal(question) || ') DESC
LIMIT ' || text_search_recall_count || '
),
rerank AS (
SELECT
chunk,
' || ai_rank_expr || ' AS score
FROM
text_search_recall
ORDER BY
score DESC
LIMIT ' || rerank_recall_count || '
),
concat_top_chunks AS (
SELECT string_agg(chunk, E''\n\n----\n\n'') AS merged_chunks FROM rerank
)
SELECT ' || ai_gen_expr || '
FROM concat_top_chunks;
';
EXECUTE sql INTO final_answer;
RETURN final_answer;
END;
$$ LANGUAGE plpgsql;
Busca híbrida com classificação
CREATE OR REPLACE FUNCTION qa_hybrid_retrieval(
question TEXT,
corpus_table TEXT,
embedding_model TEXT DEFAULT NULL,
llm_model TEXT DEFAULT NULL,
ranking_model TEXT DEFAULT NULL,
prompt TEXT DEFAULT 'Please answer the following question in ${language} based on the reference information.\n\n Question: ${question}\n\n Reference information:\n\n ${context}',
language TEXT DEFAULT 'Chinese',
text_search_recall_count INT DEFAULT 20,
vector_recall_count INT DEFAULT 20,
rerank_recall_count INT DEFAULT 5,
vector_col TEXT DEFAULT 'embedding_vector',
text_search_col TEXT DEFAULT 'chunk'
)
RETURNS TEXT AS
$$
DECLARE
final_answer TEXT;
sql TEXT;
embedding_expr TEXT;
ai_rank_expr TEXT;
ai_gen_expr TEXT;
embedding_model_valid BOOLEAN;
llm_model_valid BOOLEAN;
ranking_model_valid BOOLEAN;
BEGIN
embedding_model_valid := (embedding_model IS NOT NULL AND trim(embedding_model) != '');
llm_model_valid := (llm_model IS NOT NULL AND trim(llm_model) != '');
ranking_model_valid := (ranking_model IS NOT NULL AND trim(ranking_model) != '');
IF embedding_model_valid THEN
embedding_expr := 'ai_embed(' || quote_literal(embedding_model) || ', ' || quote_literal(question) || ')';
ELSE
embedding_expr := 'ai_embed(' || quote_literal(question) || ')';
END IF;
IF ranking_model_valid THEN
ai_rank_expr := 'ai_rank(' || quote_literal(ranking_model) || ', ' || quote_literal(question) || ', chunk)';
ELSE
ai_rank_expr := 'ai_rank(' || quote_literal(question) || ', chunk)';
END IF;
IF llm_model_valid THEN
ai_gen_expr := 'ai_gen(' || quote_literal(llm_model) ||
', replace(replace(replace(' || quote_literal(prompt) ||
', ''${question}'', ' || quote_literal(question) || '), ''${context}'', merged_chunks), ''${language}'', ' || quote_literal(language) || ') )';
ELSE
ai_gen_expr := 'ai_gen(replace(replace(replace(' || quote_literal(prompt) ||
', ''${question}'', ' || quote_literal(question) || '), ''${context}'', merged_chunks), ''${language}'', ' || quote_literal(language) || '))';
END IF;
sql := '
WITH
embedding_recall AS (
SELECT
chunk
FROM
' || corpus_table || '
ORDER BY
approx_cosine_distance(' || vector_col || ', ' || embedding_expr || ') DESC
LIMIT ' || vector_recall_count || '
),
text_search_recall AS (
SELECT
chunk
FROM
' || corpus_table || '
ORDER BY
text_search(' || text_search_col || ', ' || quote_literal(question) || ') DESC
LIMIT ' || text_search_recall_count || '
),
union_recall AS (
SELECT chunk FROM embedding_recall
UNION
SELECT chunk FROM text_search_recall
),
rerank AS (
SELECT
chunk,
' || ai_rank_expr || ' AS score
FROM
union_recall
ORDER BY
score DESC
LIMIT ' || rerank_recall_count || '
),
concat_top_chunks AS (
SELECT string_agg(chunk, E''\n\n----\n\n'') AS merged_chunks FROM rerank
)
SELECT ' || ai_gen_expr || '
FROM concat_top_chunks;
';
EXECUTE sql INTO final_answer;
RETURN final_answer;
END;
$$ LANGUAGE plpgsql;
Busca híbrida com RRF
CREATE OR REPLACE FUNCTION qa_hybrid_retrieval_rrf(
question TEXT,
corpus_table TEXT,
embedding_model TEXT DEFAULT NULL,
llm_model TEXT DEFAULT NULL,
ranking_model TEXT DEFAULT NULL,
prompt TEXT DEFAULT 'Please answer the following question in ${language} based on the reference information.\n\n Question: ${question}\n\n Reference information:\n\n ${context}',
language TEXT DEFAULT 'Chinese',
text_search_recall_count INT DEFAULT 20,
vector_recall_count INT DEFAULT 20,
rerank_recall_count INT DEFAULT 5,
rrf_k INT DEFAULT 60,
vector_col TEXT DEFAULT 'embedding_vector',
text_search_col TEXT DEFAULT 'chunk'
)
RETURNS TEXT AS
$$
DECLARE
final_answer TEXT;
sql TEXT;
embedding_expr TEXT;
ai_rank_expr TEXT;
ai_gen_expr TEXT;
embedding_model_valid BOOLEAN;
llm_model_valid BOOLEAN;
ranking_model_valid BOOLEAN;
BEGIN
embedding_model_valid := (embedding_model IS NOT NULL AND trim(embedding_model) <> '');
llm_model_valid := (llm_model IS NOT NULL AND trim(llm_model) <> '');
ranking_model_valid := (ranking_model IS NOT NULL AND trim(ranking_model) <> '');
IF embedding_model_valid THEN
embedding_expr := 'ai_embed(' || quote_literal(embedding_model) || ', ' || quote_literal(question) || ')';
ELSE
embedding_expr := 'ai_embed(' || quote_literal(question) || ')';
END IF;
IF ranking_model_valid THEN
ai_rank_expr := 'ai_rank(' || quote_literal(ranking_model) || ', ' || quote_literal(question) || ', chunk)';
ELSE
ai_rank_expr := 'ai_rank(' || quote_literal(question) || ', chunk)';
END IF;
IF llm_model_valid THEN
ai_gen_expr := 'ai_gen(' || quote_literal(llm_model) ||
', replace(replace(replace(' || quote_literal(prompt) ||
', ''${question}'', ' || quote_literal(question) || '), ''${context}'', merged_chunks), ''${language}'', ' || quote_literal(language) || ') )';
ELSE
ai_gen_expr := 'ai_gen(replace(replace(replace(' || quote_literal(prompt) ||
', ''${question}'', ' || quote_literal(question) || '), ''${context}'', merged_chunks), ''${language}'', ' || quote_literal(language) || '))';
END IF;
sql := '
WITH embedding_recall AS (
SELECT
chunk,
vec_score,
ROW_NUMBER() OVER (ORDER BY vec_score DESC) AS rank_vec
FROM (
SELECT
chunk,
approx_cosine_distance(' || vector_col || ', ' || embedding_expr || ') AS vec_score
FROM
' || corpus_table || '
) t
ORDER BY vec_score DESC
LIMIT ' || vector_recall_count || '
),
text_search_recall AS (
SELECT
chunk,
text_score,
ROW_NUMBER() OVER (ORDER BY text_score DESC) AS rank_text
FROM (
SELECT
chunk,
text_search(' || text_search_col || ', ' || quote_literal(question) || ') AS text_score
FROM
' || corpus_table || '
) ts
WHERE text_score > 0
ORDER BY text_score DESC
LIMIT ' || text_search_recall_count || '
),
rrf_scores AS (
SELECT
chunk,
SUM(1.0 / (' || rrf_k || ' + rank_val)) AS rrf_score
FROM (
SELECT chunk, rank_vec AS rank_val FROM embedding_recall
UNION ALL
SELECT chunk, rank_text AS rank_val FROM text_search_recall
) sub
GROUP BY chunk
),
top_chunks AS (
SELECT chunk
FROM rrf_scores
ORDER BY rrf_score DESC
LIMIT ' || rerank_recall_count || '
),
concat_top_chunks AS (
SELECT string_agg(chunk, E''\n\n----\n\n'') AS merged_chunks FROM top_chunks
)
SELECT ' || ai_gen_expr || '
FROM concat_top_chunks;
';
EXECUTE sql INTO final_answer;
RETURN final_answer;
END;
$$ LANGUAGE plpgsql;