Object Storage Service:Gunakan AI VibeFlow untuk menghasilkan data vektor

Konsol

1. Masuk ke Konsol OSS.

2. Pada panel navigasi kiri, klik AI VibeFlow.

3. Pada halaman AI VibeFlow, klik Create Data Pipeline.

4. Pada panel Create Data Pipeline, konfigurasikan item-item berikut sesuai petunjuk, lalu klik OK.

   • Basic settings

     • Rule Name: maksimal 64 karakter. Huruf besar dan kecil, angka, tanda hubung (-), dan garis bawah (_) didukung. Nama harus unik dalam akun dan wilayah yang sama.

     • Rule Description: opsional, maksimal 200 karakter.

   • Source data settings

     • Source Bucket: pilih wilayah terlebih dahulu, lalu pilih bucket target.

     • Source File Scope: All Files memproses semua objek dalam bucket; Specify File Name Prefix hanya memproses objek yang sesuai dengan awalan tersebut.

     • Source File Type: Anda dapat memilih beberapa jenis dari Text, Image, dan Video. Pilihan harus sesuai dengan kemampuan model vektor yang akan digunakan nanti.

     • Processing Mode: Process Existing Data Only (file yang sudah ada saat pipeline dibuat); Process Incremental Data Only (file yang baru diunggah akan memicu pemrosesan secara otomatis); Process Existing and Incremental Data.

   • Vectorization settings

     • Vector Model Source: saat ini hanya Alibaba Cloud Model Studio yang didukung.

     • API Key: masukkan Kunci API Model Studio, yang diawali dengan sk- dan panjangnya antara 35 hingga 128 karakter.

     • Vector Model: model teks seperti text-embedding-v4 (default) dan text-embedding-v3; model multimodal seperti tongyi-embedding-vision-plus dan tongyi-embedding-vision-flash. Pastikan dimensi output model sama dengan dimensi indeks vektor.

     • Video Snapshot Frequency: hanya dapat dikonfigurasi untuk model multimodal. Rentang nilainya (0, 1], dengan nilai default 1,0. Nilai yang lebih kecil mengambil lebih sedikit frame.

     • Cascade Delete Settings: Retain vector data when the source object is deleted (default); Delete corresponding vector data when the source object is deleted.

   • Destination data settings

     • Vector Bucket: harus berada di wilayah yang sama dengan bucket sumber.

     • Vector Index: pilih indeks target dari daftar indeks bucket vektor yang dipilih. Dimensi indeks harus sama dengan dimensi output model vektor.

     • Vector Key Naming Method: opsional. Jika Anda memasukkan awalan, formatnya adalah {prefix}/{ObjectKey}. Jika dibiarkan kosong, kunci vektor akan sama dengan ObjectKey.

     • Use ObjectTag as Metadata: opsional, maksimal 10 kunci, digunakan untuk penyaringan skalar selanjutnya saat pengambilan.

     • Use UserMeta as Metadata: opsional, maksimal 10 kunci. Sistem secara otomatis menambahkan awalan X-OSS-META-.

   • Exception handling and authorization

     • Exception Handling: Continue tasks despite errors; Skip exceptions and continue with subsequent tasks while recording exception details (Anda harus mengonfigurasi Exception Information Bucket dan Exception Log Prefix untuk menerima pesan kesalahan; bucket penerima harus berada di wilayah yang sama dengan bucket sumber dan tidak boleh merupakan bucket sumber itu sendiri); Stop subsequent tasks on failure.

     • Authorization Role: pilih New RAM Role untuk secara otomatis membuat dan mengikat oss-embedding-{uuid}.

   • Confirm the settings: periksa semua pengaturan dan klik OK setelah memastikan semuanya benar.

ossutil

<?xml version="1.0" encoding="UTF-8" ?>
<DataPipelineConfiguration>
  <Sources>
      <InputBucket>my-bucket</InputBucket>
      <InputDataScope>All</InputDataScope>
  </Sources>
  <DataPipelineEmbeddingConfiguration>
      <EmbeddingProvider>bailian</EmbeddingProvider>
      <ApiKey>sk-xxxx</ApiKey>
      <Model>qwen2.5-vl-embedding</Model>
  </DataPipelineEmbeddingConfiguration>
  <Destination>
      <VectorBucketName>my-vector-bucket</VectorBucketName>
      <VectorIndexNames>myIndex</VectorIndexNames>
  </Destination>
  <DataPipelineError>
      <ErrorMode>ignore</ErrorMode>
  </DataPipelineError>
</DataPipelineConfiguration>

ossutil api do-data-pipe-line-action --action putDataPipelineConfiguration \
  --parameters dataPipelineName=my-data-pipeline \
  --parameters role=acs:ram::<AccountId>:role/my-data-pipeline-role \
  --body file://config.xml \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;
import java.util.Arrays;
import java.util.Collections;

public class PutDataPipelineConfigurationSample {
    public static void main(String[] args) throws Exception {
        String role = "acs:ram::<AccountId>:role/my-data-pipeline-role";
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            DataPipelineSourceFilterConfiguration filterConfig =
                    DataPipelineSourceFilterConfiguration.newBuilder()
                            .prefixSet(Arrays.asList("data/"))
                            .objectMediaTypes(Arrays.asList("image", "video"))
                            .build();

            DataPipelineSource source = DataPipelineSource.newBuilder()
                    .inputBucket("my-bucket")
                    .inputDataScope("All")
                    .ignoreDelete(true)
                    .filterConfiguration(filterConfig)
                    .build();

            DataPipelineEmbeddingConfiguration embeddingConfig =
                    DataPipelineEmbeddingConfiguration.newBuilder()
                            .embeddingProvider("bailian")
                            .apiKey("sk-xxxx")
                            .model("qwen2.5-vl-embedding")
                            .fps(1.0f)
                            .build();

            DataPipelineDestination destination = DataPipelineDestination.newBuilder()
                    .vectorBucketName("my-vector-bucket")
                    .vectorIndexNames(Collections.singletonList("my-index"))
                    .build();

            PutDataPipelineConfigurationConfiguration config =
                    PutDataPipelineConfigurationConfiguration.newBuilder()
                            .dataPipelineDescription("Data pipeline sample")
                            .sources(Collections.singletonList(source))
                            .dataPipelineEmbeddingConfiguration(embeddingConfig)
                            .destination(destination)
                            .build();

            PutDataPipelineConfigurationResult result = client.putDataPipelineConfiguration(
                    PutDataPipelineConfigurationRequest.newBuilder()
                            .dataPipelineName("my-data-pipeline")
                            .role(role)
                            .putDataPipelineConfigurationConfiguration(config)
                            .build());

            System.out.printf("Status code:%d, request id:%s%n",
                    result.statusCode(), result.requestId());

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"
	role := "acs:ram::<AccountId>:role/my-data-pipeline-role"
	apiKey := "sk-xxxx"
	dataPipelineName := "my-data-pipeline"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	result, err := client.PutDataPipelineConfiguration(context.TODO(), &dataprocess.PutDataPipelineConfigurationRequest{
		DataPipelineName: oss.Ptr(dataPipelineName),
		Role:             oss.Ptr(role),
		DataPipelineConfiguration: &dataprocess.DataPipelineConfiguration{
			DataPipelineDescription: oss.Ptr("Vectorize business data using the BERT multimodal model"),
			Sources: []dataprocess.DataPipelineSource{
				{
					InputBucket:    oss.Ptr("bucket"),
					InputDataScope: oss.Ptr("All"),
					FilterConfiguration: &dataprocess.DataPipelineSourceFilterConfiguration{
						PrefixSet:        []string{"prefix1"},
						ObjectMediaTypes: []string{"text"},
					},
				},
			},
			DataPipelineEmbeddingConfiguration: &dataprocess.DataPipelineEmbeddingConfiguration{
				ApiKey:            oss.Ptr(apiKey),
				EmbeddingProvider: oss.Ptr("bailian"),
				FPS:               oss.Ptr(float64(1)),
				Model:             oss.Ptr("qwen2.5-vl-embedding"),
			},
			Destination: &dataprocess.DataPipelineDestination{
				VectorBucketName:    oss.Ptr("my-vector-bucket"),
				VectorIndexNames:    []string{"index"},
				VectorKeyPrefix:     oss.Ptr("prefix"),
				ObjectTagToMetadata: []string{"key1"},
				UsermetaToMetadata:  []string{"x-oss-meta-key1"},
			},
			DataPipelineError: &dataprocess.DataPipelineError{
				ErrorMode:   oss.Ptr("ignoreAndRecord"),
				ErrorBucket: oss.Ptr("my-error-bucket"),
				ErrorPrefix: oss.Ptr("error-output/"),
			},
		},
	})

	if err != nil {
		log.Fatalf("failed to put pipeline configuration %v", err)
	}
	log.Printf("put pipeline configuration result:%#v\n", result)
}

API

Panggil operasi PutDataPipelineConfiguration untuk membuat DataPipeline.

Konsol

1. Pada halaman AI VibeFlow, klik Rule Name dari DataPipeline target atau klik View Details pada kolom Actions.

2. Pada halaman detail, lihat informasi lengkap, termasuk pengaturan dasar, pengaturan data sumber, pengaturan vektorisasi, pengaturan tujuan, pengaturan penanganan eksepsi, dan pengaturan otorisasi.

ossutil

ossutil api do-data-pipe-line-action --action getDataPipelineConfiguration \
  --parameters dataPipelineName=my-data-pipeline \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;

public class GetDataPipelineConfigurationSample {
    public static void main(String[] args) throws Exception {
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            GetDataPipelineConfigurationResult result = client.getDataPipelineConfiguration(
                    GetDataPipelineConfigurationRequest.newBuilder()
                            .dataPipelineName("my-data-pipeline")
                            .build());

            System.out.printf("Status code:%d, request id:%s%n",
                    result.statusCode(), result.requestId());

            DataPipelineConfiguration cfg = result.dataPipelineConfiguration();
            if (cfg != null) {
                System.out.printf("Pipeline name:%s, status:%s, role:%s, createTime:%s%n",
                        cfg.dataPipelineName(), cfg.status(),
                        cfg.dataPipelineRole(), cfg.createTime());
            }

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"
	dataPipelineName := "my-data-pipeline"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	result, err := client.GetDataPipelineConfiguration(context.TODO(), &dataprocess.GetDataPipelineConfigurationRequest{
		DataPipelineName: oss.Ptr(dataPipelineName),
	})

	if err != nil {
		log.Fatalf("failed to get pipeline configuration %v", err)
	}
	log.Printf("get pipeline configuration result:%#v\n", result)
}

API

Panggil operasi GetDataPipelineConfiguration untuk mendapatkan konfigurasi DataPipeline.

Konsol

Pada halaman AI VibeFlow, Anda dapat langsung melihat daftar semua DataPipeline akun saat ini di wilayah yang dipilih, termasuk informasi seperti nama aturan, deskripsi, status, dan waktu pembuatan.

ossutil

ossutil api do-data-pipe-line-action --action listDataPipelineConfigurations \
  --parameters maxResults=100 \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;

public class ListDataPipelineConfigurationsSample {
    public static void main(String[] args) throws Exception {
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            String nextToken = null;
            do {
                ListDataPipelineConfigurationsRequest.Builder reqBuilder =
                        ListDataPipelineConfigurationsRequest.newBuilder().maxResults(100);
                if (nextToken != null) {
                    reqBuilder.nextToken(nextToken);
                }

                ListDataPipelineConfigurationsResult result =
                        client.listDataPipelineConfigurations(reqBuilder.build());

                System.out.printf("Status code:%d, request id:%s%n",
                        result.statusCode(), result.requestId());

                if (result.dataPipelineConfigurations() != null) {
                    for (DataPipelineConfiguration cfg : result.dataPipelineConfigurations()) {
                        System.out.printf("Pipeline: name=%s, status=%s, createTime=%s%n",
                                cfg.dataPipelineName(), cfg.status(), cfg.createTime());
                    }
                }

                nextToken = result.nextToken();
            } while (nextToken != null && !nextToken.isEmpty());

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	p := client.NewListDataPipelineConfigurationsPaginator(&dataprocess.ListDataPipelineConfigurationsRequest{})

	var i int
	log.Println("Data Pipeline Configurations:")
	for p.HasNext() {
		i++

		page, err := p.NextPage(context.TODO())
		if err != nil {
			log.Fatalf("failed to get page %v, %v", i, err)
		}

		for _, data := range page.DataPipelineConfigurations {
			log.Printf("Data Pipeline Name:%v, Data Pipeline Description:%v, Data Pipeline Role:%v, Status:%v, Phase:%v\n", oss.ToString(data.DataPipelineName), oss.ToString(data.DataPipelineDescription), oss.ToString(data.DataPipelineRole), oss.ToString(data.Status), oss.ToString(data.Phase))
		}
	}
}

API

Panggil operasi ListDataPipelineConfigurations untuk menampilkan daftar DataPipeline.

Konsol

Pada halaman AI VibeFlow, temukan DataPipeline target lalu klik Suspend pada kolom Actions.

ossutil

ossutil api do-data-pipe-line-action --action pauseDataPipeline \
  --parameters dataPipelineName=my-data-pipeline \
  --region cn-hangzhou

SDK

package com.example.oss.dataprocess;

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;

public class PauseDataPipelineSample {
    public static void main(String[] args) throws Exception {
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            PauseDataPipelineResult result = client.pauseDataPipeline(
                    PauseDataPipelineRequest.newBuilder()
                            .dataPipelineName("my-data-pipeline")
                            .build());

            System.out.printf("Status code:%d, request id:%s%n",
                    result.statusCode(), result.requestId());

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"
	bucketName := "exampleBucket"
	dataPipelineName := "my-data-pipeline"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	result, err := client.PauseDataPipeline(context.TODO(), &dataprocess.PauseDataPipelineRequest{
		Bucket:           oss.Ptr(bucketName),
		DataPipelineName: oss.Ptr(dataPipelineName),
	})

	if err != nil {
		log.Fatalf("failed to pause pipeline %v", err)
	}
	log.Printf("pause pipeline result:%#v\n", result)
}

API

Panggil operasi PauseDataPipeline untuk menjeda DataPipeline.

Konsol

Pada halaman AI VibeFlow, temukan DataPipeline target lalu klik Restart pada kolom Actions.

ossutil

ossutil api do-data-pipe-line-action --action restartDataPipeline \
  --parameters dataPipelineName=my-data-pipeline \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;

public class RestartDataPipelineSample {
    public static void main(String[] args) throws Exception {
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            RestartDataPipelineResult result = client.restartDataPipeline(
                    RestartDataPipelineRequest.newBuilder()
                            .dataPipelineName("my-data-pipeline")
                            .build());

            System.out.printf("Status code:%d, request id:%s%n",
                    result.statusCode(), result.requestId());

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"
	dataPipelineName := "my-data-pipeline"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	result, err := client.RestartDataPipeline(context.TODO(), &dataprocess.RestartDataPipelineRequest{
		DataPipelineName: oss.Ptr(dataPipelineName),
	})

	if err != nil {
		log.Fatalf("failed to restart pipeline %v", err)
	}
	log.Printf("restart pipeline result:%#v\n", result)
}

API

Panggil operasi RestartDataPipeline untuk memulai ulang DataPipeline.

Konsol

Pada halaman AI VibeFlow, temukan DataPipeline target, klik Delete pada kolom Actions, lalu konfirmasi untuk menyelesaikan penghapusan.

ossutil

ossutil api do-data-pipe-line-action --action deleteDataPipelineConfiguration \
  --parameters dataPipelineName=my-data-pipeline \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;

public class DeleteDataPipelineConfigurationSample {
    public static void main(String[] args) throws Exception {
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            DeleteDataPipelineConfigurationResult result = client.deleteDataPipelineConfiguration(
                    DeleteDataPipelineConfigurationRequest.newBuilder()
                            .dataPipelineName("my-data-pipeline")
                            .build());

            System.out.printf("Status code:%d, request id:%s%n",
                    result.statusCode(), result.requestId());

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"
	dataPipelineName := "my-data-pipeline"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	result, err := client.DeleteDataPipelineConfiguration(context.TODO(), &dataprocess.DeleteDataPipelineConfigurationRequest{
		DataPipelineName: oss.Ptr(dataPipelineName),
	})

	if err != nil {
		log.Fatalf("failed to delete pipeline configuration %v", err)
	}
	log.Printf("delete pipeline configuration result:%#v\n", result)
}

API

Panggil operasi DeleteDataPipelineConfiguration untuk menghapus DataPipeline.