Object Storage Service:Gunakan AI VibeFlow untuk menghasilkan data vektor

Konsol

1. Masuk ke Konsol OSS.

2. Di panel navigasi sebelah kiri, klik AI VibeFlow.

3. Di halaman AI VibeFlow, klik Create DataPipeline.

4. Di panel Create DataPipeline, konfigurasikan item berikut sesuai petunjuk, lalu klik OK.

   • Pengaturan dasar

     • Rule Name: maksimal 64 karakter. Huruf besar dan kecil, angka, tanda hubung (-), dan garis bawah (_) didukung. Nama harus unik dalam akun dan wilayah yang sama.

     • Rule Description: opsional, maksimal 200 karakter.

   • Pengaturan data sumber

     • Source Bucket: pilih wilayah terlebih dahulu, lalu pilih bucket target.

     • Source File Scope: All Files memproses semua objek di bucket; Specified Object Name Prefix hanya memproses objek yang sesuai dengan awalan tersebut.

     • Source File Types: Anda dapat memilih beberapa jenis dari Text, Image, dan Video. Pilihan ini harus sesuai dengan kemampuan model vektor yang akan digunakan nanti.

     • Processing Mode: Process only existing initial data (file yang sudah ada saat pipeline dibuat); Process only subsequent incremental data (file yang baru diunggah akan memicu pemrosesan secara otomatis); Process both existing and subsequent incremental data.

   • Pengaturan vektorisasi

     • Vector Model Source: saat ini hanya Alibaba Cloud Model Studio yang didukung.

     • API KEY: masukkan kunci API Model Studio, yang diawali dengan sk- dan memiliki panjang 35 hingga 128 karakter.

     • Vector Model: model teks seperti text-embedding-v4 (default) dan text-embedding-v3; model multimodal seperti tongyi-embedding-vision-plus dan tongyi-embedding-vision-flash. Pastikan dimensi output model sama dengan dimensi indeks vektor.

     • Video Frame Capture Frequency: hanya dapat dikonfigurasi untuk model multimodal. Rentang nilainya (0, 1], dengan nilai default 1,0. Nilai yang lebih kecil menangkap lebih sedikit frame.

     • Cascade Deletion Settings: Retain the corresponding vector data when the source object is deleted (default); Delete the corresponding vector data when the source object is deleted.

   • Pengaturan data tujuan

     • Vector Bucket: harus berada di wilayah yang sama dengan bucket sumber.

     • Vector Index: pilih indeks target dari daftar indeks bucket vektor yang dipilih. Dimensi indeks harus sama dengan dimensi output model vektor.

     • Vector Key Naming Rule: opsional. Jika Anda memasukkan awalan, formatnya adalah {prefix}/{ObjectKey}. Jika dibiarkan kosong, kunci vektor akan sama dengan ObjectKey.

     • Use ObjectTag as metadata: opsional, maksimal 10 kunci, digunakan untuk filtering skalar selama pengambilan.

     • Use UserMeta as metadata: opsional, maksimal 10 kunci. Sistem secara otomatis menambahkan awalan X-OSS-META-.

   • Penanganan pengecualian dan otorisasi

     • Exception Handling Method: Skip the exception and continue with subsequent processing tasks; Skip the exception and continue with subsequent processing tasks, and obtain the exception details (Anda harus mengonfigurasi bucket dan awalan untuk menerima pesan kesalahan; bucket penerima harus berada di wilayah yang sama dengan bucket sumber dan tidak boleh merupakan bucket sumber itu sendiri); Terminate subsequent tasks immediately when an exception occurs.

     • Authorized Role: pilih Create Role untuk secara otomatis membuat dan mengikat oss-embedding-{uuid}.

   • Konfirmasi pengaturan: periksa semua pengaturan dan klik OK setelah Anda memastikan semuanya benar.

ossutil

<?xml version="1.0" encoding="UTF-8" ?>
<DataPipelineConfiguration>
  <Sources>
      <InputBucket>my-bucket</InputBucket>
      <InputDataScope>All</InputDataScope>
  </Sources>
  <DataPipelineEmbeddingConfiguration>
      <EmbeddingProvider>bailian</EmbeddingProvider>
      <ApiKey>sk-xxxx</ApiKey>
      <Model>qwen2.5-vl-embedding</Model>
  </DataPipelineEmbeddingConfiguration>
  <Destination>
      <VectorBucketName>my-vector-bucket</VectorBucketName>
      <VectorIndexNames>my-index</VectorIndexNames>
  </Destination>
  <DataPipelineError>
      <ErrorMode>ignore</ErrorMode>
  </DataPipelineError>
</DataPipelineConfiguration>

ossutil api do-data-pipe-line-action --action putDataPipelineConfiguration \
  --parameters dataPipelineName=my-data-pipeline \
  --parameters role=acs:ram::<AccountId>:role/my-data-pipeline-role \
  --body file://config.xml \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;
import java.util.Arrays;
import java.util.Collections;

public class PutDataPipelineConfigurationSample {
    public static void main(String[] args) throws Exception {
        String role = "acs:ram::<AccountId>:role/my-data-pipeline-role";
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            DataPipelineSourceFilterConfiguration filterConfig =
                    DataPipelineSourceFilterConfiguration.newBuilder()
                            .prefixSet(Arrays.asList("data/"))
                            .objectMediaTypes(Arrays.asList("image", "video"))
                            .build();

            DataPipelineSource source = DataPipelineSource.newBuilder()
                    .inputBucket("my-bucket")
                    .inputDataScope("All")
                    .ignoreDelete(true)
                    .filterConfiguration(filterConfig)
                    .build();

            DataPipelineEmbeddingConfiguration embeddingConfig =
                    DataPipelineEmbeddingConfiguration.newBuilder()
                            .embeddingProvider("bailian")
                            .apiKey("sk-xxxx")
                            .model("qwen2.5-vl-embedding")
                            .fps(1.0f)
                            .build();

            DataPipelineDestination destination = DataPipelineDestination.newBuilder()
                    .vectorBucketName("my-vector-bucket")
                    .vectorIndexNames(Collections.singletonList("my-index"))
                    .build();

            PutDataPipelineConfigurationConfiguration config =
                    PutDataPipelineConfigurationConfiguration.newBuilder()
                            .dataPipelineDescription("Data pipeline sample")
                            .sources(Collections.singletonList(source))
                            .dataPipelineEmbeddingConfiguration(embeddingConfig)
                            .destination(destination)
                            .build();

            PutDataPipelineConfigurationResult result = client.putDataPipelineConfiguration(
                    PutDataPipelineConfigurationRequest.newBuilder()
                            .dataPipelineName("my-data-pipeline")
                            .role(role)
                            .putDataPipelineConfigurationConfiguration(config)
                            .build());

            System.out.printf("Status code:%d, request id:%s%n",
                    result.statusCode(), result.requestId());

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"
	role := "acs:ram::<AccountId>:role/my-data-pipeline-role"
	apiKey := "sk-xxxx"
	dataPipelineName := "my-data-pipeline"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	result, err := client.PutDataPipelineConfiguration(context.TODO(), &dataprocess.PutDataPipelineConfigurationRequest{
		DataPipelineName: oss.Ptr(dataPipelineName),
		Role:             oss.Ptr(role),
		DataPipelineConfiguration: &dataprocess.DataPipelineConfiguration{
			DataPipelineDescription: oss.Ptr("Vectorize business data using the BERT multimodal model"),
			Sources: []dataprocess.DataPipelineSource{
				{
					InputBucket:    oss.Ptr("bucket"),
					InputDataScope: oss.Ptr("All"),
					FilterConfiguration: &dataprocess.DataPipelineSourceFilterConfiguration{
						PrefixSet:        []string{"prefix1"},
						ObjectMediaTypes: []string{"text"},
					},
				},
			},
			DataPipelineEmbeddingConfiguration: &dataprocess.DataPipelineEmbeddingConfiguration{
				ApiKey:            oss.Ptr(apiKey),
				EmbeddingProvider: oss.Ptr("bailian"),
				FPS:               oss.Ptr(float64(1)),
				Model:             oss.Ptr("qwen2.5-vl-embedding"),
			},
			Destination: &dataprocess.DataPipelineDestination{
				VectorBucketName:    oss.Ptr("my-vector-bucket"),
				VectorIndexNames:    []string{"index"},
				VectorKeyPrefix:     oss.Ptr("prefix"),
				ObjectTagToMetadata: []string{"key1"},
				UsermetaToMetadata:  []string{"x-oss-meta-key1"},
			},
			DataPipelineError: &dataprocess.DataPipelineError{
				ErrorMode:   oss.Ptr("ignoreAndRecord"),
				ErrorBucket: oss.Ptr("my-error-bucket"),
				ErrorPrefix: oss.Ptr("error-output/"),
			},
		},
	})

	if err != nil {
		log.Fatalf("failed to put pipeline configuration %v", err)
	}
	log.Printf("put pipeline configuration result:%#v\n", result)
}

API

Panggil operasi PutDataPipelineConfiguration untuk membuat DataPipeline.

Konsol

1. Di halaman AI VibeFlow, klik Rule Name DataPipeline target atau klik Details di kolom Actions.

2. Di halaman detail, lihat informasi lengkap, termasuk pengaturan dasar, pengaturan data sumber, pengaturan vektorisasi, pengaturan tujuan, pengaturan penanganan pengecualian, dan pengaturan otorisasi.

ossutil

ossutil api do-data-pipe-line-action --action getDataPipelineConfiguration \
  --parameters dataPipelineName=my-data-pipeline \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;

public class GetDataPipelineConfigurationSample {
    public static void main(String[] args) throws Exception {
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            GetDataPipelineConfigurationResult result = client.getDataPipelineConfiguration(
                    GetDataPipelineConfigurationRequest.newBuilder()
                            .dataPipelineName("my-data-pipeline")
                            .build());

            System.out.printf("Status code:%d, request id:%s%n",
                    result.statusCode(), result.requestId());

            DataPipelineConfiguration cfg = result.dataPipelineConfiguration();
            if (cfg != null) {
                System.out.printf("Pipeline name:%s, status:%s, role:%s, createTime:%s%n",
                        cfg.dataPipelineName(), cfg.status(),
                        cfg.dataPipelineRole(), cfg.createTime());
            }

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"
	dataPipelineName := "my-data-pipeline"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	result, err := client.GetDataPipelineConfiguration(context.TODO(), &dataprocess.GetDataPipelineConfigurationRequest{
		DataPipelineName: oss.Ptr(dataPipelineName),
	})

	if err != nil {
		log.Fatalf("failed to get pipeline configuration %v", err)
	}
	log.Printf("get pipeline configuration result:%#v\n", result)
}

API

Panggil operasi GetDataPipelineConfiguration untuk mendapatkan konfigurasi DataPipeline.

Konsol

Di halaman AI VibeFlow, Anda dapat langsung melihat daftar semua DataPipeline akun saat ini di wilayah yang dipilih, termasuk informasi seperti nama aturan, deskripsi, status, dan waktu pembuatan.

ossutil

ossutil api do-data-pipe-line-action --action listDataPipelineConfigurations \
  --parameters maxResults=100 \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;

public class ListDataPipelineConfigurationsSample {
    public static void main(String[] args) throws Exception {
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            String nextToken = null;
            do {
                ListDataPipelineConfigurationsRequest.Builder reqBuilder =
                        ListDataPipelineConfigurationsRequest.newBuilder().maxResults(100);
                if (nextToken != null) {
                    reqBuilder.nextToken(nextToken);
                }

                ListDataPipelineConfigurationsResult result =
                        client.listDataPipelineConfigurations(reqBuilder.build());

                System.out.printf("Status code:%d, request id:%s%n",
                        result.statusCode(), result.requestId());

                if (result.dataPipelineConfigurations() != null) {
                    for (DataPipelineConfiguration cfg : result.dataPipelineConfigurations()) {
                        System.out.printf("Pipeline: name=%s, status=%s, createTime=%s%n",
                                cfg.dataPipelineName(), cfg.status(), cfg.createTime());
                    }
                }

                nextToken = result.nextToken();
            } while (nextToken != null && !nextToken.isEmpty());

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	p := client.NewListDataPipelineConfigurationsPaginator(&dataprocess.ListDataPipelineConfigurationsRequest{})

	var i int
	log.Println("Data Pipeline Configurations:")
	for p.HasNext() {
		i++

		page, err := p.NextPage(context.TODO())
		if err != nil {
			log.Fatalf("failed to get page %v, %v", i, err)
		}

		for _, data := range page.DataPipelineConfigurations {
			log.Printf("Data Pipeline Name:%v, Data Pipeline Description:%v, Data Pipeline Role:%v, Status:%v, Phase:%v\n", oss.ToString(data.DataPipelineName), oss.ToString(data.DataPipelineDescription), oss.ToString(data.DataPipelineRole), oss.ToString(data.Status), oss.ToString(data.Phase))
		}
	}
}

API

Panggil operasi ListDataPipelineConfigurations untuk menampilkan daftar DataPipeline.

Konsol

Di halaman AI VibeFlow, temukan DataPipeline target lalu klik Pause di kolom Actions.

ossutil

ossutil api do-data-pipe-line-action --action pauseDataPipeline \
  --parameters dataPipelineName=my-data-pipeline \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;

public class PauseDataPipelineSample {
    public static void main(String[] args) throws Exception {
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            PauseDataPipelineResult result = client.pauseDataPipeline(
                    PauseDataPipelineRequest.newBuilder()
                            .dataPipelineName("my-data-pipeline")
                            .build());

            System.out.printf("Status code:%d, request id:%s%n",
                    result.statusCode(), result.requestId());

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"
	dataPipelineName := "my-data-pipeline"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	result, err := client.PauseDataPipeline(context.TODO(), &dataprocess.PauseDataPipelineRequest{
		DataPipelineName: oss.Ptr(dataPipelineName),
	})

	if err != nil {
		log.Fatalf("failed to pause pipeline %v", err)
	}
	log.Printf("pause pipeline result:%#v\n", result)
}

API

Panggil operasi PauseDataPipeline untuk menjeda DataPipeline.

Konsol

Di halaman AI VibeFlow, temukan DataPipeline target lalu klik Restart di kolom Actions.

ossutil

ossutil api do-data-pipe-line-action --action restartDataPipeline \
  --parameters dataPipelineName=my-data-pipeline \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;

public class RestartDataPipelineSample {
    public static void main(String[] args) throws Exception {
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            RestartDataPipelineResult result = client.restartDataPipeline(
                    RestartDataPipelineRequest.newBuilder()
                            .dataPipelineName("my-data-pipeline")
                            .build());

            System.out.printf("Status code:%d, request id:%s%n",
                    result.statusCode(), result.requestId());

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"
	dataPipelineName := "my-data-pipeline"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	result, err := client.RestartDataPipeline(context.TODO(), &dataprocess.RestartDataPipelineRequest{
		DataPipelineName: oss.Ptr(dataPipelineName),
	})

	if err != nil {
		log.Fatalf("failed to restart pipeline %v", err)
	}
	log.Printf("restart pipeline result:%#v\n", result)
}

API

Panggil operasi RestartDataPipeline untuk memulai ulang DataPipeline.

Konsol

Di halaman AI VibeFlow, temukan DataPipeline target, klik Delete di kolom Actions, lalu konfirmasi untuk menyelesaikan penghapusan.

ossutil

ossutil api do-data-pipe-line-action --action deleteDataPipelineConfiguration \
  --parameters dataPipelineName=my-data-pipeline \
  --region cn-hangzhou

SDK

import com.aliyun.sdk.service.oss2.credentials.CredentialsProvider;
import com.aliyun.sdk.service.oss2.credentials.EnvironmentVariableCredentialsProvider;
import com.aliyun.sdk.service.oss2.dataprocess.OSSDataProcessClient;
import com.aliyun.sdk.service.oss2.dataprocess.models.*;

public class DeleteDataPipelineConfigurationSample {
    public static void main(String[] args) throws Exception {
        CredentialsProvider provider = new EnvironmentVariableCredentialsProvider();
        try (OSSDataProcessClient client = OSSDataProcessClient.newBuilder()
                .region("cn-hangzhou")
                .credentialsProvider(provider)
                .build()) {

            DeleteDataPipelineConfigurationResult result = client.deleteDataPipelineConfiguration(
                    DeleteDataPipelineConfigurationRequest.newBuilder()
                            .dataPipelineName("my-data-pipeline")
                            .build());

            System.out.printf("Status code:%d, request id:%s%n",
                    result.statusCode(), result.requestId());

        } catch (Exception e) {
            System.out.printf("error:%n%s", e);
        }
    }
}

package main

import (
	"context"
	"log"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/dataprocess"
)

func main() {
	region := "cn-hangzhou"
	dataPipelineName := "my-data-pipeline"

	cfg := oss.LoadDefaultConfig().
		WithCredentialsProvider(credentials.NewEnvironmentVariableCredentialsProvider()).
		WithRegion(region)

	client := dataprocess.NewClient(cfg)

	result, err := client.DeleteDataPipelineConfiguration(context.TODO(), &dataprocess.DeleteDataPipelineConfigurationRequest{
		DataPipelineName: oss.Ptr(dataPipelineName),
	})

	if err != nil {
		log.Fatalf("failed to delete pipeline configuration %v", err)
	}
	log.Printf("delete pipeline configuration result:%#v\n", result)
}

API

Panggil operasi DeleteDataPipelineConfiguration untuk menghapus DataPipeline.