Documentation
Home Page Simple Log Service Developer Reference API Reference Data structure GCSIngestionConfigurationSource

GCSIngestionConfigurationSource

Updated at:
Copy as MD
Product

Parameter

Type

Description

Example

object
Specifies the configuration for ingesting data from a Google Cloud Storage (GCS) data source.

endpoint

string
The service endpoint for GCS. You can use a custom endpoint for private or accelerated connections.

oss-cn-hangzhou.aliyuncs.com
bucket

string
The name of the GCS bucket that contains the source data files.

gcsbucket
compressionCodec

string
The compression format of the source files. Supported values are none, gzip, and zstd. If not specified, the system infers the format from the file extension.

none
encoding

string
The character encoding of the source files. The default value is UTF-8.

UTF-8
interval

string
The interval for checking for new data. Specify the value in a duration format, such as 15m for 15 minutes. Set to never to perform a one-time ingestion.

never
pattern

string
A regular expression that specifies which files to ingest. The pattern is matched against the full object key within the specified prefix.

.*
prefix

string
The object key prefix used to discover files. This limits the scope of ingestion to a specific "folder" within the bucket.

prefix
restoreObjectEnabled

boolean
Specifies whether to automatically restore objects from archival storage classes before ingestion. Set to true to enable this feature. The default is false.

startTime

integer
The start of the time range for data ingestion, specified as a Unix timestamp (in seconds). Only objects modified at or after this time are ingested.

1714274081
endTime

integer
The end of the time range for data ingestion, specified as a Unix timestamp (in seconds). Only objects modified before this time are ingested.

1714360481
timeField

string
The name of the field in your data that contains the timestamp. This timestamp is used as the event time for the ingested records.

__time__
timeFormat

string
The format of the timestamp in the timeField, specified using the Java SimpleDateFormat pattern. For example: yyyy-MM-dd'T'HH:mm:ss.SSSZ.

yyyy-MM-dd HH:mm:ss
timePattern

string
A regular expression used to extract a timestamp from unstructured data, such as a log entry or filename, if a structured timeField is not available.

[0-9]{0,2}\/[0-9a-zA-Z]+\/[0-9:,]+
timeZone

string
The time zone for parsing timestamps that lack explicit time zone information. Specify a valid time zone identifier, such as UTC or America/Los_Angeles.

GMT+08:00
tagPackId

boolean
The ID of a predefined tag pack to apply to the ingested data. Tag packs contain rules for data enrichment and categorization.

processorId

string
The unique ID of the processor or pipeline that handles the ingested data.

ingest-processor-1756802123-953901
accessKeyId

string
The access key ID for authenticating with the GCS service. This parameter is required.

accessKeySecret

string
The secret access key corresponding to the accessKeyId. This value is sensitive and must be managed securely.

format

object
A nested object that defines the format of the source data, such as CSV, JSON, or Parquet.

any
Specifies format-specific properties as key-value pairs. For example, you can set a delimiter for CSV files.

advancedParameters

object
A nested object for specifying advanced configuration options as key-value pairs.

any
Specifies additional advanced parameters as key-value pairs.

Previous：GCSIngestionNext：GCSIngestionConfiguration