Doris supports multiple data sources, formats, and import features to help you load data efficiently.
Supported data sources
Doris provides multiple data import methods. Select a method based on your data source.
Supported data formats
The supported data formats vary by import method.
|
Import method |
Supported formats |
|
Broker Load |
Parquet, ORC, CSV, GZIP |
|
Stream Load |
CSV, GZIP, JSON |
|
Routine Load |
CSV, JSON |
Import details
Doris data import has the following features.
Atomicity of imports
Every import job in Doris, whether it is a batch import using Broker Load or a single-row import using an INSERT statement, is a complete transaction. An import transaction ensures that all data in a batch is committed atomically, preventing partial data writes.
Each import job has a label. A label is unique within a database and uniquely identifies an import job. You can specify a label, or the system can generate one automatically.
A label ensures that the corresponding import job succeeds only once. If you reuse a label from a successful import, the request is rejected with the error Label already used. This mechanism provides At-Most-Once semantics in Doris. Combined with the At-Least-Once semantics of an upstream system, you can achieve Exactly-Once semantics for data import.
Sync and async imports
Import methods are either synchronous or asynchronous. For synchronous imports, the returned result indicates whether the import succeeded or failed. For asynchronous imports, a successful response indicates only that the job was submitted, not that the data was imported. Use the appropriate command to check the import job status.