Before creating an Airflow instance in Data Management Service (DMS), set up the cloud resources it depends on. This page explains what to prepare and how to configure Internet access if your workflows need it.
What you'll need
| Resource | Purpose | Required |
|---|---|---|
| Object Storage Service (OSS) bucket | Stores Airflow logs, dependent Python libraries, and task output data | Yes |
| Internet NAT gateway + Elastic IP Address (EIP) | Routes outbound Internet traffic from the Airflow VPC | Only if your Airflow instance needs Internet access |
All resources must be in the same region as your Airflow instance. For supported regions, see Supported regions and zones.
Create an OSS bucket
DMS automatically retrieves bucket information from the OSS console. No manual registration is needed.
Go to the Buckets page and click Create Bucket.
Enter a bucket name and select the same region as your Airflow instance.
Scroll to the bottom and click Create. Follow the on-screen instructions to complete the process. For more information, see Quick Start.
In the bucket, create a folder to store Airflow data.
Enable Internet access for Airflow (optional)
Skip this section if your Airflow instance does not need to access the Internet.
To route outbound traffic from the Airflow VPC to the Internet, create an Internet NAT gateway and associate it with an EIP. This involves two steps: getting the VPC ID of your workspace, then creating the NAT gateway in that VPC.
Get the VPC ID of your workspace
Log on to the DMS console V5.0.
Go to the Workspace page: click the
icon in the upper-left corner and choose All Features > Data+AI > Workspace.If you are not using the simplified console, choose Data+AI > Workspace from the top menu bar.
Click the name of the target workspace.
In the lower-left corner, click the Settings
icon, select Workspace Management, and note the VPC ID.
Create an Internet NAT gateway
Log on to the VPC console.
In the navigation pane, click NAT Gateway > Internet NAT Gateway.
On the Internet NAT Gateway page, click Create Internet NAT Gateway.
On the purchase page, set the following parameters and click Buy Now in the lower-right corner.
Parameter Description Instance Name Name of the Internet NAT gateway instance Region Must match the region of your workspace Network and Zone Must match the VPC of your workspace Network Type Select Internet NAT Gateway EIP Select Select EIP if you already have an EIP in the same region. Otherwise, click Purchase EIP On the Parameters page, verify the settings and click Activate Now.
Wait for the purchase to complete. The Purchase succeeded! message confirms the gateway is ready.