Community Blog A Tangled Web: Advanced Networking on Alibaba Cloud - Friday Blog, Week 58

A Tangled Web: Advanced Networking on Alibaba Cloud - Friday Blog, Week 58

Learn about the different ways that Alibaba Cloud VPCs can connect to the public Internet, and how you can limit access when needed.

By: Jeremy Pedersen

It's time to start getting deeper into a subject I have mostly avoided: cloud networking.

Don't get me wrong: I love networking, it's a fascinating subject! It's just so...big.

To make things simpler, I'll be breaking this subject down in a multipart series. Why? Because networking is complicated and there are way too many networking tools to fit into one blog post!

There are generally three classes of networking tools on Alibaba Cloud:

  1. Tools that help you publicly connect to Alibaba Cloud resources
  2. Tools that help you privately connect to Alibaba Cloud resources
  3. Tools that help you interconnect Alibaba Cloud resources.

Today we will be focusing on the first area: public Internet connections. Specifically, I want to talk about routing and security. How can you decide which systems are Internet accessible and which ones are not? More importantly, how do you enforce it?

Basic Background: VPCs

Before we can talk about connecting things to the Internet, we have to talk about VPCs, or Virtual Private Clouds.

The VPC serves as the network foundation on which other Alibaba Cloud services depend. Except for some services that are always designed to be accessible via the Internet (like Object Storage Service, most Alibaba Cloud services will need to connect to a VPC group to function. This includes things like virtual machines (ECS instances), databases (RDS instances), and load balancers (ALB and CLB instances).

Today, I want to focus on how things within the VPC connect to the Internet, and how you can control that access.

VPC Connectivity Basics: VPC Route Table

If you've got experience with Amazon AWS, you'll know that their VPC has something called an Internet Gateway.

This makes controlling Internet access something you can do at the VPC level:

  • If a subnet in your AWS VPC has a route to the Internet Gateway, then EC2 instances in that subnet can access the Internet
  • If a subnet does not have a route to the Internet Gateway, then no EC2 instance in the subnet can access the Internet

Simple! This makes it easy to create fully private VPCs, fully public VPCs, and VPCs where some subnets route to the Internet and some do not.

Sadly, this is not how it works on Alibaba Cloud.

We can see this just by taking a glance at the routing tables for an AWS VPC and an Alibaba Cloud VPC.



The Alibaba Cloud VPC:


Notice that the AWS VPC has an explicit route to the Internet gateway (plus another local route for traffic between subnets), whereas the Alibaba Cloud VPC only has routes for its three subnets (called VSwitches) and one route for Alibaba Cloud internal services.

Bottom line: Alibaba Cloud VPCs don't have an internet "route", because by default all Alibaba Cloud VPCs allow connections to the Internet!

This is a pretty major difference that can trip up experienced AWS users.

This means that on Alibaba Cloud, controlling Internet access is about controlling the ability to create public IP addresses, not the ability to control VPC routing tables.

We'll dive deeper into how this works in the next few sections.

Count The Ways: Reaching The Internet From A VPC

We'll focus on ECS instance Internet access for now, saving other services for a future blog post.

Essentially, there are four ways an ECS instance can reach the Internet.

  1. A static public IP, assigned at ECS instance creation time.
  2. An Elastic IP (EIP), which can be added to an ECS instance at any time.
  3. A NAT Gateway, both for outbound (SNAT) and inbound (DNAT) traffic.
  4. A Server Load Balancer (CLB or ALB), for inbound traffic only.

Let's look at each of these in turn, and see what we can do to restrict access in each case.

Case 1: Restricting ECS Public IPs

The normal way to restrict operations within an Alibaba Cloud account is to create a RAM user or RAM Role and then restrict the operations it can perform using RAM policy.

The ECS API breaks the ECS instance creation process into two API calls:

  1. CreateInstance
  2. AllocatePublicIpAddress

Creating a RAM Policy that Denies AllocatePublicIPAddress will prevent a user from binding a public IP to new ECS instances.

Note: This is not enough to guarantee that an ECS instance can't reach the internet. We've still got to worry about NAT Gateways, Elastic IPs, and Load Balancers! Read on.

Case 2: Restrict Who Can Create And Bind EIPs

Simply making sure a user can't call AllocatePublicIPAddress isn't enough to guarantee that an ECS instance isn't granted a public IP address, because Elastic IPs are bound to ECS instances through a separate set of API calls.

Making sure a user cannot create and bind EIPs would require also restricting access to the EIP service. At a minimum, your RAM policy would need to restrict access to make the following API calls:

  • AllocateEipAddress
  • AllocateEipAddressPro
  • AllocateEipSegmentAddress
  • AssociateEipAddress
  • ModifyEipAddressAttribute

It would probably be wise to restrict the ability to unbind and delete EIPs as well, for those users who do not need that level of access.

This takes care of direct access to the Internet. But what about indirect access via a NAT Gateway or Load Balancer?

Case 3: Restricting Access Via Server Load Balancer

Restricting ECS permissions won't prevent an ECS instance from being added to a load balancer's backend server pool, but we can restrict the ability of a RAM user or RAM role to:

  1. Create new load balancers
  2. Associate servers with a load balancer's backend server group

How this is done will depend on whether we're talking about Application Load Balancer (ALB) or Classic Load Balancer (CLB). The load balancer API reference tells us which API calls we'd need to restrict.

Writing a custom RAM policy for this is left as an exercise for the reader!

Case 4: Restricting Access Via NAT Gateway

Setting up a NAT Gateway gives you a lot of control over which ECS instances can reach the Internet (and which instances are reachable from the Internet as well).

NAT Gateways allow you to create two types of rules:

  • SNAT Rules, which control traffic out to the Internet
  • DNAT Rules, which control traffic in from the Internet

SNAT Rule Configuration

SNAT rules can be configured to allow outbound traffic for:

  1. Every VSwitch within the VPC
  2. A single VSwitch
  3. A custom CIDR address range
  4. A single ECS instance

This means you can use NAT Gateway to create a VPC that contains "public" and "private" VSwitches (a VSwitch is a network subnet), just by controlling the SNAT rule configuration.

So long as you are careful to restrict RAM users (so that they cannot create EIPs, bind ECS public IPs, or alter NAT rules) then you can exercise complete control over which parts of your VPC have Internet access, using this approach.

DNAT Configuration

DNAT rules also give you a convenient way to expose a particular private IP address to traffic from the Internet. For instance, if you need to gain access to an ECS instance from the Internet for administrative tasks, you could create a DNAT rule that forwards traffic on port 22 (SSH) to a particular private IP inside your VPC.

The NAT Gateway documentation is a good source of configuration examples.

Other Ways To Control Access

What if you want to put rules in place that can further restrict public Internet access, even when instances have been granted a public IP address? There are two ways:

  1. Security Group rules
  2. VPC Group Network ACL (NACL) rules

Method 1: Outbound Security Group Rules

Setting strict outbound security rules is one potential way to limit Internet access for ECS instances, even if they do have a public IP address:



  • Works for both standard ECS public IPs and EIPs
  • Works for all ECS instance generations and types


  • Not fine-grained (rules are stateful and applied at the VPC level, not the VSwitch level)
  • To be effective, you have to limit which users can create, bind, unbind and edit Security Groups
  • If a user has CreateInstance permissions, you need to make sure they aren't able to create a new Security Group at ECS instance creation time (which would circumvent your Security Group rules)

Method 2: Network ACL Rules

This method gives you stateless, VSwitch-level control over exactly which IP address ranges are accessible to an ECS instance, no matter how its network interfaces are configured.

Better, since these settings apply at the VSwitch level, users who do not have permission to access VPC settings cannot circumvent them, even if they have permission to create ECS instances.


  • Stateless
  • Fine-grained (operates at the VSwitch level)
  • Overrides Security Group rules


  • More difficult to use than Security Group rules
  • Does not work for all ECS instance types

That last point is a critical one: since Network ACL doesn't work for all instance classes, a user can create an ECS instance from a class that doesn't support Network ACL, which will result in any Network ACL rules you have created failing to take effect for that instance.

Other Ways To Limit Access

Restricting Internet access in an effective way requires a combination of active measures to:

  1. Restrict the ability to create public egress and ingress points (EIP, NAT Gateway, Server Load Balancer, public ECS IP)
  2. Restrict the ability of traffic to leave the VPC network (Network ACL, Security Groups, NAT Gateway SNAT rules)

But you can go further than this and adopt passive and corrective measures. For instance, creating a set of Cloud Config rules that can check for open Security Groups or instances which have been granted public IP addresses.

Tip: Using a three-pronged strategy (limit egress points, restrict traffic flow, and check for configuration changes) is the best way to guarantee only the traffic you want to allow is reaching the Internet.

That's it for this week, see you next time!

I've Got A Question!

Great! Reach out to me at jierui.pjr@alibabacloud.com and I'll do my best to answer in a future Friday Q&A blog.

You can also follow the Alibaba Cloud Academy LinkedIn Page. We'll re-post these blogs there each Friday.

Not a LinkedIn person? We're also on Twitter and YouTube.

0 1 0
Share on


72 posts | 135 followers

You may also like



72 posts | 135 followers

Related Products