How to restrict specific IP addresses from accessing applications in an ASM instance - Alibaba Cloud Service Mesh

This topic describes how to restrict specific IP addresses from accessing applications in a Service Mesh (ASM) instance.

Prerequisites

A cluster is added to an ASM instance of v1.15.3.25 or later. For more information, see Add a cluster to an ASM instance.
Automatic sidecar proxy injection is enabled. For more information, see Enable automatic sidecar proxy injection.
The HTTPBin application is deployed and can be accessed. For more information, see Deploy the HTTPBin application.

Usage notes

In this example, the externalTrafficPolicy field of the gateway is set to Local. If the externalTrafficPolicy field is set to Cluster, source IP addresses are not preserved.

Scenario 1: No Layer 7 proxy is deployed between the client that initiates requests to access applications in the ASM instance and the ASM gateway

Run the following command to access the HTTPBin application:

curl 47.111.XXX.XX:80/ -I

Expand to view the logs of the gateway

{
    "authority_for": "47.111.XXX.XX",
    "bytes_received": "0",
    "bytes_sent": "0",
    "downstream_local_address": "10.0.70.17:80",
    "downstream_remote_address": "106.XX.XX.1:58656",
    "duration": "2",
    "istio_policy_status": "-",
    "method": "HEAD",
    "path": "/",
    "protocol": "HTTP/1.1",
    "request_id": "b1aa0770-4f64-4798-894a-4a04a0ba****",
    "requested_server_name": "-",
    "response_code": "200",
    "response_flags": "-",
    "route_name": "httpbin",
    "start_time": "2023-08-31T06:42:47.380Z",
    "trace_id": "-",
    "upstream_cluster": "outbound|8000||httpbin.default.svc.cluster.local",
    "upstream_host": "10.0.70.85:80",
    "upstream_local_address": "10.0.70.17:56110",
    "upstream_response_time": "2",
    "upstream_service_time": "2",
    "upstream_transport_failure_reason": "-",
    "user_agent": "curl/7.88.1",
    "x_forwarded_for": "106.11.XX.X"
}

The actual IP address of the client is 106.11.XX.X. You can see that the values of the downstream_remote_address and x_forwarded_for fields are correct in the gateway logs. In this case, the configurations of ipBlocks and remoteIpBlocks on the gateway can take effect.

Expand to view the access logs of the sidecar proxy

{
    "authority_for": "47.111.XXX.XX",
    "bytes_received": "0",
    "bytes_sent": "0",
    "downstream_local_address": "10.0.70.91:80",
    "downstream_remote_address": "106.11.XX.X:0",
    "duration": "1",
    "istio_policy_status": "-",
    "method": "HEAD",
    "path": "/",
    "protocol": "HTTP/1.1",
    "request_id": "20c94846-3e12-458a-b7d5-88ac867f****",
    "requested_server_name": "outbound_.8000_._.httpbin.default.svc.cluster.local",
    "response_code": "200",
    "response_flags": "-",
    "route_name": "default",
    "start_time": "2023-08-31T10:06:33.809Z",
    "trace_id": "-",
    "upstream_cluster": "inbound|80||",
    "upstream_host": "10.0.70.91:80",
    "upstream_local_address": "127.0.X.X:43247",
    "upstream_response_time": "1",
    "upstream_service_time": "1",
    "upstream_transport_failure_reason": "-",
    "user_agent": "curl/7.88.1",
    "x_forwarded_for": "106.11.XX.X"
}

You can see that the value of the x_forwarded_for field in the access logs of the sidecar proxy is the actual IP address of the client. The value of the downstream_remote_address field is also the actual IP address of the client, but the port information is lost and displayed as 0.

Example 1: Configure a blacklist or whitelist on the gateway

Test ipBlocks

Use the following content to create an authorization policy named gateway-test.yaml.

You can use ASM security policies or the blacklist or whitelist of the gateway to simplify configurations.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: gateway-test
  namespace: istio-system
spec:
  action: DENY
  rules:
    - from:
        - source:
            ipBlocks:
              - 106.11.XX.X
  selector:
    matchLabels:
      istio: ingressgateway

Run the following command to deploy the authorization policy:
```
kubectl apply -f gateway-test.yaml
```

Run the following command to access the HTTPBin application:

curl 47.111.175.XX:XX/ -I

Expected output:

HTTP/1.1 403 Forbidden
content-length: 19
content-type: text/plain
date: Thu, 31 Aug 2023 06:56:00 GMT
server: istio-envoy

Test remoteIpBlocks

Use the following content to create an authorization policy named gateway-test.yaml.

You can use ASM security policies or the blacklist or whitelist of the gateway to simplify configurations.

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: gateway-test
  namespace: istio-system
spec:
  action: DENY
  rules:
    - from:
        - source:
            remoteIpBlocks:
              - 106.11.XX.X
  selector:
    matchLabels:
      istio: ingressgateway

Run the following command to deploy the authorization policy:
```
kubectl apply -f gateway-test.yaml
```

Run the following command to access the HTTPBin application:

curl 47.111.175.XX:XX/ -I

Expected output:

HTTP/1.1 403 Forbidden
content-length: 19
content-type: text/plain
date: Thu, 31 Aug 2023 09:59:02 GMT
server: istio-envoy

Example 2: Configure a blacklist or whitelist on the sidecar proxy

Test ipBlocks

Use the following content to create an authorization policy named gateway-test.yaml:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: gateway-test
  namespace: default
spec:
  action: DENY
  rules:
    - from:
        - source:
            ipBlocks:
              - 106.11.XX.X
  selector:
    matchLabels:
      app: httpbin

Run the following command to deploy the authorization policy:
```
kubectl apply -f gateway-test.yaml
```
Run the following command to access the HTTPBin application:
```
curl 47.111.175.XX:XX/ -I
```
Expected output:
```
HTTP/1.1 200 OK
server: istio-envoy
date: Thu, 31 Aug 2023 10:14:01 GMT
content-type: text/html; charset=utf-8
content-length: 9593
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 2
```
You can see that the value of the downstream_remote_address field in the access logs of the sidecar proxy is the client IP address, which is not blocked.
The value of downstream_remote_address is not always an IP address specified in the ipBlocks field. The ipBlocks field specifies the IP addresses of the clients that are not allowed to directly establish TCP connections with the current proxy. In this case, the actual source IP addresses of the TCP connections are the pod IP addresses of the gateway. downstream_remote_address is a flexible field and its value is not always the physical IP address of the downstream connection.
The downstream_remote_address field indicates the remote address of the downstream connection. If the address is an IP address, it contains the IP address and port. It may not be the real physical address of the remote end. The address may be inferred from the Proxy Protocol filter or an x_forwarded_for request header.
You can replace the address in the ipBlocks field with the pod IP address of the gateway. After the configuration, you cannot access the HTTPBin application.

Test remoteIpBlocks

Use the following content to create an authorization policy named gateway-test.yaml:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: gateway-test
  namespace: default
spec:
  action: DENY
  rules:
    - from:
        - source:
            remoteIpBlocks:
              - 106.11.XX.X
  selector:
    matchLabels:
      app: httpbin

Run the following command to deploy the authorization policy:
```
kubectl apply -f gateway-test.yaml
```
Run the following command to access the HTTPBin application:
```
curl 47.111.XXX.XX:80/ -I
```
Expected output:
```
HTTP/1.1 403 Forbidden
content-length: 19
content-type: text/plain
date: Thu, 31 Aug 2023 11:06:42 GMT
server: istio-envoy
x-envoy-upstream-service-time: 0
```
The remoteIpBlocks configuration takes effect. In this case, the value of the x_forwarded_for field is the same as the value of the downstream_remote_address field.

Scenario 2: A Layer 7 proxy is deployed between the client that initiates requests to access applications in the ASM instance and the ASM gateway

In this scenario, the x_forwarded_for request header in the requests received by the ASM gateway should have a default value, which is added by the Layer 7 proxy.

Expand to view the gateway logs

{
    "authority_for": "47.111.XXX.XX",
    "bytes_received": "0",
    "bytes_sent": "0",
    "downstream_local_address": "10.0.70.17:80",
    "downstream_remote_address": "106.11.XX.X:62232",
    "duration": "4",
    "istio_policy_status": "-",
    "method": "HEAD",
    "path": "/",
    "protocol": "HTTP/1.1",
    "request_id": "4f8ee609-1479-4f75-ba9d-4fa6d646****",
    "requested_server_name": "-",
    "response_code": "200",
    "response_flags": "-",
    "route_name": "httpbin",
    "start_time": "2023-08-31T11:25:11.376Z",
    "trace_id": "-",
    "upstream_cluster": "outbound|8000||httpbin.default.svc.cluster.local",
    "upstream_host": "10.0.70.91:80",
    "upstream_local_address": "10.0.70.17:42646",
    "upstream_response_time": "4",
    "upstream_service_time": "4",
    "upstream_transport_failure_reason": "-",
    "user_agent": "curl/7.88.1",
    "x_forwarded_for": "56.5.X.X, 72.9.X.X, 98.1.X.X,106.11.XX.X"
}

Before the requests reach the ASM gateway, the value of the x_forwarded_for request header should be 56.5.X.X, 72.9.X.X, 98.1.X.X. The last IP address 106.11.XX.X is added by the ASM gateway. downstream_remote_address indicates the IP address of the peer end that directly connects to the gateway. In this case, it is a physical IP address and contains a valid port.

Expand to view the logs of the sidecar proxy

{
    "authority_for": "47.111.XXX.XX",
    "bytes_received": "0",
    "bytes_sent": "0",
    "downstream_local_address": "10.0.70.91:80",
    "downstream_remote_address": "106.11.XX.X:0",
    "duration": "1",
    "istio_policy_status": "-",
    "method": "HEAD",
    "path": "/",
    "protocol": "HTTP/1.1",
    "request_id": "4f8ee609-1479-4f75-ba9d-4fa6d646****",
    "requested_server_name": "outbound_.8000_._.httpbin.default.svc.cluster.local",
    "response_code": "200",
    "response_flags": "-",
    "route_name": "default",
    "start_time": "2023-08-31T11:25:11.378Z",
    "trace_id": "-",
    "upstream_cluster": "inbound|80||",
    "upstream_host": "10.0.70.91:80",
    "upstream_local_address": "127.0.X.X:34607",
    "upstream_response_time": "1",
    "upstream_service_time": "1",
    "upstream_transport_failure_reason": "-",
    "user_agent": "curl/7.88.1",
    "x_forwarded_for": "56.5.X.X, 72.9.X.X, 98.1.X.X,106.11.XX.X"
}

The logs of the sidecar proxy indicate that the value of the x_forwarded_for request header is the same as that of the gateway. The value of downstream_remote_address is not the actual physical address. It is the last IP address in the value of the x_forwarded_for request header. (The x_forwarded_for field contains only IP addresses. Therefore, the port here is 0.)

The following section does not test the ipBlocks field (this field indicates the actual physical IP address of the peer end). Only the remoteIpBlocks field is tested. In this case, you need to treat the x_forwarded_for field as an array. It rejects an IP address from accessing the ASM gateway. The last IP address in x_forwarded_for in the preceding logs is not allowed to access the ASM gateway.

Example 1: Set remoteIpBlocks to the last IP address in the x_forwarded_for field

Use the following content to create an authorization policy named gateway-test.yaml:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: gateway-test-ap-wg-gateway-test-istio-system-gateway-ingressgateway
  namespace: istio-system
spec:
  action: DENY
  rules:
    - from:
        - source:
            remoteIpBlocks:
              - 106.11.XX.X
  selector:
    matchLabels:
      istio: ingressgateway

Run the following command to deploy the authorization policy:
```
kubectl apply -f gateway-test.yaml
```

Run the following command to access the HTTPBin application:

curl 47.111.175.XX:XX/ -H 'X-Forwarded-For: 56.5.X.X, 72.9.X.X, 98.1.X.X' -I

Expected output:

HTTP/1.1 403 Forbidden
content-length: 19
content-type: text/plain
date: Thu, 31 Aug 2023 11:50:36 GMT
server: istio-envoy

The output indicates that the last IP address in the x_forwarded_for field is not allowed to access the ASM gateway.

Example 2: Set the numTrustedProxies field of the ingress gateway to 2

Add the following configuration to the spec field in the YAML file of the ingress gateway. For more information about how to edit the YAML file of an ingress gateway, see Manage the ingress gateway in the ASM console.
```
podAnnotations:
    proxy.istio.io/config: '{"gatewayTopology" : { "numTrustedProxies": 2 } }'
```
Note
This configuration causes the gateway to restart.
After the gateway is restarted, run the following command to access the HTTPBin application:
```
curl 47.111.XXX.XX:80/ -H 'X-Forwarded-For: 56.5.X.X, 72.9.X.X, 98.1.X.X' -I
```
Expected output:
```
HTTP/1.1 200 OK
server: istio-envoy
date: Thu, 31 Aug 2023 12:10:15 GMT
content-type: text/html; charset=utf-8
content-length: 9593
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 2
```
The output indicates that the access is successful and the value of the numTrustedProxies field affects the judgment results of the remoteIpBlocks field.
When numTrustedProxies is set to 2, the ASM gateway considers the two proxies closest to itself to be trusted and the third closest proxy to be untrustworthy. Therefore, the remoteIpBlocks field rejects the IP address of the third closest proxy from accessing the gateway. If numTrustedProxies is not set, the default value is 0.

Example 3: Use an authorization policy to reject access from the third-to-last IP address in the x_forwarded_for field of the gateway logs

Use the following content to create an authorization policy named gateway-test.yaml:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: gateway-test-ap-wg-gateway-test-istio-system-gateway-ingressgateway
  namespace: istio-system
spec:
  action: DENY
  rules:
    - from:
        - source:
            remoteIpBlocks:
              - 72.9.X.X
  selector:
    matchLabels:
      istio: ingressgateway

Run the following command to deploy the authorization policy:
```
kubectl apply -f gateway-test.yaml
```

Run the following command to access the HTTPBin application:

curl 47.111.XXX.XX:80/ -H 'X-Forwarded-For: 56.5.X.X, 72.9.X.X, 98.1.X.X' -I

Expected output:

HTTP/1.1 403 Forbidden
content-length: 19
content-type: text/plain
date: Thu, 31 Aug 2023 12:38:09 GMT
server: istio-envoy

The output indicates that access from the third-to-last IP address specified in the x_forwarded_for field is rejected.

Example 4: Change the effective scope of the preceding authorization policy to the HTTPBin application

Use the following content to create an authorization policy named gateway-test.yaml:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: gateway-test
  namespace: default
spec:
  action: DENY
  rules:
    - from:
        - source:
            remoteIpBlocks:
              - 72.9.X.X
  selector:
    matchLabels:
      app: httpbin

Run the following command to deploy the authorization policy:
```
kubectl apply -f gateway-test.yaml
```
Run the following command to access the HTTPBin application:
```
curl 47.111.XXX.XX:80/ -H 'X-Forwarded-For: 56.5.X.X, 72.9.X.X, 98.1.X.X' -I
```
Expected output:
```
HTTP/1.1 200 OK
server: istio-envoy
date: Thu, 31 Aug 2023 12:39:36 GMT
content-type: text/html; charset=utf-8
content-length: 9593
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 6
```
The output indicates that the access is successful and the authorization policy is invalid. The preceding numTrustedProxies field is configured for the gateway. The default value of numTrustedProxies for the sidecar proxy is still 0. The sidecar proxy verifies the last IP address of the received x_forwarded_for request header. Therefore, the request is not rejected.
If you change the value of remoteIpBlocks in the authorization policy to the last IP address of x_forwarded_for in the sidecar proxy logs, the access is rejected.