全部产品
Search
文档中心

服务网格 ASM:限制特定IP访问网格内应用

更新时间:Oct 09, 2023

本文介绍如何限制特定IP访问网格内应用。

前提条件

使用说明

本文网关的externalTrafficPolicy均设置为LocalexternalTrafficPolicy设置为Cluster无法保留源IP。

场景一:ASM网关之前不存在7层代理

执行以下命令,直接访问httpbin。

curl 47.111.XXX.XX:80/ -I

展开查看网关侧产生的日志

{
    "authority_for": "47.111.XXX.XX",
    "bytes_received": "0",
    "bytes_sent": "0",
    "downstream_local_address": "10.0.70.17:80",
    "downstream_remote_address": "106.XX.XX.1:58656",
    "duration": "2",
    "istio_policy_status": "-",
    "method": "HEAD",
    "path": "/",
    "protocol": "HTTP/1.1",
    "request_id": "b1aa0770-4f64-4798-894a-4a04a0ba****",
    "requested_server_name": "-",
    "response_code": "200",
    "response_flags": "-",
    "route_name": "httpbin",
    "start_time": "2023-08-31T06:42:47.380Z",
    "trace_id": "-",
    "upstream_cluster": "outbound|8000||httpbin.default.svc.cluster.local",
    "upstream_host": "10.0.70.85:80",
    "upstream_local_address": "10.0.70.17:56110",
    "upstream_response_time": "2",
    "upstream_service_time": "2",
    "upstream_transport_failure_reason": "-",
    "user_agent": "curl/7.88.1",
    "x_forwarded_for": "106.11.XX.X"
}

客户端实际IP为106.11.XX.X。从网关日志可以看到downstream_remote_addressx_forwarded_for中的值都正确。此时,在网关上配置ipBlocksremoteIpBlocks都可以生效。

展开查看Sidecar侧的访问日志

{
    "authority_for": "47.111.XXX.XX",
    "bytes_received": "0",
    "bytes_sent": "0",
    "downstream_local_address": "10.0.70.91:80",
    "downstream_remote_address": "106.11.XX.X:0",
    "duration": "1",
    "istio_policy_status": "-",
    "method": "HEAD",
    "path": "/",
    "protocol": "HTTP/1.1",
    "request_id": "20c94846-3e12-458a-b7d5-88ac867f****",
    "requested_server_name": "outbound_.8000_._.httpbin.default.svc.cluster.local",
    "response_code": "200",
    "response_flags": "-",
    "route_name": "default",
    "start_time": "2023-08-31T10:06:33.809Z",
    "trace_id": "-",
    "upstream_cluster": "inbound|80||",
    "upstream_host": "10.0.70.91:80",
    "upstream_local_address": "127.0.X.X:43247",
    "upstream_response_time": "1",
    "upstream_service_time": "1",
    "upstream_transport_failure_reason": "-",
    "user_agent": "curl/7.88.1",
    "x_forwarded_for": "106.11.XX.X"
}

从Sidecar侧的访问日志可以看到x_forwarded_for中的值依然为客户端真实IP。downstream_remote_address也是客户端真实IP,但是丢失了端口信息,显示为0

示例一:网关侧配置黑白名单

测试ipBlocks

  1. 使用以下内容,创建授权策略gateway-test.yaml。

    您可以使用ASM安全策略或网关的黑白名单简化配置。

    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: gateway-test
      namespace: istio-system
    spec:
      action: DENY
      rules:
        - from:
            - source:
                ipBlocks:
                  - 106.11.XX.X
      selector:
        matchLabels:
          istio: ingressgateway
  2. 执行以下命令,部署授权策略。

    kubectl apply -f gateway-test.yaml
  3. 执行以下命令,访问httpbin。

    curl 47.111.175.XX:XX/ -I

    预期输出:

    HTTP/1.1 403 Forbidden
    content-length: 19
    content-type: text/plain
    date: Thu, 31 Aug 2023 06:56:00 GMT
    server: istio-envoy

测试remoteIpBlocks

  1. 使用以下内容,创建授权策略gateway-test.yaml。

    您可以使用ASM安全策略或网关的黑白名单简化配置。

    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: gateway-test
      namespace: istio-system
    spec:
      action: DENY
      rules:
        - from:
            - source:
                remoteIpBlocks:
                  - 106.11.XX.X
      selector:
        matchLabels:
          istio: ingressgateway
    
  2. 执行以下命令,部署授权策略。

    kubectl apply -f gateway-test.yaml
  3. 执行以下命令,访问httpbin。

    curl 47.111.175.XX:XX/ -I

    预期输出:

    HTTP/1.1 403 Forbidden
    content-length: 19
    content-type: text/plain
    date: Thu, 31 Aug 2023 09:59:02 GMT
    server: istio-envoy

示例二:Sidecar侧配置黑白名单

测试ipBlocks

  1. 使用以下内容,创建授权策略gateway-test.yaml。

    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: gateway-test
      namespace: default
    spec:
      action: DENY
      rules:
        - from:
            - source:
                ipBlocks:
                  - 106.11.XX.X
      selector:
        matchLabels:
          app: httpbin
  2. 执行以下命令,部署授权策略。

    kubectl apply -f gateway-test.yaml
  3. 执行以下命令,访问httpbin。

    curl 47.111.175.XX:XX/ -I

    预期输出:

    HTTP/1.1 200 OK
    server: istio-envoy
    date: Thu, 31 Aug 2023 10:14:01 GMT
    content-type: text/html; charset=utf-8
    content-length: 9593
    access-control-allow-origin: *
    access-control-allow-credentials: true
    x-envoy-upstream-service-time: 2

    从Sidecar侧的访问日志可以看到,downstream_remote_address是客户端IP,但是没有被拦截。

    downstream_remote_address并不完全等于ipBlocks中限制的字段。ipBlocks限制“和当前代理直接建立TCP连接的客户端IP”,所以真实发送过来的源IP是网关的Pod IP。downstream_remote_address是一个较为灵活的字段,并不总是下游的物理IP。

    downstream_remote_address表示下游连接的远端地址。如果该地址是一个IP地址,则它包含IP和端口。它不是远端的真实物理地址,该地址可能来自于ProxyProtocol过滤器或x_forwarded_for请求头。

    您可以测试将ipBlocks中的地址换成网关的Pod IP,配置后将无法访问httpbin。

测试remoteIpBlocks

  1. 使用以下内容,创建授权策略gateway-test.yaml。

    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: gateway-test
      namespace: default
    spec:
      action: DENY
      rules:
        - from:
            - source:
                remoteIpBlocks:
                  - 106.11.XX.X
      selector:
        matchLabels:
          app: httpbin
  2. 执行以下命令,部署授权策略。

    kubectl apply -f gateway-test.yaml
  3. 执行以下命令,访问httpbin。

    curl 47.111.XXX.XX:80/ -I

    预期输出:

    HTTP/1.1 403 Forbidden
    content-length: 19
    content-type: text/plain
    date: Thu, 31 Aug 2023 11:06:42 GMT
    server: istio-envoy
    x-envoy-upstream-service-time: 0

    remoteIpBlocks配置生效,因为x_forwarded_for中的值正常。

场景二:ASM网关之前存在7层代理

此场景下,ASM网关收到的请求中,x_forwarded_for请求头中应有默认值(由网关之前的7层代理提添加)。

展开查看网关侧日志

{
    "authority_for": "47.111.XXX.XX",
    "bytes_received": "0",
    "bytes_sent": "0",
    "downstream_local_address": "10.0.70.17:80",
    "downstream_remote_address": "106.11.XX.X:62232",
    "duration": "4",
    "istio_policy_status": "-",
    "method": "HEAD",
    "path": "/",
    "protocol": "HTTP/1.1",
    "request_id": "4f8ee609-1479-4f75-ba9d-4fa6d646****",
    "requested_server_name": "-",
    "response_code": "200",
    "response_flags": "-",
    "route_name": "httpbin",
    "start_time": "2023-08-31T11:25:11.376Z",
    "trace_id": "-",
    "upstream_cluster": "outbound|8000||httpbin.default.svc.cluster.local",
    "upstream_host": "10.0.70.91:80",
    "upstream_local_address": "10.0.70.17:42646",
    "upstream_response_time": "4",
    "upstream_service_time": "4",
    "upstream_transport_failure_reason": "-",
    "user_agent": "curl/7.88.1",
    "x_forwarded_for": "56.5.X.X, 72.9.X.X, 98.1.X.X,106.11.XX.X"
}

到达网关之前的x_forwarded_for请求头应为56.5.X.X, 72.9.X.X, 98.1.X.X,最后的一个IP(106.11.XX.X)由ASM网关添加。downstream_remote_address表示与网关直接相连的对端IP(此时是真实的物理IP,所以是一个有效的端口)。

展开查看Sidecar侧日志

{
    "authority_for": "47.111.XXX.XX",
    "bytes_received": "0",
    "bytes_sent": "0",
    "downstream_local_address": "10.0.70.91:80",
    "downstream_remote_address": "106.11.XX.X:0",
    "duration": "1",
    "istio_policy_status": "-",
    "method": "HEAD",
    "path": "/",
    "protocol": "HTTP/1.1",
    "request_id": "4f8ee609-1479-4f75-ba9d-4fa6d646****",
    "requested_server_name": "outbound_.8000_._.httpbin.default.svc.cluster.local",
    "response_code": "200",
    "response_flags": "-",
    "route_name": "default",
    "start_time": "2023-08-31T11:25:11.378Z",
    "trace_id": "-",
    "upstream_cluster": "inbound|80||",
    "upstream_host": "10.0.70.91:80",
    "upstream_local_address": "127.0.X.X:34607",
    "upstream_response_time": "1",
    "upstream_service_time": "1",
    "upstream_transport_failure_reason": "-",
    "user_agent": "curl/7.88.1",
    "x_forwarded_for": "56.5.X.X, 72.9.X.X, 98.1.X.X,106.11.XX.X"
}

从Sidecar的日志可以看到,x_forwarded_for请求头和网关侧相同。downstream_remote_address此时并不是真实的物理地址,而是来自于x_forwarded_for请求头的最后一个值(由于x_forwarded_for中只有IP,所以此处端口为0)。

下文不再测试ipBlocks字段(该字段表示对端真实物理IP),仅测试remoteIpBlocks字段。这种情况下,需要将x_forwarded_for看作一个数组,此时限制“谁可以访问ASM网关”,即限制上述日志中x_forwarded_for的最后一个元素。

示例一:配置remoteIpBlocks为x_forwarded_for中的最后一个地址

  1. 使用以下内容,创建授权策略gateway-test.yaml。

    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: gateway-test-ap-wg-gateway-test-istio-system-gateway-ingressgateway
      namespace: istio-system
    spec:
      action: DENY
      rules:
        - from:
            - source:
                remoteIpBlocks:
                  - 106.11.XX.X
      selector:
        matchLabels:
          istio: ingressgateway
  2. 执行以下命令,部署授权策略。

    kubectl apply -f gateway-test.yaml
  3. 执行以下命令,访问httpbin。

    curl 47.111.175.XX:XX/ -H 'X-Forwarded-For: 56.5.X.X, 72.9.X.X, 98.1.X.X' -I

    预期输出:

    HTTP/1.1 403 Forbidden
    content-length: 19
    content-type: text/plain
    date: Thu, 31 Aug 2023 11:50:36 GMT
    server: istio-envoy

    预期输出表明成功限制x_forwarded_for中的最后一个地址。

示例二:设置网关的numTrustedProxies为2

  1. 在入口网关YAML的spec字段中添加如下配置。关于如何编辑入口网关,请参见登录ASM控制台管理入口网关

    podAnnotations:
        proxy.istio.io/config: '{"gatewayTopology" : { "numTrustedProxies": 2 } }'
    说明

    该配置会导致网关重启。

  2. 网关重启后,执行以下命令,访问httpbin。

    curl 47.111.XXX.XX:80/ -H 'X-Forwarded-For: 56.5.X.X, 72.9.X.X, 98.1.X.X' -I

    预期输出:

    HTTP/1.1 200 OK
    server: istio-envoy
    date: Thu, 31 Aug 2023 12:10:15 GMT
    content-type: text/html; charset=utf-8
    content-length: 9593
    access-control-allow-origin: *
    access-control-allow-credentials: true
    x-envoy-upstream-service-time: 2

    预期输出表明访问成功,numTrustedProxies影响了remoteIpBlocks中的判断结果。

    numTrustedProxies设置为2,表示网关会认为最靠近ASM网关的2个代理是可信的,第三近的代理不可信。因此,remoteIpBlocks限制第三近的代理IP。numTrustedProxies在不配置的情况下,默认值为0。

示例三:授权策略中限制网关日志x_forwarded_for的倒数第三个地址

  1. 使用以下内容,创建授权策略gateway-test.yaml。

    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: gateway-test-ap-wg-gateway-test-istio-system-gateway-ingressgateway
      namespace: istio-system
    spec:
      action: DENY
      rules:
        - from:
            - source:
                remoteIpBlocks:
                  - 72.9.X.X
      selector:
        matchLabels:
          istio: ingressgateway
  2. 执行以下命令,部署授权策略。

    kubectl apply -f gateway-test.yaml
  3. 执行以下命令,访问httpbin。

    curl 47.111.XXX.XX:80/ -H 'X-Forwarded-For: 56.5.X.X, 72.9.X.X, 98.1.X.X' -I

    预期输出:

    HTTP/1.1 403 Forbidden
    content-length: 19
    content-type: text/plain
    date: Thu, 31 Aug 2023 12:38:09 GMT
    server: istio-envoy

    预期输出表明成功限制x_forwarded_for的倒数第三个地址。

示例四:变更上述授权策略的生效范围为httpbin应用

  1. 使用以下内容,创建授权策略gateway-test.yaml。

    apiVersion: security.istio.io/v1beta1
    kind: AuthorizationPolicy
    metadata:
      name: gateway-test
      namespace: default
    spec:
      action: DENY
      rules:
        - from:
            - source:
                remoteIpBlocks:
                  - 72.9.X.X
      selector:
        matchLabels:
          app: httpbin
  2. 执行以下命令,部署授权策略。

    kubectl apply -f gateway-test.yaml
  3. 执行以下命令,访问httpbin。

    curl 47.111.XXX.XX:80/ -H 'X-Forwarded-For: 56.5.X.X, 72.9.X.X, 98.1.X.X' -I

    预期输出:

    HTTP/1.1 200 OK
    server: istio-envoy
    date: Thu, 31 Aug 2023 12:39:36 GMT
    content-type: text/html; charset=utf-8
    content-length: 9593
    access-control-allow-origin: *
    access-control-allow-credentials: true
    x-envoy-upstream-service-time: 6

    预期输出表明访问成功,授权策略规则失效。因为上述numTrustedProxies是为网关配置的,Sidecar的numTrustedProxies默认值还是0,它会校验自己收到的x_forwarded_for请求头的最后一个值,因此没有成功拒绝当前请求。

    您可以修改该授权策略中的remoteIpBlocks为Sidecar日志中x_forwarded_for的最后一个IP,即可看到访问被拒绝。