6. Cross-language microservice framework-Istio Ingress and Egress details (solving the problem that Istio cannot access the Internet)-Alibaba Cloud Developer Community

Another key point in microservices is gateway. The gateway theory includes the inbound gateway and the outbound gateway. In the traditional sense, it is difficult to control the outbound network, however, it is very easy for Istio (because all outbound traffic passes through Istio). The Ingress gateway controls the flow direction of parsed route data, and the egress gateway controls access restrictions, ingress and Egress are used in Istio to implement gateway functions.

Attached:

meow's blog: w-blog.cn

Istio official address: https://preliminary.istio.io/zh

Istio Chinese documentation: https://preliminary.istio.io/zh/docs/

PS: build and demonstrate the latest istio version 1.0.3.

1. Ingress (Portal Gateway)

the Istio gateway runs and configures routing rules and how traffic enters the cluster. We use httpbin as an experimental project.

>kubectl apply -n istio-test -f istio-1.0.3/samples/httpbin/httpbin.yaml

1. Determine the entry IP address and port

run the following command to determine whether your Kubernetes cluster is running in an environment that supports external load balancer.

> kubectl get svc istio-ingressgateway -n istio-system
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)                                                                                                                   AGE
istio-ingressgateway   LoadBalancer   10.43.92.244   172.16.0.203   80:31380/TCP,443:31390/TCP,31400:31400/TCP,15011:30921/TCP,8060:30126/TCP,853:30117/TCP,15030:31865/TCP,15031:30683/TCP   22h

If the environment no longer uses server load balancer in the internal network, you need to use the corresponding IP address and port. The author uses the internal network, so the corresponding access can be through any node

1. use Istio gateway to configure Ingress

Ingress Gateway describes the load balancer that operates at the edge of the grid and is used to receive incoming HTTP/TCP connections. It configures exposed ports, protocols, and so on, but unlike Kubernetes Ingress Resources, it does not include any traffic routing configurations. The inbound traffic is configured using Istio routing rules, which are exactly the same as internal service requests.

Let's see how to configure traffic for Gateway on HTTP port 80.

kubectl apply -n istio-test -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "httpbin.example.com"
EOF

Configure the route for the traffic that enters through the Gateway:

kubectl apply -n istio-test -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "httpbin.example.com"
  gateways:
  - httpbin-gateway
  http:
  - match:
    - uri:
        prefix: /status
    - uri:
        prefix: /delay
    route:
    - destination:
        port:
          number: 8000
        host: httpbin
EOF

here, we create a virtual service configuration httpbin for the service, which contains two routing rules, allowing path/status and path traffic/delay.

The Gateway list specifies that only httpbin-gateway is allowed. All other external requests are rejected and a 404 response is returned.

Note that in this configuration, internal requests from other services in the grid are not bound by these rules, but are simply circular routes by default. To apply these (or other rules) to internal calls, we can add special values mesh to the list of gateways.

1. Use curl to access the httpbin service:

curl -I -HHost:httpbin.example.com http://172.16.0.203:31380/status/200
HTTP/1.1 200 OK
server: envoy
date: Thu, 08 Nov 2018 02:35:52 GMT
content-type: text/html; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 0
x-envoy-upstream-service-time: 46

note that the-H flag is used here to set the Host HTTP Header to httpbin.example.com ". This operation is required because the above Ingress Gateway is configured to process "httpbin.example.com", but there is no DNS binding for the host in the Test environment, only sending the request to the Ingress IP.

1. If you access any other URL that is not explicitly disclosed, you will see an HTTP 404 error:

curl -I -HHost:httpbin.example.com http://172.16.0.203:31380/headers
HTTP/1.1 404 Not Found
date: Thu, 08 Nov 2018 02:36:32 GMT
server: envoy
transfer-encoding: chunked

2. Egress (Egress Gateway)

the Ingress gateway is well understood by everyone. Isn't it an NGINX domain name resolution routing control? What's the use of your egress gateway? In today's increasingly refined O & M Management, the control of export traffic is becoming more and more important. What can be accessed and what cannot be accessed should be determined for each program, such restrictions can avoid abnormal traffic and external attacks.

By default, pods in the Istio service mesh cannot access URLs outside the cluster because their iptables transparently forward all outbound traffic to Sidecar, you can only process the internal targets of the cluster.

This leads to the problems mentioned at the beginning of the article. Istio cannot access the Internet. , if your database is not in the cluster, you will find that it cannot be connected at all

we still use sleep as our example.

> kubectl apply -n istio-test -f istio-1.0.3/samples/sleep/sleep.yaml
> export SOURCE_POD=$(kubectl get -n istio-test pod -l app=sleep -o jsonpath={.items..metadata.name})
# 尝试访问(访问任何外部地址都会出现404)
> kubectl exec -n istio-test -it $SOURCE_POD -c sleep bash
crul -I baidu.com
bash: crul: command not found
root@sleep-76df4f989c-9hvvd:/# curl -I baidu.com
HTTP/1.1 404 Not Found
date: Thu, 08 Nov 2018 03:37:16 GMT
server: envoy
transfer-encoding: chunked

1. Configure external services

kubectl apply -n istio-test -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: baidu-ext
spec:
  hosts:
  - baidu.com
  ports:
  - number: 80
    name: http
    protocol: HTTP
  resolution: DNS
  location: MESH_EXTERNAL
EOF

create a ServiceEntry and VirtualService to allow access to external HTTPS services. Note: For TLS protocols including HTTPS, you need to create ServiceEntry in addition to TLS VirtualService.

> kubectl exec -n istio-test -it $SOURCE_POD -c sleep bash
> curl -I baidu.com
HTTP/1.1 200 OK
date: Thu, 08 Nov 2018 03:37:57 GMT
server: envoy
last-modified: Tue, 12 Jan 2010 13:48:00 GMT
etag: "51-47cf7e6ee8400"
accept-ranges: bytes
content-length: 81
cache-control: max-age=86400
expires: Fri, 09 Nov 2018 03:37:57 GMT
content-type: text/html
x-envoy-upstream-service-time: 67

1. Configure an external HTTPS service

only http is configured for access. If https is used, the following situations occur:

> curl -I https://baidu.com
curl: (35) Unknown SSL protocol error in connection to baidu.com:443 

create a ServiceEntry and a VirtualService to allow access to external HTTPS services. Note that for TLS protocols (including HTTPS), you need to ServiceEntry in addition to VirtualService. VirtualService must include tls rules and sni_hosts in the match clause to enable SNI routing.

kubectl apply -n istio-test -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: baidu
spec:
  hosts:
  - baidu.com
  ports:
  - number: 443
    name: https
    protocol: HTTPS
  resolution: DNS
  location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: baidu
spec:
  hosts:
  - baidu.com
  tls:
  - match:
    - port: 443
      sni_hosts:
      - baidu.com
    route:
    - destination:
        host: baidu.com
        port:
          number: 443
      weight: 100
EOF

> curl -I https://baidu.com
HTTP/1.1 302 Moved Temporarily
Server: bfe/1.0.8.18
Date: Thu, 08 Nov 2018 03:41:05 GMT
Content-Type: text/html
Content-Length: 161
Connection: keep-alive
Location: http://www.baidu.com/

1. Configure routing rules for external services

Istio routing rules can be configured for the traffic that accesses external services through ServiceEntry, which is similar to the traffic in the grid. Use istioctl to set a timeout rule for the httpbin.org service.

In the test Pod, use curl to call the/delay endpoint of the external service httpbin.org:

> kubectl apply -n istio-test -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: httpbin-ext
spec:
  hosts:
  - httpbin.org
  ports:
  - number: 80
    name: http
    protocol: HTTP
  resolution: DNS
  location: MESH_EXTERNAL
EOF
> time curl -o /dev/null -s -w "%{http_code}\n" http://httpbin.org/delay/5
200

real    0m5.752s
user    0m0.013s
sys     0m0.022s

use kubectl to set a 3-second timeout for access to external services in httpbin.org

kubectl apply -n istio-test -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin-ext
spec:
  hosts:
    - httpbin.org
  http:
  - timeout: 3s
    route:
      - destination:
          host: httpbin.org
        weight: 100
EOF

This time, a 504 (Gateway Timeout) response will be received after 3 seconds. Although httpbin.org was still waiting for him for 5 seconds, Istio cut off the request in 3 seconds.

time curl -o /dev/null -s -w "%{http_code}\n" http
504

real    0m3.046s
user    0m0.010s
sys     0m0.010s

1. Directly call external services

confirming restrictions can bring more control to avoid errors, but many times it still brings a lot of troubles. What should I do if I do not want Istio to restrict access at will? Of course, you can configure the range of Istio. First, we need to determine the range of internal Cluster IP addresses:

• use minikube in the range of 10.0.0.1/24
• the rancher range is 10.43.0.1/24.

Solution 1 (HELM):

note that the Helm command, especially the -- namespace parameter, should be used as before when Istio was deployed. After installing the original Istio command, add -- set global.proxy.includeIPRanges = "10.0.0.1/24" -x templates/sidecar-injector-configmap.yaml.

helm template install/kubernetes/helm/istio <安装 Istio 时所使用的参数> --set global.proxy.includeIPRanges="10.0.0.1/24" -x templates/sidecar-injector-configmap.yaml | kubectl apply -f -

And then re-deployment sleep.

Solution 2 (adaptation plan):

because the author is not using helm for istio installation, directly using the official demo to install, we can first find the includeOutboundIPRanges and then modify the following * to the corresponding IP address segment

PS: yaml generated by different versions is different. Note that

istio-1.0.1 version

- "[[ index .ObjectMeta.Annotations "traffic.sidecar.istio.io/includeOutboundIPRanges"  ]]"
[[ else -]]
- "10.43.0.1/24"
[[ end -]]

istio1.0.3 version

- "[[ annotation .ObjectMeta `traffic.sidecar.istio.io/includeOutboundIPRanges`  "10.43.0.1/24"  ]]"

then redeploy Istio without any access restrictions