当前位置：
  1. 首页
  2. 开源
  3. 开源项目
  4. 正文

Kubernetes中使用Envoy详解：Sidecar、Gateway与流量管理

开源项目 0 1

概述

Envoy是云原生架构中最重要的代理组件之一，在Kubernetes中Envoy可以以多种方式部署使用。本文将详细介绍在Kubernetes集群中使用Envoy的各种场景，包括Sidecar代理部署、Envoy Gateway配置、流量管理、熔断限流、mTLS安全配置以及可观测性实践。

Envoy在Kubernetes中的应用场景

  • Sidecar代理：在每个Pod中注入Envoy代理，实现服务间的透明代理
  • API网关：使用Envoy作为入口网关，统一处理南北向流量
  • 服务网格数据平面：配合Istio、Linkerd等服务网格使用
  • 边缘代理：处理外部流量入口
  • 流量分割：实现金丝雀发布、A/B测试

方式一：手动部署Envoy Sidecar

1. 创建Envoy ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: envoy-sidecar-config
  namespace: default
data:
  envoy.yaml: |
    admin:
      address:
        socket_address:
          protocol: TCP
          address: 127.0.0.1
          port_value: 9901

    static_resources:
      listeners:
      - name: inbound-listener
        address:
          socket_address:
            protocol: TCP
            address: 0.0.0.0
            port_value: 15001
        filter_chains:
        - filters:
          - name: envoy.filters.network.tcp_proxy
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
              stat_prefix: inbound_proxy
              clusters:
              - name: original-destination
                type: ORIGINAL_DST
                lb_policy: ORIGINAL_DST_LB

      - name: outbound-listener
        address:
          socket_address:
            protocol: TCP
            address: 0.0.0.0
            port_value: 15001
        filter_chains:
        - filters:
          - name: envoy.filters.network.http_connection_manager
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
              codec_type: AUTO
              stat_prefix: outbound_proxy
              http_filters:
              - name: envoy.filters.http.router
              route_config:
                name: outbound-routes
                virtual_hosts:
                - name: outbound
                  domains: ["*"]
                  routes:
                  - match:
                      prefix: "/"
                    route:
                      cluster: original-destination

2. 创建Sidecar Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-with-envoy
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: app-with-envoy
  template:
    metadata:
      labels:
        app: app-with-envoy
    spec:
      containers:
      # 应用程序容器
      - name: app
        image: myapp:latest
        ports:
        - containerPort: 8080
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 500m
            memory: 256Mi
      
      # Envoy Sidecar容器
      - name: envoy
        image: envoyproxy/envoy:v1.35.0
        ports:
        - name: envoy-inbound
          containerPort: 15001
        - name: envoy-admin
          containerPort: 9901
        volumeMounts:
        - name: envoy-config
          mountPath: /etc/envoy
          readOnly: true
        command: ["envoy"]
        args: ["-c", "/etc/envoy/envoy.yaml"]
        resources:
          requests:
            cpu: 50m
            memory: 64Mi
          limits:
            cpu: 200m
            memory: 128Mi
        livenessProbe:
          httpGet:
            path: /server_info
            port: 9901
          initialDelaySeconds: 10
        readinessProbe:
          httpGet:
            path: /server_info
            port: 9901
          initialDelaySeconds: 5
      
      volumes:
      - name: envoy-config
        configMap:
          name: envoy-sidecar-config

方式二：使用Istio Sidecar自动注入

1. 安装Istio

# 下载并安装Istio
curl -L https://istio.io/downloadIstio | sh -
cd istio-1.24.0

# 安装Istio
istioctl install --set profile=default -y

# 启用Sidecar自动注入
kubectl label namespace default istio-injection=enabled

2. 部署使用Sidecar的应用

apiVersion: apps/v1
kind: Deployment
metadata:
  name: bookinfo-reviews
spec:
  replicas: 3
  selector:
    matchLabels:
      app: reviews
  template:
    metadata:
      labels:
        app: reviews
      annotations:
        sidecar.istio.io/inject: "true"
    spec:
      containers:
      - name: reviews
        image: istio/examples-bookinfo-reviews-v3:1.18.0
        ports:
        - containerPort: 9080

方式三：使用Envoy Gateway

1. 安装Envoy Gateway

# 使用Helm安装
helm repo add envoy-gateway https://envoygateway.io/charts
helm repo update

# 创建命名空间
kubectl create namespace envoy-gateway

# 安装Envoy Gateway
helm install envoy-gateway envoy-gateway/envoy-gateway \
  --namespace envoy-gateway \
  --set image.tag=v1.2.0

2. 创建Gateway

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: prod-gateway
  namespace: envoy-gateway
spec:
  gatewayClassName: envoy-gateway
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces:
        from: Selector
        selector:
          matchLabels:
            shared-gateway: "true"

3. 创建HTTPRoute

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: backend-route
  namespace: default
  labels:
    shared-gateway: "true"
spec:
  parentRefs:
  - name: prod-gateway
    namespace: envoy-gateway
  rules:
  - matches:
    - path:
        value: /api/users
    backendRefs:
    - name: users-service
      port: 8080
  - matches:
    - path:
        value: /api/orders
    backendRefs:
    - name: orders-service
      port: 8080
  - backendRefs:
    - name: default-service
      port: 8080

高级流量管理

1. 配置流量分割（金丝雀发布）

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: canary-route
spec:
  parentRefs:
  - name: prod-gateway
    namespace: envoy-gateway
  rules:
  - matches:
    - headers:
      - name: x-canary
        value: "true"
    backendRefs:
    - name: app-v2
      port: 8080
      weight: 100
  - backendRefs:
    - name: app-v1
      port: 8080
      weight: 100

2. 配置熔断器

clusters:
- name: backend-service
  connect_timeout: 1s
  type: STRICT_DNS
  lb_policy: ROUND_ROBIN
  # 熔断配置
  circuit_breakers:
    thresholds:
    - priority: DEFAULT
      max_connections: 1000
      max_pending_requests: 1000
      max_requests: 1000
      max_retries: 3
  # 离群检测
  outlier_detection:
    interval: 5s
    base_ejection_time: 30s
    max_ejection_percent: 50
    consecutive_5xx: 5

3. 配置限流

http_filters:
- name: envoy.filters.http.local_ratelimit
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
    stat_prefix: http_local_rate_limiter
    token_bucket:
      max_tokens: 10000
      tokens_per_fill: 10000
      fill_interval: 60s
    filter_enabled:
      default_value:
        numerator: 100
        denominator: HUNDRED
    response_headers_to_add:
    - header:
        key: x-rate-limit-limit
        value: "10000"
    - header:
        key: x-rate-limit-remaining
        value: "9999"

4. 配置重试和超时

routes:
- match:
    prefix: /reliable-api
  route:
    cluster: backend
    timeout: 30s
    retry_policy:
      retry_on: 5xx,connect-failure
      num_retries: 3
      per_try_timeout: 10s
      retry_back_off:
        base_interval: 1s
        max_interval: 10s

mTLS安全配置

1. 创建TLS Secret

# 创建TLS Secret
kubectl create secret tls envoy-tls-secret \
  --cert=server.crt \
  --key=server.key \
  --namespace=default

2. 配置Envoy TLS终止

transport_socket:
  name: envoy.transport_sockets.tls
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
    common_tls_context:
      tls_certificates:
      - certificate_chain:
          filename: /etc/envoy/tls/server.crt
        private_key:
          filename: /etc/envoy/tls/server.key

3. 使用Istio实现mTLS

# PeerAuthentication - 强制mTLS
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: mtls-strict
  namespace: default
spec:
  mtls:
    mode: STRICT

# DestinationRule
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: mtls-destination-rule
spec:
  host: "*.default.svc.cluster.local"
  trafficPolicy:
    tls:
      mode: ISTIO_MUTUAL

可观测性配置

1. ServiceMonitor配置

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: envoy-monitor
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: envoy
  namespaceSelector:
    matchNames:
    - default
  endpoints:
  - port: admin
    path: /stats/prometheus
    interval: 15s

2. 配置访问日志

access_log:
- name: envoy.access_loggers.file
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
    path: /var/log/envoy/access.log
    log_format:
      text_format: "[%Y-%m-%d %T.%fZ] %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT%

使用Kubernetes Operator管理Envoy

创建EnvoyDeployment

apiVersion: operator.envoy.io/v1alpha1
kind: EnvoyDeployment
metadata:
  name: envoy-deployment
spec:
  replicas: 2
  version: v1.35.0
  admin:
    accessLogPath: /dev/null
    address:
      socketAddress:
        address: 127.0.0.1
        portValue: 9901
  service:
    type: LoadBalancer
    ports:
    - name: http
      port: 80
      targetPort: 80

最佳实践

1. 配置资源限制

resources:
  requests:
    cpu: "100m"
    memory: "128Mi"
  limits:
    cpu: "500m"
    memory: "512Mi"

2. 配置健康检查

livenessProbe:
  httpGet:
    path: /server_info
    port: 9901
  initialDelaySeconds: 10
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /server_info
    port: 9901
  initialDelaySeconds: 5
  periodSeconds: 5

3. 配置Pod反亲和性

affinity:
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
    - weight: 100
      podAffinityTerm:
        labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - envoy-proxy
        topologyKey: kubernetes.io/hostname

4. 配置Pod中断预算

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: envoy-pdb
spec:
  minAvailable: 1
  selector:
    matchLabels:
      app: envoy

故障排查

1. 查看Envoy配置

# 进入Envoy Pod
kubectl exec -it envoy-pod -- /bin/bash

# 查看完整配置
curl http://localhost:9901/config_dump

# 查看集群状态
curl http://localhost:9901/clusters

# 查看监听器状态
curl http://localhost:9901/listeners

2. 常用故障排查命令

# 查看Envoy日志
kubectl logs -f envoy-pod -c envoy

# 查看Pod事件
kubectl describe pod envoy-pod

# 检查ConfigMap
kubectl get configmap envoy-sidecar-config -o yaml

# 设置日志级别
curl -X POST http://localhost:9901/logging?level=debug

3. 常见问题Q&A

Q: Envoy Sidecar无法启动

解决方法：

  1. 检查ConfigMap配置是否正确
  2. 验证ConfigMap是否正确挂载
  3. 查看Pod事件和日志

Q: 流量无法路由

解决方法：

  1. 检查路由配置
  2. 验证后端服务是否正常
  3. 检查Cluster配置

Q: TLS配置错误

解决方法：

  1. 验证证书文件路径
  2. 检查证书是否过期
  3. 验证私钥与证书匹配

总结

在Kubernetes中使用Envoy可以极大地提升服务的可观测性、安全性和流量管理能力。通过本文的介绍，你应该已经掌握了：

  • 手动部署Envoy Sidecar
  • 使用Istio自动注入Sidecar
  • 使用Envoy Gateway配置入口网关
  • 配置流量分割、熔断、限流
  • 配置mTLS双向认证
  • 配置可观测性（监控、日志、追踪）
  • 使用Operator管理Envoy
  • 故障排查方法

参考资源

上一篇 Ubuntu 22.04高可用Kubernetes 1.34集群部署（3 Master节点）
下一篇 OpenClaw技能完全指南：用途与使用方法详解

相关文章

发表回复

后才能评论