37-Service

concepts/services-networking/service/

Service

An abstract way to expose an application running on a set of Pods as a network service. 一种将在一组pod上运行的应用程序公开为网络服务的抽象方法。

With Kubernetes you don’t need to modify your application to use an unfamiliar service discovery mechanism. Kubernetes gives Pods their own IP addresses and a single DNS name for a set of Pods, and can load-balance across them. 使用kubernetes,您不需要修改应用程序,就可以使用不熟悉的服务发现机制。kubernetes为pod提供了自己的ip地址和一组pod的单个dns名称,并且可以在它们之间实现负载平衡。

Motivation

Kubernetes Pods are mortal. They are born and when they die, they are not resurrected. If you use a Deployment to run your app, it can create and destroy Pods dynamically. k8s pod 是凡人。他们出生了,死了也不会复活。如果使用部署运行应用程序,它可以动态创建和销毁pod。

Each Pod gets its own IP address, however in a Deployment, the set of Pods running in one moment in time could be different from the set of Pods running that application a moment later. 每个pod都有自己的ip地址,但是在部署中,一瞬间运行的pod集可能不同于一瞬间运行该应用程序的pod集。

This leads to a problem: if some set of Pods (call them “backends”) provides functionality to other Pods (call them “frontends”) inside your cluster, how do the frontends find out and keep track of which IP address to connect to, so that the frontend can use the backend part of the workload? 这会导致一个问题:如果集群中的一些pod(称为“后端”)向其他pod(称为“前端”)提供功能,那么前端如何发现并跟踪要连接到的IP地址,以便前端可以使用工作负载的后端部分?

Enter Services.

Service resources

In Kubernetes, a Service is an abstraction which defines a logical set of Pods and a policy by which to access them (sometimes this pattern is called a micro-service). The set of Pods targeted by a Service is usually determined by a selector (see below for why you might want a Service without a selector). 在kubernetes中,服务是一个抽象,它定义了pod的逻辑集和访问它们的策略(有时这种模式被称为微服务)。服务所针对的pod集通常由选择器决定(请参见下面的说明为什么您可能需要不带选择器的服务)。

For example, consider a stateless image-processing backend which is running with 3 replicas. Those replicas are fungible—frontends do not care which backend they use. While the actual Pods that compose the backend set may change, the frontend clients should not need to be aware of that, nor should they need to keep track of the set of backends themselves. 例如,考虑一个无状态的图像处理后端,它使用3个副本运行。这些副本是可替换的前端,不关心它们使用哪个后端。虽然组成后端集的实际pod可能会发生变化,但前端客户机不应该知道这一点,也不应该自己跟踪后端集。

The Service abstraction enables this decoupling. 服务抽象支持这种分离。

Cloud-native service discovery

If you’re able to use Kubernetes APIs for service discovery in your application, you can query the API server for Endpoints, that get updated whenever the set of Pods in a Service changes. 如果您能够在应用程序中使用kubernetes api进行服务发现,那么您可以向api服务器查询端点,只要服务中的pod集发生更改,这些端点就会得到更新。

For non-native applications, Kubernetes offers ways to place a network port or load balancer in between your application and the backend Pods. 对于非本机应用程序,kubernetes提供了在应用程序和后端pod之间放置网络端口或负载平衡器的方法。

Defining a Service

A Service in Kubernetes is a REST object, similar to a Pod. Like all of the REST objects, you can POST a Service definition to the API server to create a new instance. kubernetes中的服务是一个rest对象,类似于pod。与所有rest对象一样,您可以将服务定义发布到api服务器以创建新实例。

For example, suppose you have a set of Pods that each listen on TCP port 9376 and carry a label app=MyApp: 例如,假设您有一组pod,每个pod侦听TCP端口9376,并带有一个标签app=myapp:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: MyApp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376

This specification creates a new Service object named “my-service”, which targets TCP port 9376 on any Pod with the app=MyApp label. 此规范创建一个名为“My Service”的新服务对象,该对象以任何带有app=MyApp标签的POD上的TCP端口9376为目标。

Kubernetes assigns this Service an IP address (sometimes called the “cluster IP”), which is used by the Service proxies (see Virtual IPs and service proxies below). KubNeNes将该服务分配为IP地址(有时称为“群集IP”),该服务地址由服务代理使用(参见下面的虚拟IP和服务代理)。

The controller for the Service selector continuously scans for Pods that match its selector, and then POSTs any updates to an Endpoint object also named “my-service”. 服务选择器的控制器持续扫描与其选择器匹配的pod,然后将任何更新发布到一个名为“我的服务”的端点对象。

Note: A Service can map any incoming port to a targetPort. By default and for convenience, the targetPort is set to the same value as the port field. 服务可以将任何传入端口映射到目标端口。默认情况下,为了方便起见,targetport设置为与port字段相同的值。

Port definitions in Pods have names, and you can reference these names in the targetPort attribute of a Service. This works even if there is a mixture of Pods in the Service using a single configured name, with the same network protocol available via different port numbers. This offers a lot of flexibility for deploying and evolving your Services. For example, you can change the port numbers that Pods expose in the next version of your backend software, without breaking clients. pods中的端口定义有名称,您可以在服务的targetport属性中引用这些名称。即使服务中混合了使用单个配置名称的pod,并且通过不同的端口号提供相同的网络协议,这种方法仍然有效。这为部署和开发服务提供了很大的灵活性。例如,您可以更改pods在下一个版本的后端软件中公开的端口号,而不破坏客户端。

The default protocol for Services is TCP; you can also use any other supported protocol. 服务的默认协议是TCP;您也可以使用任何其他受支持的协议。

As many Services need to expose more than one port, Kubernetes supports multiple port definitions on a Service object. Each port definition can have the same protocol, or a different one. 由于许多服务需要公开多个端口,因此kubernetes支持服务对象上的多个端口定义。每个端口定义可以有相同的协议,也可以有不同的协议。

Services without selectors

Services most commonly abstract access to Kubernetes Pods, but they can also abstract other kinds of backends. For example: 服务通常抽象访问kubernetes pods,但它们也可以抽象其他类型的后端。例如:

  • You want to have an external database cluster in production, but in your test environment you use your own databases. 您希望在生产环境中有一个外部数据库集群,但在测试环境中使用自己的数据库。
  • You want to point your Service to a Service in a different Namespace or on another cluster. 您要将服务指向其他命名空间或其他群集上的服务。
  • You are migrating a workload to Kubernetes. Whilst evaluating the approach, you run only a proportion of your backends in Kubernetes. 您正在将工作负载迁移到kubernetes。在评估方法时,你只在库伯内特斯运行一部分你的后端。

In any of these scenarios you can define a Service without a Pod selector. For example: 在任何这些场景中,您都可以定义一个不带pod选择器的服务。例如:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376

Because this Service has no selector, the corresponding Endpoint object is not created automatically. You can manually map the Service to the network address and port where it’s running, by adding an Endpoint object manually: 由于此服务没有选择器,因此相应的终结点对象不会自动创建。通过手动添加终结点对象,可以将服务手动映射到其运行的网络地址和端口:

apiVersion: v1
kind: Endpoints
metadata:
  name: my-service
subsets:
  - addresses:
      - ip: 192.0.2.42
    ports:
      - port: 9376

Note:

The endpoint IPs must not be: loopback (127.0.0.0/8 for IPv4, ::1/128 for IPv6), or link-local (169.254.0.0/16 and 224.0.0.0/24 for IPv4, fe80::/64 for IPv6). 终结点IP不能是:环回(IPv4为127.0.0.0/8,IPv6为:1/128)或链路本地(IPv4为169.254.0.0/16和224.0.0.0/24,IPv6为fe80::64)。

Endpoint IP addresses cannot be the cluster IPs of other Kubernetes Services, because kube-proxy doesn’t support virtual IPs as a destination. 端点IP地址不能是其他Kubernetes服务的群集IP,因为Kube代理不支持将虚拟IP作为目标。

Accessing a Service without a selector works the same as if it had a selector. In the example above, traffic is routed to the single endpoint defined in the YAML: 192.0.2.42:9376 (TCP). 不使用选择器访问服务的工作方式与使用选择器的工作方式相同。在上面的示例中,流量被路由到yaml中定义的单个端点。

An ExternalName Service is a special case of Service that does not have selectors and uses DNS names instead. For more information, see the ExternalName section later in this document. externalname服务是一种特殊的服务,它没有选择器,而是使用dns名称。有关详细信息,请参阅本文档后面的externalname部分。

Endpoint Slices

FEATURE STATE: Kubernetes v1.16 alpha

Endpoint Slices are an API resource that can provide a more scalable alternative to Endpoints. Although conceptually quite similar to Endpoints, Endpoint Slices allow for distributing network endpoints across multiple resources. By default, an Endpoint Slice is considered “full” once it reaches 100 endpoints, at which point additional Endpoint Slices will be created to store any additional endpoints. 端点切片是一个api资源,可以提供一个更可伸缩的端点替代方案。虽然在概念上与端点非常相似,但端点切片允许跨多个资源分发网络端点。默认情况下,当一个端点切片到达100个端点时,它被认为是“完整的”,此时将创建额外的端点切片来存储任何额外的端点。

Endpoint Slices provide additional attributes and functionality which is described in detail in Endpoint Slices.

Virtual IPs and service proxies

Every node in a Kubernetes cluster runs a kube-proxy. kube-proxy is responsible for implementing a form of virtual IP for Services of type other than ExternalName. kubernetes集群中的每个节点都运行kube代理。kube代理负责为非externalname类型的服务实现一种形式的虚拟ip。

Why not use round-robin DNS?

A question that pops up every now and then is why Kubernetes relies on proxying to forward inbound traffic to backends. 一个时不时出现的问题是,为什么kubernetes依赖代理将入站流量转发到后端。
What about other approaches? For example, would it be possible to configure DNS records that have multiple A values (or AAAA for IPv6), and rely on round-robin name resolution? 其他方法呢?例如,是否可以配置具有多个a值(或ipv6为aaaa)的dns记录,并依赖于循环名称解析?

There are a few reasons for using proxying for Services: 对服务使用代理有几个原因:

  • There is a long history of DNS implementations not respecting record TTLs, and caching the results of name lookups after they should have expired. dns实现有很长的一段历史,它不尊重记录ttl,并在名称查找的结果应该过期后缓存它们。
  • Some apps do DNS lookups only once and cache the results indefinitely. 有些应用程序只做一次dns查找,并无限期地缓存结果。
  • Even if apps and libraries did proper re-resolution, the low or zero TTLs on the DNS records could impose a high load on DNS that then becomes difficult to manage. 即使应用程序和库进行了正确的重新解析,dns记录上的低ttl或零ttl也可能会对dns施加高负载,这将变得难以管理。

Version compatibility

Since Kubernetes v1.0 you have been able to use the userspace proxy mode. Kubernetes v1.1 added iptables mode proxying, and in Kubernetes v1.2 the iptables mode for kube-proxy became the default. Kubernetes v1.8 added ipvs proxy mode. 自从kubernetes v1.0以来,您就可以使用用户空间代理模式。kubernetes v1.1增加了iptables模式代理,在kubernetesv1.2中,kube代理的iptables模式成为默认模式。Kubernetes v1.8增加了IPV代理模式。

User space proxy mode

In this mode, kube-proxy watches the Kubernetes master for the addition and removal of Service and Endpoint objects. For each Service it opens a port (randomly chosen) on the local node. Any connections to this “proxy port” is proxied to one of the Service’s backend Pods (as reported via Endpoints). kube-proxy takes the SessionAffinity setting of the Service into account when deciding which backend Pod to use. 在此模式下,kube代理监视kubernetes主服务器添加和删除服务和端点对象。对于每个服务,它在本地节点上打开一个端口(随机选择)。与这个“代理端口”的任何连接都被代理到服务的后端POD(如通过端点报告)。kube代理在决定使用哪个后端pod时,会考虑服务的sessionaffinity设置。

Lastly, the user-space proxy installs iptables rules which capture traffic to the Service’s clusterIP (which is virtual) and port. The rules redirect that traffic to the proxy port which proxies the backend Pod. 最后,用户空间代理安装iptables规则,这些规则捕获到服务的clusterip(虚拟)和端口的流量。这些规则将业务重定向到代理端口,代理端口代理后端POD。

By default, kube-proxy in userspace mode chooses a backend via a round-robin algorithm. 默认情况下,用户空间模式下的kube代理通过循环算法选择后端。

Services overview diagram for userspace proxy

iptables proxy mode

In this mode, kube-proxy watches the Kubernetes control plane for the addition and removal of Service and Endpoint objects. For each Service, it installs iptables rules, which capture traffic to the Service’s clusterIP and port, and redirect that traffic to one of the Service’s backend sets. For each Endpoint object, it installs iptables rules which select a backend Pod. 在此模式下,kube代理监视kubernetes控制平面以添加和删除服务和端点对象。对于每个服务,它都安装iptables规则,这些规则捕获到服务的“clusterip”和“port”的流量,并将该流量重定向到服务的一个后端集。对于每个端点对象,它安装iptables规则来选择后端pod。

By default, kube-proxy in iptables mode chooses a backend at random. 默认情况下,iptables模式下的kube代理随机选择后端。

Using iptables to handle traffic has a lower system overhead, because traffic is handled by Linux netfilter without the need to switch between userspace and the kernel space. This approach is also likely to be more reliable. 使用iptables处理流量的系统开销较低,因为流量由linux netfilter处理,而不需要在用户空间和内核空间之间切换。这种方法也可能更加可靠。

If kube-proxy is running in iptables mode and the first Pod that’s selected does not respond, the connection fails. This is different from userspace mode: in that scenario, kube-proxy would detect that the connection to the first Pod had failed and would automatically retry with a different backend Pod. 如果kube代理正在iptables模式下运行,并且选择的第一个pod没有响应,则连接失败。这与用户空间模式不同:在这种情况下,kube代理将检测到与第一个pod的连接已失败,并将使用不同的后端pod自动重试。

You can use Pod readiness probes to verify that backend Pods are working OK, so that kube-proxy in iptables mode only sees backends that test out as healthy. Doing this means you avoid having traffic sent via kube-proxy to a Pod that’s known to have failed. 您可以使用pod readiness probe来验证后端pod是否正常工作,以便iptables模式下的kube代理只看到测试正常的后端。这样做意味着您可以避免通过kube代理将流量发送到已知已失败的pod。

Services overview diagram for iptables proxy

IPVS proxy mode

FEATURE STATE: Kubernetes v1.11 stable

In ipvs mode, kube-proxy watches Kubernetes Services and Endpoints, calls netlink interface to create IPVS rules accordingly and synchronizes IPVS rules with Kubernetes Services and Endpoints periodically. This control loop ensures that IPVS status matches the desired state. When accessing a Service, IPVS directs traffic to one of the backend Pods. 在“ipvs”模式下,kube代理监视kubernetes服务和端点,调用“netlink”接口相应地创建ipvs规则,并定期将ipvs规则与kubernetes服务和端点同步。此控制循环确保IPV状态与所需状态匹配。当访问一个服务时,ipvs将流量定向到一个后端pod。

The IPVS proxy mode is based on netfilter hook function that is similar to iptables mode, but uses hash table as the underlying data structure and works in the kernel space. That means kube-proxy in IPVS mode redirects traffic with a lower latency than kube-proxy in iptables mode, with much better performance when synchronising proxy rules. Compared to the other proxy modes, IPVS mode also supports a higher throughput of network traffic. ipvs代理模式基于netfilter钩子函数,类似iptables模式,但使用哈希表作为底层数据结构,工作在内核空间。这意味着,ipvs模式下的kube代理比iptables模式下的kube代理具有更低的延迟来重定向流量,在同步代理规则时具有更好的性能。与其他代理模式相比,ipvs模式还支持更高的网络流量吞吐量。

IPVS provides more options for balancing traffic to backend Pods; these are: IPV提供了更多的选项来平衡到后端播客的流量;这些选项包括:

  • rr: round-robin
  • lc: least connection (smallest number of open connections)
  • dh: destination hashing
  • sh: source hashing
  • sed: shortest expected delay
  • nq: never queue

Note:

To run kube-proxy in IPVS mode, you must make the IPVS Linux available on the node before you starting kube-proxy. 要在ipvs模式下运行kube代理,在启动kube代理之前,必须使ipvs linux在节点上可用。

When kube-proxy starts in IPVS proxy mode, it verifies whether IPVS kernel modules are available. If the IPVS kernel modules are not detected, then kube-proxy falls back to running in iptables proxy mode. 当kube proxy以ipvs proxy模式启动时,它验证ipvs内核模块是否可用。如果未检测到ipvs内核模块,那么kube代理将返回到iptables代理模式下运行。

Services overview diagram for IPVS proxy

In these proxy models, the traffic bound for the Service’s IP:Port is proxied to an appropriate backend without the clients knowing anything about Kubernetes or Services or Pods. 在这些代理模型中,服务的IP端口的业务绑定被代理到适当的后端,而客户端不知道任何关于Kubernetes或服务或POD的信息。

If you want to make sure that connections from a particular client are passed to the same Pod each time, you can select the session affinity based the on client’s IP addresses by setting service.spec.sessionAffinity to “ClientIP” (the default is “None”). You can also set the maximum session sticky time by setting service.spec.sessionAffinityConfig.clientIP.timeoutSeconds appropriately. (the default value is 10800, which works out to be 3 hours). 如果要确保每次都将来自特定客户端的连接传递到同一pod,可以通过将service.spec.session affinity设置为“client ip”(默认值为“none”),基于客户端的IP地址选择会话关联。还可以通过设置Service .Script SuffiTysCuff.clitP.TimeOffice适当地设置最大会话粘性时间。(默认值为10800,计算结果为3小时)。

Multi-Port Services

For some Services, you need to expose more than one port. Kubernetes lets you configure multiple port definitions on a Service object. When using multiple ports for a Service, you must give all of your ports names so that these are unambiguous. For example: 对于某些服务,需要公开多个端口。kubernetes允许您在服务对象上配置多个端口定义。当为一个服务使用多个端口时,您必须给出所有端口的名称,以便这些端口是明确的。例如:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: MyApp
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 9376
    - name: https
      protocol: TCP
      port: 443
      targetPort: 9377

Note:

As with Kubernetes names in general, names for ports must only contain lowercase alphanumeric characters and -. Port names must also start and end with an alphanumeric character. 与一般的kubernetes名称一样,端口名称只能包含小写字母数字字符和-。端口名还必须以字母数字字符开头和结尾。

For example, the names 123-abc and web are valid, but 123_abc and -web are not. 例如,名称123 abc和web是有效的,但123_abc和-web不是。

Choosing your own IP address

You can specify your own cluster IP address as part of a Service creation request. To do this, set the .spec.clusterIP field. For example, if you already have an existing DNS entry that you wish to reuse, or legacy systems that are configured for a specific IP address and difficult to re-configure. 您可以在服务创建请求中指定自己的群集IP地址。为此,请设置.spec.clusterip字段。例如,如果您已有希望重用的DNS条目,或者为特定IP地址配置的遗留系统,并且难以重新配置。

The IP address that you choose must be a valid IPv4 or IPv6 address from within the service-cluster-ip-range CIDR range that is configured for the API server. If you try to create a Service with an invalid clusterIP address value, the API server will return a 422 HTTP status code to indicate that there’s a problem. 您选择的IP地址必须是为API服务器配置的“服务群集IP范围”CIDR范围内的有效IPv4或IPv6地址。如果试图创建一个clusterip地址值无效的服务,api服务器将返回一个422http状态代码以指示存在问题。

Discovering services

Kubernetes supports 2 primary modes of finding a Service - environment variables and DNS. kubernetes支持查找服务环境变量和dns的两种主要模式。

Environment variables

When a Pod is run on a Node, the kubelet adds a set of environment variables for each active Service. It supports both Docker links compatible variables (see makeLinkVariables) and simpler {SVCNAME}_SERVICE_HOST and {SVCNAME}_SERVICE_PORT variables, where the Service name is upper-cased and dashes are converted to underscores. 当pod在节点上运行时,kubelet为每个活动服务添加一组环境变量。它同时支持Docker链接兼容变量(请参阅makelinkvariables)和更简单的{svcname}u service}u host和{svcname}u service}u port变量,其中服务名称为大写,破折号转换为下划线。

For example, the Service "redis-master" which exposes TCP port 6379 and has been allocated cluster IP address 10.0.0.11, produces the following environment variables: 例如,公开TCP端口6379并已分配群集IP地址10.0.0.11的服务“redis master”生成以下环境变量:

REDIS_MASTER_SERVICE_HOST=10.0.0.11
REDIS_MASTER_SERVICE_PORT=6379
REDIS_MASTER_PORT=tcp://10.0.0.11:6379
REDIS_MASTER_PORT_6379_TCP=tcp://10.0.0.11:6379
REDIS_MASTER_PORT_6379_TCP_PROTO=tcp
REDIS_MASTER_PORT_6379_TCP_PORT=6379
REDIS_MASTER_PORT_6379_TCP_ADDR=10.0.0.11

Note:

When you have a Pod that needs to access a Service, and you are using the environment variable method to publish the port and cluster IP to the client Pods, you must create the Service before the client Pods come into existence. Otherwise, those client Pods won’t have their environment variables populated. 当您有一个需要访问服务的POD,并且您正在使用环境变量方法将端口和集群IP发布到客户端POD时,您必须在客户端POD出现之前创建服务。否则,这些客户机pod将不会填充其环境变量。

If you only use DNS to discover the cluster IP for a Service, you don’t need to worry about this ordering issue. 如果您只使用dns来发现服务的群集ip,则不必担心此排序问题。

DNS

You can (and almost always should) set up a DNS service for your Kubernetes cluster using an add-on. 您可以(而且几乎总是应该)使用附加组件为kubernetes集群设置dns服务。

A cluster-aware DNS server, such as CoreDNS, watches the Kubernetes API for new Services and creates a set of DNS records for each one. If DNS has been enabled throughout your cluster then all Pods should automatically be able to resolve Services by their DNS name. 集群感知dns服务器(如coredns)监视kubernetes api以获取新服务,并为每个服务创建一组dns记录。如果在整个集群中启用了dns,那么所有pod都应该能够根据其dns名称自动解析服务。

For example, if you have a Service called "my-service" in a Kubernetes Namespace "my-ns", the control plane and the DNS Service acting together create a DNS record for "my-service.my-ns". Pods in the "my-ns" Namespace should be able to find it by simply doing a name lookup for my-service ("my-service.my-ns" would also work). 例如,如果在kubernetes名称空间“my ns”中有一个名为“my service”的服务,则控制平面和dns服务一起为“my service.my ns”创建一个dns记录。“my ns”名称空间中的pod应该可以通过简单地查找“my service”(“my service.my ns”)的名称来找到它。

Pods in other Namespaces must qualify the name as my-service.my-ns. These names will resolve to the cluster IP assigned for the Service. 其他命名空间中的pod必须将名称限定为“my service.my ns”。这些名称将解析为为服务分配的群集IP。

Kubernetes also supports DNS SRV (Service) records for named ports. If the "my-service.my-ns" Service has a port named "http" with protocol set to TCP, you can do a DNS SRV query for _http._tcp.my-service.my-ns to discover the port number for "http", as well as the IP address. kubernetes还支持命名端口的dns srv(服务)记录。如果“my service.my ns”服务的端口名为“http”,协议设置为TCP,则可以对“http.\u tcp.my-service.my-ns”执行DNS SRV查询,以发现“http”的端口号以及IP地址。

The Kubernetes DNS server is the only way to access ExternalName Services. You can find more information about ExternalName resolution in DNS Pods and Services. kubernetes dns服务器是访问“externalname”服务的唯一方式。有关“externalname”分辨率的详细信息,请参见

Headless Services

Sometimes you don’t need load-balancing and a single Service IP. In this case, you can create what are termed “headless” Services, by explicitly specifying "None" for the cluster IP (.spec.clusterIP). 有时不需要负载平衡和单个服务IP。在这种情况下,您可以通过显式地为集群ip(.spec.clusterip)指定“none”来创建所谓的“headless”服务。

You can use a headless Service to interface with other service discovery mechanisms, without being tied to Kubernetes’ implementation. 您可以使用无头服务与其他服务发现机制接口,而不必绑定到kubernetes的实现。

For headless Services, a cluster IP is not allocated, kube-proxy does not handle these Services, and there is no load balancing or proxying done by the platform for them. How DNS is automatically configured depends on whether the Service has selectors defined: 对于headless'services',没有分配集群ip,kube代理不处理这些服务,平台也没有为它们进行负载平衡或代理。如何自动配置DNS取决于服务是否定义了选择器:

With selectors

For headless Services that define selectors, the endpoints controller creates Endpoints records in the API, and modifies the DNS configuration to return records (addresses) that point directly to the Pods backing the Service. 对于定义选择器的无头服务,端点控制器在api中创建端点记录,并修改dns配置以返回直接指向支持服务的pod的记录(地址)。

Without selectors

For headless Services that do not define selectors, the endpoints controller does not create Endpoints records. However, the DNS system looks for and configures either: 对于不定义选择器的无头服务,端点控制器不创建端点记录。但是,DNS系统查找并配置:

  • CNAME records for ExternalName-type Services.
  • A records for any Endpoints that share a name with the Service, for all other types.

Publishing Services (ServiceTypes)

For some parts of your application (for example, frontends) you may want to expose a Service onto an external IP address, that’s outside of your cluster. 对于应用程序的某些部分(例如,前端),您可能希望将服务公开到集群外部的外部IP地址。

Kubernetes ServiceTypes allow you to specify what kind of Service you want. The default is ClusterIP. kubernetes servicetypes允许您指定所需的服务类型。默认值是clusterip。

Type values and their behaviors are: 类型值及其行为是:

  • ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType. 在群集内部IP上公开服务。选择此值将使服务只能从群集中访问。这是默认的服务类型。

  • NodePort: Exposes the Service on each Node’s IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You’ll be able to contact the NodePort Service, from outside the cluster, by requesting :. 在静态端口(node port)公开每个节点的ip上的服务。自动创建nodeport服务路由到的clusterip服务。您可以通过请求从群集中外部联系nodeport服务:。

  • LoadBalancer: Exposes the Service externally using a cloud provider’s load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created. 使用云提供程序的负载平衡器对外公开服务。自动创建外部负载平衡器路由到的nodeport和clusterip服务。

  • ExternalName: Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record 通过返回cname记录,将服务映射到externalname字段(例如foo.bar.example.com)的内容

    with its value. No proxying of any kind is set up. 它的价值。未设置任何代理。

    Note: You need CoreDNS version 1.7 or higher to use the ExternalName type. 需要CoreDNS版本1.7或更高版本才能使用ExternalName类型。

You can also use Ingress to expose your Service. Ingress is not a Service type, but it acts as the entry point for your cluster. It lets you consolidate your routing rules into a single resource as it can expose multiple services under the same IP address. 您还可以使用入口来公开您的服务。入口不是服务类型,但它充当集群的入口点。它允许您将路由规则整合到单个资源中,因为它可以在同一IP地址下公开多个服务。

Type NodePort

If you set the type field to NodePort, the Kubernetes control plane allocates a port from a range specified by --service-node-port-range flag (default: 30000-32767). Each node proxies that port (the same port number on every Node) into your Service. Your Service reports the allocated port in its .spec.ports[*].nodePort field. 如果将type字段设置为node port,那么kubernetes控制平面将从--service node port range flag(默认值:30000-32767)指定的范围分配端口。每个节点将端口(每个节点上的相同端口号)代理到服务中。您的服务在其.spec.ports[*].nodeport字段中报告分配的端口。

If you want to specify particular IP(s) to proxy the port, you can set the --nodeport-addresses flag in kube-proxy to particular IP block(s); this is supported since Kubernetes v1.10. This flag takes a comma-delimited list of IP blocks (e.g. 10.0.0.0/8, 192.0.2.0/25) to specify IP address ranges that kube-proxy should consider as local to this node. 如果要指定特定的IP来代理端口,可以将kube proxy中的--nodeport addresses标志设置为特定的IP块;这是kubernetes v1.10支持的。此标志采用以逗号分隔的IP块列表(例如10.0.0.0/8、192.0.2.0/25)来指定kube代理应视为此节点本地的IP地址范围。

For example, if you start kube-proxy with the --nodeport-addresses=127.0.0.0/8 flag, kube-proxy only selects the loopback interface for NodePort Services. The default for --nodeport-addresses is an empty list. This means that kube-proxy should consider all available network interfaces for NodePort. (That’s also compatible with earlier Kubernetes releases). 例如,如果使用--nodeport addresses=127.0.0.0/8标志启动kube proxy,kube proxy只为nodeport服务选择环回接口。--nodeport地址的默认值是空列表。这意味着kube代理应该考虑nodeport的所有可用网络接口。(这也与早期的kubernetes版本兼容)。

If you want a specific port number, you can specify a value in the nodePort field. The control plane will either allocate you that port or report that the API transaction failed. This means that you need to take care about possible port collisions yourself. You also have to use a valid port number, one that’s inside the range configured for NodePort use. 如果需要特定的端口号,可以在nodeport字段中指定一个值。控制平面将为您分配该端口或报告API事务失败。这意味着您需要自己处理可能的端口冲突。您还必须使用有效的端口号,该端口号在为nodeport使用配置的范围内。

Using a NodePort gives you the freedom to set up your own load balancing solution, to configure environments that are not fully supported by Kubernetes, or even to just expose one or more nodes’ IPs directly. 使用nodeport可以自由设置自己的负载平衡解决方案,配置kubernetes不完全支持的环境,甚至直接公开一个或多个节点的ip。

Note that this Service is visible as :spec.ports[*].nodePort and .spec.clusterIP:spec.ports[*].port. (If the --nodeport-addresses flag in kube-proxy is set, would be filtered NodeIP(s).) 请注意,此服务的可见名称为:spec.ports[].nodeport和.spec.clusterip:spec.ports[].port。(如果在kube代理中设置了--nodeport addresses标志,则将被筛选nodeip。)

Type LoadBalancer

On cloud providers which support external load balancers, setting the type field to LoadBalancer provisions a load balancer for your Service. The actual creation of the load balancer happens asynchronously, and information about the provisioned balancer is published in the Service’s .status.loadBalancer field. For example: 在支持外部负载平衡器的云提供程序上,将“type”字段设置为“load balancer”将为您的服务提供负载平衡器。负载平衡器的实际创建是异步进行的,有关已设置平衡器的信息将发布在服务的“.status.load balancer”字段中。例如:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: MyApp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376
  clusterIP: 10.0.171.239
  type: LoadBalancer
status:
  loadBalancer:
    ingress:
    - ip: 192.0.2.127

Traffic from the external load balancer is directed at the backend Pods. The cloud provider decides how it is load balanced. 来自外部负载平衡器的通信量定向到后端播客。云提供商决定如何进行负载平衡。

Some cloud providers allow you to specify the loadBalancerIP. In those cases, the load-balancer is created with the user-specified loadBalancerIP. If the loadBalancerIP field is not specified, the loadBalancer is set up with an ephemeral IP address. If you specify a loadBalancerIP but your cloud provider does not support the feature, the loadbalancerIP field that you set is ignored. 某些云提供程序允许您指定loadbalancerip。在这些情况下,将使用用户指定的loadbalancerip创建负载平衡器。如果未指定loadbalancer ip字段,则loadbalancer将使用临时IP地址进行设置。如果指定了loadbalancerip,但您的云提供商不支持该功能,则会忽略您设置的loadbalancerip字段。

Note: If you’re using SCTP, see the caveat below about the LoadBalancer Service type. 如果您使用的是sctp,请参阅下面关于loadbalancer服务类型的警告。

Note:

On Azure, if you want to use a user-specified public type loadBalancerIP, you first need to create a static type public IP address resource. This public IP address resource should be in the same resource group of the other automatically created resources of the cluster. For example, MC_myResourceGroup_myAKSCluster_eastus. 在azure上,如果要使用用户指定的公共类型loadbalancerip,首先需要创建静态类型的公共ip地址资源。此公用IP地址资源应位于群集的其他自动创建资源的同一资源组中。例如,mc_myresourcegroup_myakscluster_eastus。

Specify the assigned IP address as loadBalancerIP. Ensure that you have updated the securityGroupName in the cloud provider configuration file. For information about troubleshooting CreatingLoadBalancerFailed permission issues see, Use a static IP address with the Azure Kubernetes Service (AKS) load balancer or CreatingLoadBalancerFailed on AKS cluster with advanced networking. 指定分配的IP地址为loadbalancerIP。确保已更新云提供程序配置文件中的securitygroupname。有关CreatingloadbalancerFailed权限问题疑难解答的信息,请参阅,将静态IP地址与azure kubernetes服务(AKS)负载平衡器一起使用,或在具有高级网络的AKS群集上CreatingloadbalancerFailed。

Internal load balancer

In a mixed environment it is sometimes necessary to route traffic from Services inside the same (virtual) network address block. 在混合环境中,有时需要从同一(虚拟)网络地址块内的服务路由流量。

In a split-horizon DNS environment you would need two Services to be able to route both external and internal traffic to your endpoints. 在拆分地平线dns环境中,需要两个服务才能将外部和内部流量路由到端点。

You can achieve this by adding one the following annotations to a Service. The annotation to add depends on the cloud Service provider you’re using. 在拆分地平线dns环境中,需要两个服务才能将外部和内部流量路由到端点。

Select one of the tabs. 选择其中一个选项卡。

TLS support on AWS

For partial TLS / SSL support on clusters running on AWS, you can add three annotations to a LoadBalancer service 对于在aws上运行的集群上的部分tls/ssl支持,可以向loadbalancer服务添加三个注释:

metadata:
  name: my-service
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-east-1:123456789012:certificate/12345678-1234-1234-1234-123456789012

The first specifies the ARN of the certificate to use. It can be either a certificate from a third party issuer that was uploaded to IAM or one created within AWS Certificate Manager. 第一个指定要使用的证书的arn。它可以是从第三方颁发者上传到IAM的证书,也可以是在AWS证书管理器中创建的证书。

metadata:
  name: my-service
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: (https|http|ssl|tcp)

The second annotation specifies which protocol a Pod speaks. For HTTPS and SSL, the ELB expects the Pod to authenticate itself over the encrypted connection, using a certificate. 第二个注释指定pod所说的协议。对于https和ssl,elb希望pod使用证书通过加密连接进行身份验证。

HTTP and HTTPS selects layer 7 proxying: the ELB terminates the connection with the user, parse headers and inject the X-Forwarded-For header with the user’s IP address (Pods only see the IP address of the ELB at the other end of its connection) when forwarding requests. http和http s选择第7层代理:当转发请求时,elb终止与用户的连接,解析头并将x-forwarded-for头插入用户的ip地址(pods只看到连接另一端elb的ip地址)。

TCP and SSL selects layer 4 proxying: the ELB forwards traffic without modifying the headers. TCP和SSL选择第4层代理:ELB在不修改报头的情况下转发通信量。

In a mixed-use environment where some ports are secured and others are left unencrypted, you can use the following annotations: 在混合使用环境中,有些端口是安全的,而另一些端口是未加密的,可以使用以下注释:

    metadata:
      name: my-service
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
        service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443,8443"

In the above example, if the Service contained three ports, 80, 443, and 8443, then 443 and 8443 would use the SSL certificate, but 80 would just be proxied HTTP. 在上面的示例中,如果服务包含三个端口、80, 443和8443,那么443和8443将使用SSL证书,但是80将仅代理HTTP。

From Kubernetes v1.9 onwards you can use predefined AWS SSL policies with HTTPS or SSL listeners for your Services. To see which policies are available for use, you can use the aws command line tool: 从kubernetes v1.9开始,您可以将预定义的awssl策略与https或ssl侦听器一起用于您的服务。要查看哪些策略可用,可以使用aws命令行工具:

aws elb describe-load-balancer-policies --query 'PolicyDescriptions[].PolicyName'

You can then specify any one of those policies using the “service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy“ annotation; for example: 然后,可以使用“service.beta.kubernetes.io/aws负载平衡器ssl协商策略”注释指定这些策略中的任何一个;例如:

    metadata:
      name: my-service
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: "ELBSecurityPolicy-TLS-1-2-2017-01"

PROXY protocol support on AWS

To enable PROXY protocol support for clusters running on AWS, you can use the following service annotation: 要对aws上运行的集群启用代理协议支持,可以使用以下服务注释:

    metadata:
      name: my-service
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"

Since version 1.3.0, the use of this annotation applies to all ports proxied by the ELB and cannot be configured otherwise. 由于版本号为0,此注释的应用程序适用于所有由端口管理的端口,不能进行其他配置。

ELB Access Logs on AWS

There are several annotations to manage access logs for ELB Services on AWS. aws上有几个注释用于管理elb服务的访问日志。

The annotation service.beta.kubernetes.io/aws-load-balancer-access-log-enabled controls whether access logs are enabled. 注释service.beta.kubernetes.io/aws load balancer access log enabled控制是否启用访问日志。

The annotation service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval controls the interval in minutes for publishing the access logs. You can specify an interval of either 5 or 60 minutes. annotation service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval以分钟为单位控制发布访问日志的间隔。您可以指定5分钟或60分钟的间隔。

The annotation service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name controls the name of the Amazon S3 bucket where load balancer access logs are stored. 注释service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name控制存储负载平衡器访问日志的amazon s3 bucket的名称。

The annotation service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix specifies the logical hierarchy you created for your Amazon S3 bucket. 注释service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix指定为amazon s3 bucket创建的逻辑层次结构。

    metadata:
      name: my-service
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-access-log-enabled: "true"
        # Specifies whether access logs are enabled for the load balancer 指定是否为负载平衡器启用访问日志
        service.beta.kubernetes.io/aws-load-balancer-access-log-emit-interval: "60"
        # The interval for publishing the access logs. You can specify an interval of either 5 or 60 (minutes).  发布访问日志的间隔。您可以指定5或60(分钟)的间隔。
        service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-name: "my-bucket"
        # The name of the Amazon S3 bucket where the access logs are stored 存储访问日志的amazon s3存储桶的名称
        service.beta.kubernetes.io/aws-load-balancer-access-log-s3-bucket-prefix: "my-bucket-prefix/prod"
        # The logical hierarchy you created for your Amazon S3 bucket, for example `my-bucket-prefix/prod` 为amazon s3存储桶创建的逻辑层次结构,例如'my bucket prefix/prod`

Connection Draining on AWS

Connection draining for Classic ELBs can be managed with the annotation service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled set to the value of "true". The annotation service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout can also be used to set maximum time, in seconds, to keep the existing connections open before deregistering the instances. 可以通过将注释service.beta.kubernetes.io/aws load balancer connection draining enabled设置为“true”来管理经典ELB的连接排出。注释'Service,beta,KurbNETS.IO/AWS负载均衡器连接排水超时'也可以用来设置最大时间,以秒为单位,在撤销实例之前保持现有的连接打开。

    metadata:
      name: my-service
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-connection-draining-enabled: "true"
        service.beta.kubernetes.io/aws-load-balancer-connection-draining-timeout: "60"

Other ELB annotations

There are other annotations to manage Classic Elastic Load Balancers that are described below. 下面将介绍用于管理经典弹性负载平衡器的其他注释。

    metadata:
      name: my-service
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
        # The time, in seconds, that the connection is allowed to be idle (no data has been sent over the connection) before it is closed by the load balancer  在负载平衡器关闭连接之前,允许其空闲(未通过连接发送任何数据)的时间(秒)

        service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
        # Specifies whether cross-zone load balancing is enabled for the load balancer

        service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: "environment=prod,owner=devops"
        # A comma-separated list of key-value pairs which will be recorded as
        # additional tags in the ELB.

        service.beta.kubernetes.io/aws-load-balancer-healthcheck-healthy-threshold: ""
        # The number of successive successful health checks required for a backend to
        # be considered healthy for traffic. Defaults to 2, must be between 2 and 10

        service.beta.kubernetes.io/aws-load-balancer-healthcheck-unhealthy-threshold: "3"
        # The number of unsuccessful health checks required for a backend to be
        # considered unhealthy for traffic. Defaults to 6, must be between 2 and 10

        service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: "20"
        # The approximate interval, in seconds, between health checks of an
        # individual instance. Defaults to 10, must be between 5 and 300
        service.beta.kubernetes.io/aws-load-balancer-healthcheck-timeout: "5"
        # The amount of time, in seconds, during which no response means a failed
        # health check. This value must be less than the service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval
        # value. Defaults to 5, must be between 2 and 60

        service.beta.kubernetes.io/aws-load-balancer-extra-security-groups: "sg-53fae93f,sg-42efd82e"
        # A list of additional security groups to be added to the ELB

Network Load Balancer support on AWS

FEATURE STATE: Kubernetes v1.15 beta

To use a Network Load Balancer on AWS, use the annotation service.beta.kubernetes.io/aws-load-balancer-type with the value set to nlb. 要在aws上使用网络负载平衡器,请使用annotation service.beta.kubernetes.io/aws-load-balancer-type,其值设置为nlb。

    metadata:
      name: my-service
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-type: "nlb"

Note: NLB only works with certain instance classes; see the AWS documentation on Elastic Load Balancing for a list of supported instance types.

Unlike Classic Elastic Load Balancers, Network Load Balancers (NLBs) forward the client’s IP address through to the node. If a Service’s .spec.externalTrafficPolicy is set to Cluster, the client’s IP address is not propagated to the end Pods. 与传统的弹性负载平衡器不同,网络负载平衡器(nlbs)将客户端的ip地址转发到节点。如果服务的.spec.externaltrafficpolicy设置为cluster,则客户端的IP地址不会传播到端pod。

By setting .spec.externalTrafficPolicy to Local, the client IP addresses is propagated to the end Pods, but this could result in uneven distribution of traffic. Nodes without any Pods for a particular LoadBalancer Service will fail the NLB Target Group’s health check on the auto-assigned .spec.healthCheckNodePort and not receive any traffic. 通过将.spec.externaltrafficpolicy设置为“local”,客户端IP地址将传播到终端pod,但这可能会导致流量分布不均。没有特定负载平衡器服务的任何播客的节点将无法通过NLB目标组对自动分配的.spec.healthchecknodeport的运行状况检查,并且不会接收任何流量。

In order to achieve even traffic, either use a DaemonSet, or specify a pod anti-affinity to not locate on the same node. 为了实现偶数通信量,可以使用守护进程,或者指定pod反关联以不在同一节点上定位。

You can also use NLB Services with the internal load balancer annotation. 您还可以将NLB服务与内部负载平衡器批注一起使用。

In order for client traffic to reach instances behind an NLB, the Node security groups are modified with the following IP rules 为了使客户端通信量能够到达NLB后面的实例,将使用以下IP规则修改节点安全组:

Rule Protocol Port(s) IpRange(s) IpRange Description
Health Check TCP NodePort(s) (.spec.healthCheckNodePort for .spec.externalTrafficPolicy = Local) VPC CIDR kubernetes.io/rule/nlb/health=
Client Traffic TCP NodePort(s) .spec.loadBalancerSourceRanges (defaults to 0.0.0.0/0) kubernetes.io/rule/nlb/client=
MTU Discovery ICMP 3,4 .spec.loadBalancerSourceRanges (defaults to 0.0.0.0/0) kubernetes.io/rule/nlb/mtu=

In order to limit which client IP’s can access the Network Load Balancer, specify loadBalancerSourceRanges. 为了限制哪些客户端IP可以访问网络负载平衡器,请指定loadbalancersourceranges。

spec:
  loadBalancerSourceRanges:
    - "143.231.0.0/16"

Note: If .spec.loadBalancerSourceRanges is not set, Kubernetes allows traffic from 0.0.0.0/0 to the Node Security Group(s). If nodes have public IP addresses, be aware that non-NLB traffic can also reach all instances in those modified security groups. 如果未设置.spec.loadbalancersourceranges,则kubernetes允许从0.0.0.0/0到节点安全组的通信。如果节点具有公共IP地址,请注意,非NLB通信也可以到达这些修改的安全组中的所有实例。

Type ExternalName

Services of type ExternalName map a Service to a DNS name, not to a typical selector such as my-service or cassandra. You specify these Services with the spec.externalName parameter. 类型为external name的服务将服务映射到dns名称,而不是像“my service”或“cassandra”这样的典型选择器。使用“spec.externalname”参数指定这些服务。

This Service definition, for example, maps the my-service Service in the prod namespace to my.database.example.com: 例如,此服务定义将“prod”命名空间中的my service服务映射到“my.database.example.com”:

apiVersion: v1
kind: Service
metadata:
  name: my-service
  namespace: prod
spec:
  type: ExternalName
  externalName: my.database.example.com

Note: ExternalName accepts an IPv4 address string, but as a DNS names comprised of digits, not as an IP address. ExternalNames that resemble IPv4 addresses are not resolved by CoreDNS or ingress-nginx because ExternalName is intended to specify a canonical DNS name. To hardcode an IP address, consider using headless Services. ExternalName接受IPv4地址字符串,但作为由数字组成的DNS名称,而不是作为IP地址。由于externalname用于指定规范的dns名称,因此coredns或ingres nginx无法解析类似于ipv4地址的externalname。要硬编码IP地址,请考虑使用

When looking up the host my-service.prod.svc.cluster.local, the cluster DNS Service returns a CNAME record with the value my.database.example.com. Accessing my-service works in the same way as other Services but with the crucial difference that redirection happens at the DNS level rather than via proxying or forwarding. Should you later decide to move your database into your cluster, you can start its Pods, add appropriate selectors or endpoints, and change the Service’s type. 查找主机my-service.prod.svc.cluster.local时,群集DNS服务返回值为my.database.example.com的CNAME记录。访问我的服务的工作方式与其他服务相同,但有一个关键的区别,重定向发生在dns级别,而不是通过代理或转发。如果以后决定将数据库移动到集群中,则可以启动其pod,添加适当的选择器或端点,并更改服务的类型。

Warning:

You may have trouble using ExternalName for some common protocols, including HTTP and HTTPS. If you use ExternalName then the hostname used by clients inside your cluster is different from the name that the ExternalName references. 对于某些常见协议(包括http和https),使用externalname可能会遇到问题。如果使用externalname,则集群内客户端使用的主机名与externalname引用的名称不同。

For protocols that use hostnames this difference may lead to errors or unexpected responses. HTTP requests will have a Host: header that the origin server does not recognize; TLS servers will not be able to provide a certificate matching the hostname that the client connected to. 对于使用主机名的协议,此差异可能会导致错误或意外响应。http请求将有一个host:header,源服务器无法识别;tls服务器将无法提供与客户端连接到的主机名匹配的证书。

Note: This section is indebted to the Kubernetes Tips - Part 1 blog post from Alen Komljen. 本节感谢Kubernetes Tips-第1部分Alen Komljen的博客文章。

External IPs

If there are external IPs that route to one or more cluster nodes, Kubernetes Services can be exposed on those externalIPs. Traffic that ingresses into the cluster with the external IP (as destination IP), on the Service port, will be routed to one of the Service endpoints. externalIPs are not managed by Kubernetes and are the responsibility of the cluster administrator. 如果有路由到一个或多个群集节点的外部IP,则可以在这些外部IP上公开Kubernetes服务。使用服务端口上的外部IP(作为目标IP)进入群集的通信将路由到其中一个服务终结点。外部性不由kubernetes管理,由集群管理员负责。

In the Service spec, externalIPs can be specified along with any of the ServiceTypes. In the example below, “my-service” can be accessed by clients on “80.11.12.10:80” (externalIP:port) 在服务规范中,externalips可以与任何服务类型一起指定。在下面的示例中,“我的服务”可由“80.11.12.10:80”上的客户端访问(externalip:port)

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: MyApp
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 9376
  externalIPs:
    - 80.11.12.10

Shortcomings

Using the userspace proxy for VIPs, work at small to medium scale, but will not scale to very large clusters with thousands of Services. The original design proposal for portals has more details on this. 使用面向VIP的用户空间代理,可以在中小型范围内工作,但不能扩展到具有数千个服务的非常大的集群。最初的门户网站设计方案对此有更多细节。

Using the userspace proxy obscures the source IP address of a packet accessing a Service. This makes some kinds of network filtering (firewalling) impossible. The iptables proxy mode does not obscure in-cluster source IPs, but it does still impact clients coming through a load balancer or node-port. 使用用户空间代理隐藏访问服务的数据包的源IP地址。这使得某些类型的网络过滤(防火墙)不可能实现。iptables代理模式在集群源ips中并不模糊,但它仍然会影响通过负载平衡器或节点端口的客户端。

The Type field is designed as nested functionality - each level adds to the previous. This is not strictly required on all cloud providers (e.g. Google Compute Engine does not need to allocate a NodePort to make LoadBalancer work, but AWS does) but the current API requires it. 类型字段被设计为嵌套功能-每个级别都添加到上一个级别。并非所有云提供商都严格要求这样做(例如,google compute engine不需要分配nodeport来让loadbalancer工作,但aws需要这样做),但当前的api需要这样做。

Virtual IP implementation

The previous information should be sufficient for many people who just want to use Services. However, there is a lot going on behind the scenes that may be worth understanding. 对于许多只想使用服务的人来说,前面的信息应该足够了。不过,幕后有很多事情可能值得理解。

Avoiding collisions

One of the primary philosophies of Kubernetes is that you should not be exposed to situations that could cause your actions to fail through no fault of your own. For the design of the Service resource, this means not making you choose your own port number for a if that choice might collide with someone else’s choice. That is an isolation failure. k8s的一个基本哲学是,你不应该暴露在可能导致你的行为失败的情况下,而不是你自己的过错。对于服务资源的设计,这意味着如果某个选择可能与其他人的选择冲突,则不会让您选择自己的端口号。这是一个隔离故障。

In order to allow you to choose a port number for your Services, we must ensure that no two Services can collide. Kubernetes does that by allocating each Service its own IP address. 为了允许您为服务选择端口号,我们必须确保两个服务不会发生冲突。kubernetes通过分配每个服务自己的ip地址来实现这一点。

To ensure each Service receives a unique IP, an internal allocator atomically updates a global allocation map in etcd prior to creating each Service. The map object must exist in the registry for Services to get IP address assignments, otherwise creations will fail with a message indicating an IP address could not be allocated. 为了确保每个服务接收到唯一的IP,内部分配器在创建每个服务之前自动更新ETCD中的全局分配映射。MAP对象必须存在于注册表中,以便获得IP地址分配服务,否则创建将失败,指示不能分配IP地址的消息。

In the control plane, a background controller is responsible for creating that map (needed to support migrating from older versions of Kubernetes that used in-memory locking). Kubernetes also uses controllers to checking for invalid assignments (eg due to administrator intervention) and for cleaning up allocated IP addresses that are no longer used by any Services. 在控制平面中,后台控制器负责创建该映射(需要支持从用于内存锁定的旧版本kubernetes迁移)。kubernetes还使用控制器来检查无效的分配(例如由于管理员的干预),以及清除已分配的不再被任何服务使用的ip地址。

Service IP addresses

Unlike Pod IP addresses, which actually route to a fixed destination, Service IPs are not actually answered by a single host. Instead, kube-proxy uses iptables (packet processing logic in Linux) to define virtual IP addresses which are transparently redirected as needed. When clients connect to the VIP, their traffic is automatically transported to an appropriate endpoint. The environment variables and DNS for Services are actually populated in terms of the Service’s virtual IP address (and port). 与实际路由到固定目的地的pod ip地址不同,服务ip实际上不是由单个主机应答的。相反,kube代理使用iptables(linux中的包处理逻辑)来定义虚拟ip地址,这些地址根据需要被透明地重定向。当客户机连接到VIP时,它们的通信量将自动传输到适当的端点。服务的环境变量和dns实际上是根据服务的虚拟ip地址(和端口)填充的。

kube-proxy supports three proxy modes—userspace, iptables and IPVS—which each operate slightly differently. kube代理支持三种代理模式:用户空间、iptables和ipv,每种模式的操作略有不同。

Userspace

As an example, consider the image processing application described above. When the backend Service is created, the Kubernetes master assigns a virtual IP address, for example 10.0.0.1. Assuming the Service port is 1234, the Service is observed by all of the kube-proxy instances in the cluster. When a proxy sees a new Service, it opens a new random port, establishes an iptables redirect from the virtual IP address to this new port, and starts accepting connections on it. 作为示例,考虑上面描述的图像处理应用程序。创建后端服务时,kubernetes主机会分配一个虚拟ip地址,例如10.0.0.1。假设服务端口是1234,集群中的所有kube代理实例都会观察到该服务。当代理看到一个新服务时,它会打开一个新的随机端口,建立一个从虚拟IP地址到这个新端口的iptables重定向,并开始接受该端口上的连接。

When a client connects to the Service’s virtual IP address, the iptables rule kicks in, and redirects the packets to the proxy’s own port. The “Service proxy” chooses a backend, and starts proxying traffic from the client to the backend. 当客户端连接到服务的虚拟IP地址时,iptables规则启动,并将数据包重定向到代理自身的端口。“服务代理”选择后端,并启动从客户端到后端的代理通信。

This means that Service owners can choose any port they want without risk of collision. Clients can simply connect to an IP and port, without being aware of which Pods they are actually accessing. 这意味着服务所有者可以选择他们想要的任何港口,而不会有碰撞的风险。客户机可以简单地连接到IP和端口,而不必知道他们实际访问的是哪个播客。

iptables

Again, consider the image processing application described above. When the backend Service is created, the Kubernetes control plane assigns a virtual IP address, for example 10.0.0.1. Assuming the Service port is 1234, the Service is observed by all of the kube-proxy instances in the cluster. When a proxy sees a new Service, it installs a series of iptables rules which redirect from the virtual IP address to per-Service rules. The per-Service rules link to per-Endpoint rules which redirect traffic (using destination NAT) to the backends. 再次,考虑上面描述的图像处理应用程序。创建后端服务时,kubernetes控制平面将分配一个虚拟IP地址,例如10.0.0.1。假设服务端口是1234,集群中的所有kube代理实例都会观察到该服务。当代理看到新服务时,它会安装一系列iptables规则,这些规则从虚拟ip地址重定向到每个服务规则。每个服务规则链接到每个端点规则,这些规则将流量(使用目标NAT)重定向到后端。

When a client connects to the Service’s virtual IP address the iptables rule kicks in. A backend is chosen (either based on session affinity or randomly) and packets are redirected to the backend. Unlike the userspace proxy, packets are never copied to userspace, the kube-proxy does not have to be running for the virtual IP address to work, and Nodes see traffic arriving from the unaltered client IP address. 当客户端连接到服务的虚拟IP地址时,iptables规则就会启动。选择一个后端(基于会话相关性或随机选择),然后将数据包重定向到后端。与userspace代理不同,数据包永远不会复制到userspace,kube代理不必运行以使虚拟ip地址工作,节点可以看到来自未更改的客户端ip地址的通信量。

This same basic flow executes when traffic comes in through a node-port or through a load-balancer, though in those cases the client IP does get altered. 当流量通过节点端口或负载平衡器传入时,也会执行相同的基本流,不过在这些情况下,客户端IP确实会发生更改。

IPVS

iptables operations slow down dramatically in large scale cluster e.g 10,000 Services. IPVS is designed for load balancing and based on in-kernel hash tables. So you can achieve performance consistency in large number of Services from IPVS-based kube-proxy. Meanwhile, IPVS-based kube-proxy has more sophisticated load balancing algorithms (least conns, locality, weighted, persistence). iptables在大规模集群(如10000个服务)中的运行速度显著减慢。ipvs是为负载平衡而设计的,它基于内核内的哈希表。因此,可以通过基于ipvs的kube代理在大量服务中实现性能一致性。同时,基于ipvs的kube代理有更复杂的负载平衡算法(最少连接、局部性、加权、持久性)。

API Object

Service is a top-level resource in the Kubernetes REST API. You can find more details about the API object at: Service API object. 服务是kubernetes rest api中的顶级资源。有关api对象的更多详细信息,请访问:serviceapi object。

Supported protocols

TCP

FEATURE STATE: Kubernetes v1.0 stable

You can use TCP for any kind of Service, and it’s the default network protocol. 您可以将TCP用于任何类型的服务,它是默认的网络协议。

UDP

FEATURE STATE: Kubernetes v1.0 stable

You can use UDP for most Services. For type=LoadBalancer Services, UDP support depends on the cloud provider offering this facility. 大多数服务都可以使用udp。对于type=loadbalancer服务,udp支持取决于提供此功能的云提供商。

HTTP

FEATURE STATE: Kubernetes v1.1 stable

If your cloud provider supports it, you can use a Service in LoadBalancer mode to set up external HTTP / HTTPS reverse proxying, forwarded to the Endpoints of the Service. 如果云提供商支持它,则可以使用loadbalancer模式的服务设置外部http/https反向代理,并将其转发到服务的端点。

Note: You can also use Ingress in place of Service to expose HTTP / HTTPS Services. 您还可以使用入口代替服务来公开http/https服务。

PROXY protocol

FEATURE STATE: Kubernetes v1.1 stable

If your cloud provider supports it (eg, AWS), you can use a Service in LoadBalancer mode to configure a load balancer outside of Kubernetes itself, that will forward connections prefixed with PROXY protocol. 如果您的云提供商支持它(如aws),则可以使用load balancer模式的服务在kubernetes自身之外配置一个负载平衡器,该服务将转发以代理协议为前缀的连接。

The load balancer will send an initial series of octets describing the incoming connection, similar to this example 负载平衡器将发送一个描述传入连接的初始八位字节序列,类似于本例

PROXY TCP4 192.0.2.202 10.0.42.7 12345 7\r\n

followed by the data from the client. 然后是来自客户端的数据。

SCTP

FEATURE STATE: Kubernetes v1.12 alpha

Kubernetes supports SCTP as a protocol value in Service, Endpoint, NetworkPolicy and Pod definitions as an alpha feature. To enable this feature, the cluster administrator needs to enable the SCTPSupport feature gate on the apiserver, for example, --feature-gates=SCTPSupport=true,…. kubernetes支持sctp作为协议值,在服务端点、networkpolicy和pod定义中作为alpha特性。要启用此功能,群集管理员需要在APIServer上启用SCTPSUPPORT功能门,例如,-feature gates=SCTPSUPPORT=true,…。

When the feature gate is enabled, you can set the protocol field of a Service, Endpoint, NetworkPolicy or Pod to SCTP. Kubernetes sets up the network accordingly for the SCTP associations, just like it does for TCP connections. 启用功能入口后,可以将服务、终结点、网络策略或POD的协议字段设置为SCTP。kubernetes为sctp关联相应地设置网络,就像它为tcp连接所做的那样。

Warnings

Support for multihomed SCTP associations

Warning:

The support of multihomed SCTP associations requires that the CNI plugin can support the assignment of multiple interfaces and IP addresses to a Pod. 支持多宿主sctp关联需要cni插件支持向pod分配多个接口和ip地址。

NAT for multihomed SCTP associations requires special logic in the corresponding kernel modules. 多宿主sctp关联的nat需要在相应的内核模块中使用特殊逻辑。

Service with type=LoadBalancer

Warning: You can only create a Service with type LoadBalancer plus protocol SCTP if the cloud provider’s load balancer implementation supports SCTP as a protocol. Otherwise, the Service creation request is rejected. The current set of cloud load balancer providers (Azure, AWS, CloudStack, GCE, OpenStack) all lack support for SCTP. 如果云提供程序的负载平衡器实现支持SCTP作为协议,则只能创建类型为load balancer plus protocol SCTP的服务。否则,服务创建请求将被拒绝。当前的云负载均衡器提供程序集(azure、aws、cloudstack、gce、openstack)都缺乏对sctp的支持。

Windows

Warning: SCTP is not supported on Windows based nodes. 基于Windows的节点不支持SCTP。

Userspace kube-proxy

Warning: The kube-proxy does not support the management of SCTP associations when it is in userspace mode. 警告:kube代理在用户空间模式下不支持sctp关联的管理。

Future work

In the future, the proxy policy for Services can become more nuanced than simple round-robin balancing, for example master-elected or sharded. We also envision that some Services will have “real” load balancers, in which case the virtual IP address will simply transport the packets there. 在未来,服务的代理策略可能会比简单的循环平衡(例如主选或分片)更加微妙。我们还设想,一些服务将有“真正的”负载平衡器,在这种情况下,虚拟IP地址将只是在那里传输数据包。

The Kubernetes project intends to improve support for L7 (HTTP) Services. kubernetes项目旨在改进对l7(http)服务的支持。

The Kubernetes project intends to have more flexible ingress modes for Services which encompass the current ClusterIP, NodePort, and LoadBalancer modes and more. KubNeNes项目打算为包含当前CultSIP、NodePort和LoadBalancer模式等服务的服务提供更灵活的入口模式。

What's next

Feedback

Was this page helpful?

k8s
本作品采用《CC 协议》,转载必须注明作者和本文链接
《L03 构架 API 服务器》
你将学到如 RESTFul 设计风格、PostMan 的使用、OAuth 流程,JWT 概念及使用 和 API 开发相关的进阶知识。
《G01 Go 实战入门》
从零开始带你一步步开发一个 Go 博客项目,让你在最短的时间内学会使用 Go 进行编码。项目结构很大程度上参考了 Laravel。
讨论数量: 0
(= ̄ω ̄=)··· 暂无内容!

讨论应以学习和精进为目的。请勿发布不友善或者负能量的内容,与人为善,比聪明更重要!