08 安全重点 认证和授权

本节我们将开始学习将 K8S 应用于生产环境中至关重要的一环,权限控制。当然,不仅是 K8S 对于任何应用于生产环境中的系统,权限管理或者说访问控制都是很重要的。

整体概览

通过前面的学习,我们已经知道 K8S 中几乎所有的操作都需要经过 kube-apiserver 处理,所以为了安全起见,K8S 为它提供了三类安全访问的措施。分别是:用于识别用户身份的认证(Authentication),用于控制用户对资源访问的授权(Authorization)以及用于资源管理方面的准入控制(Admission Control)。

下面的图基本展示了这一过程。来自客户端的请求分别经过认证,授权,准入控制之后,才能真正执行。

当然,这里说基本展示是因为我们可以直接通过 kubectl proxy 的方式直接通过 HTTP 请求访问 kube-apiserver 而无需任何认证过程。

另外,也可通过在 kube-apiserver 所启动的机器上,直接访问启动时 --insecure-port 参数配置的端口进行绕过认证和授权,默认是 8080。为了避免安全问题,也可将此参数设置为 0 以规避问题。注意:这个参数和 --insecure-bind-address 都已过期,并将在未来的版本移除。

+-----------------------------------------------------------------------------------------------------------+
|                                                                                                           |
|               +---------------------------------------------------------------------------+    +--------+ |
|               |                                                                           |    |        | |
| +--------+    |   +------------------+   +----------------+   +--------------+   +------+ |    |        | |
| |        |    |   |                  |   |                |   | Admission    |   |      | |    |        | |
| | Client +------> | Authentication   +-> | Authorization  +-> | Control      +-> |Logic | +--> | Others | |
| |        |    |   |                  |   |                |   |              |   |      | |    |        | |
| +--------+    |   +------------------+   +----------------+   +--------------+   +------+ |    |        | |
|               |                                                                           |    |        | |
|               |                                                                           |    |        | |
|               |                          Kube-apiserver                                   |    |        | |
|               +---------------------------------------------------------------------------+    +--------+ |
|                                                                                                           |
+-----------------------------------------------------------------------------------------------------------+

认证(Authentication)

认证,无非是判断当前发起请求的用户身份是否正确。例如,我们通常登录服务器时候需要输入用户名和密码,或者 SSH Keys 之类的。

在讲认证前,我们应该先理一下 K8S 中的用户。

K8S 中有两类用户,一般用户及 Service Account

对集群操作的 API 都是与用户相关联的,或者被视为匿名请求。匿名请求可通过 kube-apiserver--anonymous-auth 参数进行控制,默认是开启的,匿名用户默认的用户名为 system:anonymous,所属组为 system:unauthenticated

理完 K8S 中的用户,我们来看下 K8S 中的认证机制。

K8S 支持以下认证机制:

可选择同时开启多个认证机制。比如当我们使用 kubeadm 创建集群时,默认便会开启 X509 客户端证书和引导 Token 等认证机制。

授权(Authorization)

授权,也就是在验证当前发起请求的用户是否有相关的权限。例如,我们在 Linux 系统中常见的文件夹权限之类的。

授权是以认证的结果为基础的,授权机制检查用户通过认证后的请求中所包含的属性来进行判断。

K8S 支持多种授权机制,用户想要正确操作资源,则必须获得授权,所以 K8S 默认情况下的权限都是拒绝。当某种授权机制通过或者拒绝后,便会立即返回,不再去请求其他的授权机制;当所有授权机制都未通过时便会返回 403 错误了。

K8S 支持以下授权机制:

角色(Role)

上面提到了 RBAC,为了能更好的理解,我们需要先认识下 K8S 中的角色。K8S 中的角色从类别上主要有两类,RoleClusterRole

当已经了解到角色后,剩下给用户授权也就只是需要做一次绑定即可。在 K8S 中将这一过程称之为 binding,即 rolebindingclusterrolebinding。 我们来看下集群刚初始化后的情况:

➜  ~ kubectl get roles --all-namespaces=true
NAMESPACE     NAME                                             AGE
kube-public   kubeadm:bootstrap-signer-clusterinfo             1h
kube-public   system:controller:bootstrap-signer               1h
kube-system   extension-apiserver-authentication-reader        1h
kube-system   kube-proxy                                       1h
kube-system   kubeadm:kubelet-config-1.12                      1h
kube-system   kubeadm:nodes-kubeadm-config                     1h
kube-system   system::leader-locking-kube-controller-manager   1h
kube-system   system::leader-locking-kube-scheduler            1h
kube-system   system:controller:bootstrap-signer               1h
kube-system   system:controller:cloud-provider                 1h
kube-system   system:controller:token-cleaner                  1h
➜  ~ kubectl get rolebindings --all-namespaces=true
NAMESPACE     NAME                                             AGE
kube-public   kubeadm:bootstrap-signer-clusterinfo             1h
kube-public   system:controller:bootstrap-signer               1h
kube-system   kube-proxy                                       1h
kube-system   kubeadm:kubelet-config-1.12                      1h
kube-system   kubeadm:nodes-kubeadm-config                     1h
kube-system   system::leader-locking-kube-controller-manager   1h
kube-system   system::leader-locking-kube-scheduler            1h
kube-system   system:controller:bootstrap-signer               1h
kube-system   system:controller:cloud-provider                 1h
kube-system   system:controller:token-cleaner                  1h

可以看到默认已经存在了一些 rolerolebindings。 对于这部分暂且不做过多说明,我们来看下对于集群全局有效的 ClusterRole

➜  ~ kubectl get clusterroles
NAME                                                                   AGE
admin                                                                  1h
cluster-admin                                                          1h
edit                                                                   1h
flannel                                                                1h
system:aggregate-to-admin                                              1h
system:aggregate-to-edit                                               1h
system:aggregate-to-view                                               1h
system:auth-delegator                                                  1h
system:aws-cloud-provider                                              1h
system:basic-user                                                      1h
system:certificates.k8s.io:certificatesigningrequests:nodeclient       1h
system:certificates.k8s.io:certificatesigningrequests:selfnodeclient   1h
system:controller:attachdetach-controller                              1h
system:controller:certificate-controller                               1h
system:controller:clusterrole-aggregation-controller                   1h
system:controller:cronjob-controller                                   1h
system:controller:daemon-set-controller                                1h
system:controller:deployment-controller                                1h
system:controller:disruption-controller                                1h
system:controller:endpoint-controller                                  1h
system:controller:expand-controller                                    1h
system:controller:generic-garbage-collector                            1h
system:controller:horizontal-pod-autoscaler                            1h
system:controller:job-controller                                       1h
system:controller:namespace-controller                                 1h
system:controller:node-controller                                      1h
system:controller:persistent-volume-binder                             1h
system:controller:pod-garbage-collector                                1h
system:controller:pv-protection-controller                             1h
system:controller:pvc-protection-controller                            1h
system:controller:replicaset-controller                                1h
system:controller:replication-controller                               1h
system:controller:resourcequota-controller                             1h
system:controller:route-controller                                     1h
system:controller:service-account-controller                           1h
system:controller:service-controller                                   1h
system:controller:statefulset-controller                               1h
system:controller:ttl-controller                                       1h
system:coredns                                                         1h
system:csi-external-attacher                                           1h
system:csi-external-provisioner                                        1h
system:discovery                                                       1h
system:heapster                                                        1h
system:kube-aggregator                                                 1h
system:kube-controller-manager                                         1h
system:kube-dns                                                        1h
system:kube-scheduler                                                  1h
system:kubelet-api-admin                                               1h
system:node                                                            1h
system:node-bootstrapper                                               1h
system:node-problem-detector                                           1h
system:node-proxier                                                    1h
system:persistent-volume-provisioner                                   1h
system:volume-scheduler                                                1h
view                                                                   1h
➜  ~ kubectl get clusterrolebindings
NAME                                                   AGE
cluster-admin                                          1h
flannel                                                1h
kubeadm:kubelet-bootstrap                              1h
kubeadm:node-autoapprove-bootstrap                     1h
kubeadm:node-autoapprove-certificate-rotation          1h
kubeadm:node-proxier                                   1h
system:aws-cloud-provider                              1h
system:basic-user                                      1h
system:controller:attachdetach-controller              1h
system:controller:certificate-controller               1h
system:controller:clusterrole-aggregation-controller   1h
system:controller:cronjob-controller                   1h
system:controller:daemon-set-controller                1h
system:controller:deployment-controller                1h
system:controller:disruption-controller                1h
system:controller:endpoint-controller                  1h
system:controller:expand-controller                    1h
system:controller:generic-garbage-collector            1h
system:controller:horizontal-pod-autoscaler            1h
system:controller:job-controller                       1h
system:controller:namespace-controller                 1h
system:controller:node-controller                      1h
system:controller:persistent-volume-binder             1h
system:controller:pod-garbage-collector                1h
system:controller:pv-protection-controller             1h
system:controller:pvc-protection-controller            1h
system:controller:replicaset-controller                1h
system:controller:replication-controller               1h
system:controller:resourcequota-controller             1h
system:controller:route-controller                     1h
system:controller:service-account-controller           1h
system:controller:service-controller                   1h
system:controller:statefulset-controller               1h
system:controller:ttl-controller                       1h
system:coredns                                         1h
system:discovery                                       1h
system:kube-controller-manager                         1h
system:kube-dns                                        1h
system:kube-scheduler                                  1h
system:node                                            1h
system:node-proxier                                    1h
system:volume-scheduler                                1h

可以看到 K8S 中默认已经有很多的 ClusterRoleclusterrolebindings 了,我们选择其中一个做下探究。

查看用户权限

我们一直都在使用 kubectl 对集群进行操作,那么当前用户是什么权限呢? 对应于 RBAC 中又是什么情况呢?

➜  ~ kubectl config current-context   # 获取当前上下文
kubernetes-admin@kubernetes  # 名为 kubernetes-admin 的用户,在名为 kubernetes 的 cluster 上
➜  ~ kubectl config view users -o yaml  # 查看 user 配置,以下省略了部分内容
apiVersion: v1
clusters:                                   
- cluster:                            
    ...
contexts:                    
- context:  
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

client-certificate-data 的部分默认是不显示的,而它的内容实际是通过 base64 加密后的证书内容。我们可以通过通过以下方式进行查看

➜  ~ kubectl config view users --raw -o jsonpath='{ .users[?(@.name == "kubernetes-admin")].user.client-certificate-data}' |base64 -d  
-----BEGIN CERTIFICATE-----
MIIC8jCCAdqgAwIBAgIIGuC27C9B8LIwDQYJKoZIhvcNAQELBQAwFTETMBEGA1UE
...
kae1A/d4D5Cm5Qt7M5gr3SxqE5t+O7DP0YhuEPlfY7RzYDksYa8=
-----END CERTIFICATE-----
➜  ~ kubectl config view users --raw -o jsonpath='{ .users[?(@.name == "kubernetes-admin")].user.client-certificate-data}' |base64 -d |openssl x509 -text -noout  # 限于篇幅 省略部分输出
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1936748965290700978 (0x1ae0b6ec2f41f0b2)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=kubernetes
        Subject: O=system:masters, CN=kubernetes-admin
        ...

根据前面认证部分的内容,我们知道当前的用户是 kubernetes-admin (CN 域),所属组是 system:masters (O 域) 。

我们看下 clusterrolebindings 中的 cluster-admin

➜  ~ kubectl get clusterrolebindings  cluster-admin  -o yaml                         
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  ...
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: cluster-admin
  resourceVersion: "116"
  selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/cluster-admin
  uid: 71c550f1-e0e4-11e8-866a-fa163e938a99
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:masters

重点内容在 roleRefsubjects 中,名为 cluster-adminClusterRole 与名为 system:mastersGroup 相绑定。我们继续探究下它们所代表的含义。

先看看这个 ClusterRole 的实际内容:

➜  ~ kubectl get clusterrole cluster-admin -o yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  ...
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
  name: cluster-admin
  resourceVersion: "58"
  selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-admin
  uid: 71307108-e0e4-11e8-866a-fa163e938a99
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - '*'
- nonResourceURLs:
  - '*'
  verbs:
  - '*'

rules 中定义了它所能操作的资源及对应动作,* 是通配符。

到这里,我们就可以得出结论了,当前用户 kubernetes-admin 属于 system:masters 组,而这个组与 cluster-admin 这个 ClusterRole 所绑定,所以用户也就继承了其权限。具备了对多种资源和 API 的相关操作权限。

实践:创建权限可控的用户

前面是通过实际用户来反推它所具备的权限,接下来我们开始实践的部分,创建用户并为它进行授权。

我们要创建的用户名为 backend 所属组为 dev

创建 NameSpace

为了演示,这里创建一个新的 NameSpace ,名为 work

➜  ~ kubectl create namespace work
namespace/work created
➜  ~ kubectl get ns work
NAME   STATUS   AGE
work   Active   14s

创建用户

创建私钥

➜  ~ mkdir work
➜  ~ cd work
➜  work openssl genrsa -out backend.key 2048
Generating RSA private key, 2048 bit long modulus
..........................................+++                            
........................+++                   
e is 65537 (0x10001)            
➜  work ls                                       
backend.key 

➜  work cat backend.key                                              
-----BEGIN RSA PRIVATE KEY-----                                          
MIIEpAIBAAKCAQEAzk7blZthwSzachPxrk6pHsuaImTVh6Iw8mNDmtn6sqOqBfZS
...
bNKDWDk8HZREugaOAwjt7xaWOlr9SPCCoXrWoaA1z2215IC4qSA2Nw==
-----END RSA PRIVATE KEY-----

使用私钥生成证书请求。前面已经讲过关于认证的部分,在这里需要指定 subject 信息,传递用户名和组名

➜  work openssl req -new -key backend.key -out backend.csr -subj "/CN=backend/O=dev"
➜  work ls
backend.csr  backend.key
➜  work cat backend.csr
-----BEGIN CERTIFICATE REQUEST-----
MIICZTCCAU0CAQAwIDEQMA4GA1UEAwwHYmFja2VuZDEMMAoGA1UECgwDZGV2MIIB
...
lpoSVlNA0trJoiEiZjUqMfXX6ogBhQC4aeRfmbXkW2ZCNxsIm3PDk1Y=
-----END CERTIFICATE REQUEST-----

使用 CA 进行签名。K8S 默认的证书目录为 /etc/kubernetes/pki

➜  work openssl x509 -req -in backend.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out backend.crt -days 365
Signature ok
subject=/CN=backend/O=dev
Getting CA Private Key
➜  work ls
backend.crt  backend.csr  backend.key

查看生成的证书文件

➜  work openssl x509 -in backend.crt -text -noout
Certificate:
    Data:
        Version: 1 (0x0)
        Serial Number:
            d9:7f:62:f7:38:66:2a:7b
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=kubernetes
        Subject: CN=backend, O=dev
        ...

可以看到 CN 域和 O 域已经正确设置

添加 context

➜  work kubectl config set-credentials backend --client-certificate=/root/work/backend.crt  --client-key=/root/work/backend.key
User "backend" set.
➜  work kubectl config set-context backend-context --cluster=kubernetes --namespace=work --user=backend
Context "backend-context" created.

使用新用户测试访问

➜  work kubectl --context=backend-context get pods
Error from server (Forbidden): pods is forbidden: User "backend" cannot list resource "pods" in API group "" in the namespace "work"
# 可能看得不够清楚,我们添加 `-v` 参数来显示详情
➜  work kubectl --context=backend-context get pods -n work -v 5
I1109 05:35:11.870639   18626 helpers.go:201] server response object: [{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "pods is forbidden: User \"backend\" cannot list resource \"pods\" in API group \"\" in the namespace \"work\"",
  "reason": "Forbidden",
  "details": {
    "kind": "pods"
  },
  "code": 403
}]
F1109 05:35:11.870688   18626 helpers.go:119] Error from server (Forbidden): pods is forbidden: User "backend" cannot list resource "pods" in API group "" in the namespace "work"

可以看到已经使用了新的 backend 用户,并且默认的 Namespace 设置成了 work

创建 Role

我们想要让这个用户只具备查看 Pod 的权限。先来创建一个配置文件。

➜  work cat <<EOF > backend-role.yaml 
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: work
  name: backend-role
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "watch"]
EOF

创建并查看已生成的 Role

➜  work kubectl create -f backend-role.yaml 
role.rbac.authorization.k8s.io/backend-role created
➜  work kubectl get roles  -n work -o yaml
apiVersion: v1
items:
- apiVersion: rbac.authorization.k8s.io/v1
  kind: Role
  metadata:
    ...
    name: backend-role
    namespace: work
    selfLink: /apis/rbac.authorization.k8s.io/v1/namespaces/work/roles/backend-role
  rules:
  - apiGroups:
    - ""
    resources:
    - pods
    verbs:
    - get
    - list
    - watch
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

创建 Rolebinding

先创建一个配置文件。

➜  work cat <<EOF > backend-rolebind.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: backend-rolebinding          
  namespace: work
subjects:      
- kind: User
  name: backend
  apiGroup: ""     
roleRef:    
  kind: Role 
  name: backend-role
  apiGroup: ""
EOF

创建并查看已生成的 Rolebinding 。

➜  work kubectl create -f backend-rolebind.yaml
rolebinding.rbac.authorization.k8s.io/backend-rolebinding created
➜  work kubectl get rolebinding -o yaml -n work
apiVersion: v1
items:
- apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    name: backend-rolebinding
    namespace: work
    ...
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: Role
    name: backend-role
  subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: User
    name: backend
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

测试用户权限

➜  work kubectl --context=backend-context get pods -n work
No resources found.
➜  work kubectl --context=backend-context get ns
Error from server (Forbidden): namespaces is forbidden: User "backend" cannot list resource "namespaces" in API group "" at the cluster scope
➜  work kubectl --context=backend-context get deploy -n work
Error from server (Forbidden): deployments.extensions is forbidden: User "backend" cannot list resource "deployments" in API group "extensions" in the namespace "work"

可以看到用户已经具备查看 Pod 的权限,但并不能查看 Namespace 或者 deployment 等其他资源。当然,K8S 也内置了一种很方便的调试机制。

➜  work kubectl auth can-i list pods -n work --as="backend"
yes
➜  work kubectl auth can-i list deploy -n work --as="backend"
no - no RBAC policy matched

--as 是一种建立在 K8S 认证机制之上的机制,可以便于系统管理员验证授权情况,或进行调试。

你也可以仿照 ~/.kube/config 文件的内容,将当前生成的证书及私钥文件等写入到配置文件中,通过指定 KUBECONFIG 的环境变量,或者给 kubectl 传递 --kubeconfig 参数来使用。

总结

本节中,我们学习了 K8S 的认证及授权逻辑,K8S 支持多种认证及授权模式,可按需使用。通过 X509 客户端证书认证的方式使用比较方便也比较推荐,在客户端证书的 CN 域和 O 域可以指定用户名和所属组名。

RBAC 的授权模式现在使用最多,可以通过对 Rolesubjects (可以是用户或组) 进行绑定,以达到授权的目的。

最后,我们实际新创建了一个用户,并对其授予了预期的权限。在此过程中也涉及到了 openssl 客户端的常规操作,在之后也会常常用到。

下节,我们将开始部署实际的项目到 K8S 中,逐步掌握生成环境中对 K8S 的使用实践。

PS: 也许你会觉得切换 Namespace 之类的操作很繁琐,有一个项目:kubectx 可帮你简化这些步骤,推荐尝试。