Kubernetes网络组件Calico安装

发表于 | 分类于 |

参照官网地址:

https://docs.projectcalico.org/v2.5/getting-started/kubernetes/installation/integration

Calico组件包括

  • calico/node， 必须安装在Master节点和每个计算节点上，包括BGP agent,负责网络策略。
  • cni/plugin， 和kubelet交互发现pod。
  • calico/policy-controller ,实现kubernetes的Network Policy API.

本文中安装的calico版本为2.5,对应的组件版本分别为:

calicoctl:v1.5.0
cni-plugin:v1.10.0
cni:v0.3.0

安装calico/node

1
2
3
4
5
$ wget https://github.com/projectcalico/calicoctl/releases/download/v1.5.0/calicoctl
$ sudo chmod +x calicoctl
$ mv calicoctl /usr/bin

创建calico-node.service

1
$ vi /usr/lib/systemd/system/calico-node.service

service的内容，需要将ETCD_ENDPOINTS中地址换成真实的etcd集群地址，可以将node-image中指定所需要的镜像名称，如果不指定，默认为quay.io/calico/node:latest

下边的命令会导致calio-node一直重启，所以需要换成官网的 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[Unit]
Description=calicoctl node
After=docker.service
Requires=docker.service
[Service]
User=root
Environment=ETCD_ENDPOINTS=http://172.21.1.201:2379
PermissionsStartOnly=true
ExecStart=/usr/bin/calicoctl node run --node-image=calico/node:v2.5.1
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target

换成(替换下边的ETCD_ENDPOINTS和ExecStart命令中的镜像名称)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
[Unit]
Description=calico node
After=docker.service
Requires=docker.service
[Service]
User=root
Environment=ETCD_ENDPOINTS=http://<ETCD_IP>:<ETCD_PORT>
PermissionsStartOnly=true
ExecStart=/usr/bin/docker run --net=host --privileged --name=calico-node \
  -e ETCD_ENDPOINTS=${ETCD_ENDPOINTS} \
  -e NODENAME=${HOSTNAME} \
  -e IP= \
  -e NO_DEFAULT_POOLS= \
  -e AS= \
  -e CALICO_LIBNETWORK_ENABLED=true \
  -e IP6= \
  -e CALICO_NETWORKING_BACKEND=bird \
  -e FELIX_DEFAULTENDPOINTTOHOSTACTION=ACCEPT \
  -v /var/run/calico:/var/run/calico \
  -v /lib/modules:/lib/modules \
  -v /run/docker/plugins:/run/docker/plugins \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /var/log/calico:/var/log/calico \
  calico/node:v2.5.1
ExecStop=/usr/bin/docker rm -f calico-node
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target

启动calico-node服务

1
2
$ systemctl daemon-reload
$ systemctl start calico-node

查看服务启动情况以及容器启动情况

1
2
3
4
5
6
7
8
9
10
11
12
$ systemctl status calico-node
● calico-node.service - calicoctl node
   Loaded: loaded (/usr/lib/systemd/system/calico-node.service; disabled; vendor preset: disabled)
   Active: activating (auto-restart) since Tue 2017-09-26 09:57:00 CST; 1s ago
  Process: 23696 ExecStart=/usr/bin/calicoctl node run --node-image=calico/node:v2.5.1 (code=exited, status=0/SUCCESS)
 Main PID: 23696 (code=exited, status=0/SUCCESS)
 
$ docker ps 
 
 CONTAINER ID        IMAGE                COMMAND             CREATED             STATUS              PORTS               NAMES
1cca427ef1a1        calico/node:v2.5.1   "start_runit"       10 seconds ago      Up 9 seconds                            calico-node
当calico-node的ExecStart配置成calicoctl run命令时会导致服务一直重启，如上表示服务未启动成功，名字为calico-node的容器会一直重启。按照如上所说的修改后，再次查看 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
$ systemctl status calico-node
● calico-node.service - calicoctl node
   Loaded: loaded (/usr/lib/systemd/system/calico-node.service; disabled; vendor preset: disabled)
   Active: active (running) since Wed 2017-09-27 10:57:28 CST; 2min 4s ago
  Process: 21915 ExecStop=/usr/bin/docker rm -f calico-node (code=exited, status=0/SUCCESS)
 Main PID: 21949 (docker)
   Memory: 6.6M
   CGroup: /system.slice/calico-node.service
           └─21949 /usr/bin/docker run --net=host --privileged --name=calico-node -e ETCD_ENDPOINTS=http://172.21.1.201:2379 -e NODENAME= -e IP= -e NO_DEFAULT_POOLS= -e AS= -e CALICO_LIBNE...
Sep 27 10:57:28 k1 systemd[1]: Started calicoctl node.
Sep 27 10:57:28 k1 systemd[1]: Starting calicoctl node...
Sep 27 10:57:28 k1 docker[21949]: Skipping datastore connection test
Sep 27 10:57:28 k1 docker[21949]: IPv4 address 172.21.1.200 discovered on interface eth0
Sep 27 10:57:28 k1 docker[21949]: No AS number configured on node resource, using global value
Sep 27 10:57:28 k1 docker[21949]: Using node name: k1
Sep 27 10:57:29 k1 docker[21949]: time="2017-09-27T02:57:29Z" level=info msg="Loading config from environment"
Sep 27 10:57:29 k1 docker[21949]: Starting libnetwork service
Sep 27 10:57:29 k1 docker[21949]: Calico node started successfully

安装calico/cni-plugin

1
2
3
4
$ wget https://github.com/projectcalico/cni-plugin/releases/download/v1.10.0/calico
$ wget https://github.com/projectcalico/cni-plugin/releases/download/v1.10.0/calico-ipam
$ chmod +x calico calico-ipam
$ mv calico calico-ipam /usr/bin

cni-plugin需要标准的CNI配置文件，创建配置文件。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ mkdir -p /etc/cni/net.d
$ cat >/etc/cni/net.d/10-calico.conf <<EOF
{
    "name": "calico-k8s-network",
    "cniVersion": "0.1.0",
    "type": "calico",
    "etcd_endpoints": "http://172.21.1.201:2379",
    "log_level": "info",
    "ipam": {
        "type": "calico-ipam",
        "k8s_api_root": "http://127.0.0.1:8080"
    },
    "policy": {
        "type": "k8s"
    },
    "kubernetes": {
        "kubeconfig": "/etc/cni/net.d/calico-kubeconfig"
    }
}
EOF

注意：policy选项为k8s,使用Kubernetes Network Policy来定义网络策略.

这里需要一个calico-kubeconfig的配置文件。在/etc/cni/net.d下创建
calico-kubeconfig配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Kubeconfig file for Calico CNI plugin.
apiVersion: v1
kind: Config
clusters:
- name: local
  cluster:
    insecure-skip-tls-verify: true
    server: https://172.21.1.200:8080
users:
- name: calico
contexts:
- name: calico-context
  context:
    cluster: local
    user: calico
current-context: calico-context

安装CNI plugin

1
2
3
$ wget https://github.com/containernetworking/cni/releases/download/v0.3.0/cni-v0.3.0.tgz
$ tar -zxvf cni-v0.3.0.tgz
$ cp loopback /opt/cni/bin/

安装calico networkpolicy-controller

创建yaml文件，文件地址下载地址：

https://docs.projectcalico.org/v2.5/getting-started/kubernetes/installation/policy-controller.yaml

内容如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
# Calico Version v2.5.1
# https://docs.projectcalico.org/v2.5/releases#v2.5.1
# This manifest includes the following component versions:
#   calico/kube-policy-controller:v0.7.0
# Create this manifest using kubectl to deploy
# the Calico policy controller on Kubernetes.
# It deploys a single instance of the policy controller.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: calico-policy-controller
  namespace: kube-system
  labels:
    k8s-app: calico-policy
spec:
  # Only a single instance of the policy controller should be
  # active at a time.  Since this pod is run as a Deployment,
  # Kubernetes will ensure the pod is recreated in case of failure,
  # removing the need for passive backups.
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      name: calico-policy-controller
      namespace: kube-system
      labels:
        k8s-app: calico-policy
    spec:
      hostNetwork: true
      containers:
        - name: calico-policy-controller
          # Make sure to pin this to your desired version.
          image: quay.io/calico/kube-policy-controller:v0.7.0
          env:
            # Configure the policy controller with the location of
            # your etcd cluster.
            - name: ETCD_ENDPOINTS
              value: "<ETCD_ENDPOINTS>"
            # Location of the Kubernetes API - this shouldn't need to be
            # changed so long as it is used in conjunction with
            # CONFIGURE_ETC_HOSTS="true".
            - name: K8S_API
              value: "https://kubernetes.default:443"
            # Configure /etc/hosts within the container to resolve
            # the kubernetes.default Service to the correct clusterIP
            # using the environment provided by the kubelet.
            # This removes the need for KubeDNS to resolve the Service.
            - name: CONFIGURE_ETC_HOSTS
              value: "true"

需要修改3处地址：
image(镜像名称),ETCD_ENDPOINTS(ETCD地址),K8S_API(kubenetes 的API地址)

修改kubelet

需要将kubelet中的网络指定为calico ,修改/usr/lib/system.d/system/kubelet.service，加入如下参数

1
2
3
--network-plugin=cni
--cni-conf-dir=/etc/cni/net.d
--cni-bin-dir=/opt/cni/bin

注意：kubernetes1.4版本之前不支持cni-conf-dir和cni-bin-dir参数。替换成–network-plugin-dir=/etc/cni/net.d

如果您觉得对您有帮助，谢谢您的赞赏！