k8s部署持续集成环境

时间：2020-03-05 19:21:53 收藏：0 阅读：152

k8s部署持续集成环境

安装前的准备：
关闭防火墙
关闭selinux
安装docker-ce
1、安装harbor参考前面笔记
2、安装并配置git，参考前面笔记
3、在git这台部署nfs服务器，并且在各个节点安装nfs-utils客户端
4、创建nfs-client-provisioner客户端
cat class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment‘s env PROVISIONER_NAME‘
parameters:
archiveOnDelete: "true"

cat deployment.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: nfs-client-provisioner
spec:
replicas: 1
strategy:
type: Recreate
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:

name: nfs-client-provisioner
image: lizhenliang/nfs-client-provisioner:latest
volumeMounts:
- name: nfs-client-root
  mountPath: /persistentvolumes
  env:
- name: PROVISIONER_NAME
  value: fuseim.pri/ifs
- name: NFS_SERVER
  value: 192.168.31.64
- name: NFS_PATH
  value: /ifs/kubernetes
  volumes:
name: nfs-client-root
nfs:
server: 192.168.31.64
path: /ifs/kubernetes
cat rbac.yaml
kind: ServiceAccount
apiVersion: v1
metadata:
name: nfs-client-provisioner

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
  resources: ["persistentvolumes"]
  verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
  resources: ["persistentvolumeclaims"]
  verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
  resources: ["storageclasses"]
  verbs: ["get", "list", "watch"]
- apiGroups: [""]
  resources: ["events"]
  verbs: ["create", "update", "patch"]
  
  kind: ClusterRoleBinding
  apiVersion: rbac.authorization.k8s.io/v1
  metadata:
  name: run-nfs-client-provisioner
  subjects:
- kind: ServiceAccount
  name: nfs-client-provisioner
  namespace: default
  roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
  
  kind: Role
  apiVersion: rbac.authorization.k8s.io/v1
  metadata:
  name: leader-locking-nfs-client-provisioner
  rules:
- apiGroups: [""]
  resources: ["endpoints"]
  verbs: ["get", "list", "watch", "create", "update", "patch"]
  
  kind: RoleBinding
  apiVersion: rbac.authorization.k8s.io/v1
  metadata:
  name: leader-locking-nfs-client-provisioner
  subjects:
- kind: ServiceAccount
  name: nfs-client-provisioner
  replace with namespace where provisioner is deployed
  
  namespace: default
  roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io
  5、部署jenkins服务器，前提是需要部署k8s的core-dns（安装coredns查看前面笔记），否则没法做解析就没法安装插件
  cat ingress.yml
  apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
  name: jenkins
  annotations:
  nginx.ingress.kubernetes.io/ssl-redirect: "true"
  kubernetes.io/tls-acme: "true"
  
  如果上传插件超出默认会报"413 Request Entity Too Large", 增加 client_max_body_size
  
  nginx.ingress.kubernetes.io/proxy-body-size: 50m
  nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
  
  nginx-ingress controller版本小于 0.9.0.beta-18 的配置
  
  ingress.kubernetes.io/ssl-redirect: "true"
  ingress.kubernetes.io/proxy-body-size: 50m
  ingress.kubernetes.io/proxy-request-buffering: "off"
  spec:
  rules:
- host: jenkins.example.com
  http:
  paths:
  - path: /
    backend:
    serviceName: jenkins
    servicePort: 80
    cat rbac.yml
    
    创建名为jenkins的ServiceAccount
    
    apiVersion: v1
    kind: ServiceAccount
    metadata:
    name: jenkins

创建名为jenkins的Role，授予允许管理API组的资源Pod

kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: jenkins
rules:

apiGroups: [""]
resources: ["pods"]
verbs: ["create","delete","get","list","patch","update","watch"]
apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create","delete","get","list","patch","update","watch"]
apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list","watch"]
apiGroups: [""]
resources: ["secrets"]
verbs: ["get"]

将名为jenkins的Role绑定到名为jenkins的ServiceAccount

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: jenkins
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: jenkins
subjects:

kind: ServiceAccount
name: jenkins

cat service.yml
apiVersion: v1
kind: Service
metadata:
name: jenkins
spec:
selector:
name: jenkins
type: NodePort
ports:

  name: http
  port: 80
  targetPort: 8080
  protocol: TCP
  nodePort: 30006
-
  name: agent
  port: 50000
  protocol: TCP

cat statefulset.yml
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
name: jenkins
labels:
name: jenkins
spec:
serviceName: jenkins
replicas: 1
updateStrategy:
type: RollingUpdate
template:
metadata:
name: jenkins
labels:
name: jenkins
spec:
terminationGracePeriodSeconds: 10
serviceAccountName: jenkins
containers: (如果出现dns没法解析的情况，需要在container这一行上面加一行dnsPolicy: Default)

name: jenkins
image: jenkins/jenkins
imagePullPolicy: Always
ports:
- containerPort: 8080
- containerPort: 50000
  resources:
  limits:
  cpu: 1
  memory: 1Gi
  requests:
  cpu: 0.5
  memory: 500Mi
  env:
- name: LIMITS_MEMORY
  valueFrom:
  resourceFieldRef:
  resource: limits.memory
  divisor: 1Mi
- name: JAVA_OPTS
  value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
  volumeMounts:
- name: jenkins-home
  mountPath: /var/jenkins_home
  livenessProbe:
  httpGet:
  path: /login
  port: 8080
  initialDelaySeconds: 60
  timeoutSeconds: 5
  failureThreshold: 12
  readinessProbe:
  httpGet:
  path: /login
  port: 8080
  initialDelaySeconds: 60
  timeoutSeconds: 5
  failureThreshold: 12
  securityContext:
  fsGroup: 1000
  volumeClaimTemplates:
  - metadata:
    name: jenkins-home
    spec:
    storageClassName: "managed-nfs-storage"
    accessModes: [ "ReadWriteOnce" ]
    resources:
    requests:
    storage: 1Gi
    6、使用这几个yaml文件启动安装jenkins
    PS：当提示以下出错时
    ‘FailedCreate‘ create Pod jenkins-0 in StatefulSet jenkins failed error: pods "jenkins-0" is forbidden: pod.Spec.SecurityContext.FSGroup is forbidden
    修改/opt/kubernetes/cfg/kube-apiserver，删除里面的安全字段SecurityContext
    7、使用浏览器访问http://10.1.2.190:30006进行安装(使用kubectl get svc -o wide查看jenkins当前在190这个pod运行)
    8、安装时不选择任何插件，使用手动安装插件，安装git和kubernetes插件

原文：https://blog.51cto.com/riverxyz/2475618

k8s部署持续集成环境

cat deployment.yaml apiVersion: v1kind: ServiceAccountmetadata:name: nfs-client-provisioner

name: nfs-client-rootnfs:server: 192.168.31.64path: /ifs/kubernetescat rbac.yaml kind: ServiceAccountapiVersion: v1metadata:name: nfs-client-provisioner

apiGroups: [""]resources: ["events"]verbs: ["create", "update", "patch"]

kind: ServiceAccountname: nfs-client-provisionernamespace: defaultroleRef:kind: ClusterRolename: nfs-client-provisioner-runnerapiGroup: rbac.authorization.k8s.io

apiGroups: [""]resources: ["endpoints"]verbs: ["get", "list", "watch", "create", "update", "patch"]

replace with namespace where provisioner is deployed

如果上传插件超出默认会报"413 Request Entity Too Large", 增加 client_max_body_size

nginx-ingress controller版本小于 0.9.0.beta-18 的配置

path: /backend:serviceName: jenkinsservicePort: 80cat rbac.yml

创建名为jenkins的ServiceAccount

创建名为jenkins的Role，授予允许管理API组的资源Pod

将名为jenkins的Role绑定到名为jenkins的ServiceAccount

cat service.yml apiVersion: v1kind: Servicemetadata:name: jenkinsspec:selector:name: jenkinstype: NodePortports:

cat deployment.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner

name: nfs-client-root
nfs:
server: 192.168.31.64
path: /ifs/kubernetes
cat rbac.yaml
kind: ServiceAccount
apiVersion: v1
metadata:
name: nfs-client-provisioner

apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]

kind: ServiceAccount
name: nfs-client-provisioner
namespace: default
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.io

apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]

path: /
backend:
serviceName: jenkins
servicePort: 80
cat rbac.yml

cat service.yml
apiVersion: v1
kind: Service
metadata:
name: jenkins
spec:
selector:
name: jenkins
type: NodePort
ports: