应用编排与管理 -- Job + CronJob

Job

作用

  1. 创建一个或多个 Pod 来确保指定数量的 Pod 可以成功地运行终止
  2. 跟踪 Pod 状态,根据配置及时重试失败的 Pod
  3. 确定依赖关系,保证上一个任务运行完毕后再运行下一个任务
  4. 控制任务并行度,并根据配置确保 Pod 队列大小

实践

YAML 文件

job.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
apiVersion: batch/v1
kind: Job
metadata:
name: pi
spec:
backoffLimit: 4 # 重试次数
template: # Pod Selector
spec:
containers:
- name: pi
image: perl
ports: null
command:
- perl
- '-Mbignum=bpi'
- '-wle'
- print bpi(2000)
restartPolicy: Never # 重启策略:Never / OnFailure / Always
paral-job.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
apiVersion: batch/v1
kind: Job
metadata:
name: parallel-job
spec:
completions: 8
parallelism: 2
template:
spec:
containers:
- name: show-date
image: ubuntu
command:
- /bin/sh
args:
- '-c'
- sleep 30; date
restartPolicy: OnFailure

启动 minikube

1
2
3
4
5
6
7
8
9
10
11
12
13
14
$ minikube start --vm-driver=virtualbox --kubernetes-version=1.15.5 --image-mirror-country='cn'
😄 minikube v1.18.1 on Darwin 10.12.6
✨ Using the virtualbox driver based on user configuration
✅ Using image repository registry.cn-hangzhou.aliyuncs.com/google_containers
👍 Starting control plane node minikube in cluster minikube
🔥 Creating virtualbox VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
🐳 Preparing Kubernetes v1.15.5 on Docker 20.10.3 ...
▪ Generating certificates and keys ...
▪ Booting up control plane ...
▪ Configuring RBAC rules ...
🔎 Verifying Kubernetes components...
▪ Using image registry.cn-hangzhou.aliyuncs.com/google_containers/storage-provisioner:v4 (global image repository)
🌟 Enabled addons: storage-provisioner, default-storageclass
🏄 Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

查看 Job

自动添加 Label:controller-uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
$ kubectl apply -f job.yaml
job.batch/pi created

$ kubectl get jobs
NAME COMPLETIONS DURATION AGE
pi 1/1 10s 15s

$ kubectl get jobs pi -o yaml | head -n 15
apiVersion: batch/v1
kind: Job
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"batch/v1","kind":"Job","metadata":{"annotations":{},"name":"pi","namespace":"default"},"spec":{"backoffLimit":4,"template":{"spec":{"containers":[{"command":["perl","-Mbignum=bpi","-wle","print bpi(2000)"],"image":"perl","name":"pi","ports":null}],"restartPolicy":"Never"}}}}
creationTimestamp: "2021-07-12T00:45:44Z"
labels:
controller-uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae
job-name: pi
name: pi
namespace: default
resourceVersion: "5330"
selfLink: /apis/batch/v1/namespaces/default/jobs/pi
uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae
COMPLETIONS DURATION AGE
完成 Pod 数量 Job 实际业务运行时长 Job 创建时长

查看 Pod

Pod 名称:${job-name}-${random-suffix},Pod OwnerReference 为 Job

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
$  kubectl get pods
NAME READY STATUS RESTARTS AGE
pi-q5b9h 0/1 Completed 0 82s

$ kubectl get pod pi-q5b9h -o yaml | head -n 17
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2021-07-12T00:45:44Z"
generateName: pi-
labels:
controller-uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae
job-name: pi
name: pi-q5b9h
namespace: default
ownerReferences:
- apiVersion: batch/v1
blockOwnerDeletion: true
controller: true
kind: Job
name: pi
uid: 9efacda0-f0a1-41e4-9a5c-de287690ecae

$ kubectl logs pi-vjzt4
3.14159265358979323846264338327950288419716939937510582097494459230781640628620899862803482534211706798214808651...

并行运行

Key Value
completions Job 可以运行的总次数
parallelism 并行执行的个数(管道或缓冲队列的大小)
1
2
3
4
5
6
7
8
9
10
11
12
13
$ kubectl apply -f parallel-job.yml
job.batch/parallel-job created

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
parallel-job-4md9g 0/1 Completed 0 53s
parallel-job-4pdvc 0/1 Completed 0 2m58s
parallel-job-d5mdr 0/1 Completed 0 57s
parallel-job-grwdp 0/1 Completed 0 2m7s
parallel-job-h2shl 0/1 Completed 0 92s
parallel-job-qxfmp 0/1 Completed 0 88s
parallel-job-rjkfc 0/1 Completed 0 2m58s
parallel-job-vcv8l 0/1 Completed 0 2m4s

架构设计

管理模式

image-20210712082830228

Job Controller

image-20210712083006743

CronJob

YAML 文件

CronJob -> Job -> Pod

cronjob.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: cron-job
spec:
schedule: '*/3 * * * *' # Linux Crontab
startingDeadlineSeconds: 10 # Job 最长启动时间
concurrencyPolicy: Allow # 是否允许并行
successfulJobsHistoryLimit: 3 # 允许保留历史 Job 个数
jobTemplate: # Job Template
spec:
template: # Pod Template
spec:
containers:
- name: hello
image: busybox
command:
- /bin/sh
args:
- '-c'
- date; echo Hello from the minikube
restartPolicy: OnFailure

查看 CronJob

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ kubectl apply -f cronjob.yml
cronjob.batch/cron-job created

$ kubectl get cronjobs
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
cron-job */3 * * * * False 1 7s 2m52s

$ kubectl get cronjobs cron-job -o yaml | head -n 12
apiVersion: batch/v1beta1
kind: CronJob
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"batch/v1beta1","kind":"CronJob","metadata":{"annotations":{},"name":"cron-job","namespace":"default"},"spec":{"concurrencyPolicy":"Allow","jobTemplate":{"spec":{"template":{"spec":{"containers":[{"args":["-c","date; echo Hello from the minikube"],"command":["/bin/sh"],"image":"busybox","name":"hello"}],"restartPolicy":"OnFailure"}}}},"schedule":"*/3 * * * *","startingDeadlineSeconds":10,"successfulJobsHistoryLimit":3}}
creationTimestamp: "2021-07-12T00:21:15Z"
name: cron-job
namespace: default
resourceVersion: "4084"
selfLink: /apis/batch/v1beta1/namespaces/default/cronjobs/cron-job
uid: cee590cc-3341-4fea-a2bd-d56337c6d6ca

查看 Job

Job 的 OwnerReferences 为 CronJob

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
$ kubectl get jobs
NAME COMPLETIONS DURATION AGE
cron-job-1626049440 1/1 4s 22s

$ kubectl get jobs cron-job-1626049440 -o yaml | head -n 19
apiVersion: batch/v1
kind: Job
metadata:
creationTimestamp: "2021-07-12T00:24:05Z"
labels:
controller-uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed
job-name: cron-job-1626049440
name: cron-job-1626049440
namespace: default
ownerReferences:
- apiVersion: batch/v1beta1
blockOwnerDeletion: true
controller: true
kind: CronJob
name: cron-job
uid: cee590cc-3341-4fea-a2bd-d56337c6d6ca
resourceVersion: "4223"
selfLink: /apis/batch/v1/namespaces/default/jobs/cron-job-1626049440
uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed

查看 Pod

Pod 的 OwnerReferences 为 Job

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
cron-job-1626049440-vfcb8 0/1 Completed 0 60s

$ kubectl logs cron-job-1626049440-vfcb8
Mon Jul 12 00:24:09 UTC 2021
Hello from the minikube

$ kubectl get pods cron-job-1626049440-vfcb8 -o yaml | head -n 20
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2021-07-12T00:24:05Z"
generateName: cron-job-1626049440-
labels:
controller-uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed
job-name: cron-job-1626049440
name: cron-job-1626049440-vfcb8
namespace: default
ownerReferences:
- apiVersion: batch/v1
blockOwnerDeletion: true
controller: true
kind: Job
name: cron-job-1626049440
uid: e871c373-06ff-4ae7-a6c3-333a5fdd20ed
resourceVersion: "4222"
selfLink: /api/v1/namespaces/default/pods/cron-job-1626049440-vfcb8
uid: 5dd30c4c-2fac-4bd0-80e4-9d1fc92d79b9

参考资料

  1. CNCF × Alibaba 云原生技术公开课
0%