版本：Next

分离集群模式

分离集群模式中，SeaTunnel Engine 的 Master 与 Worker 分别运行在独立进程中。Master 负责作业调度、REST API、任务提交和 IMap 状态存储；Worker 负责执行任务，不参与 Master 选举，也不存储 IMap 数据。

这是 Kubernetes 生产环境推荐的部署方式。Master 和 Worker 都应使用 StatefulSet 部署，避免滚动更新、节点驱逐或重新调度时同时终止过多节点，导致选主抖动或可用 slot 突然不足。

组件	推荐工作负载	最小副本数	生产建议
Master	StatefulSet	1	至少 2 个，用于 HA 和 IMap 备份
Worker	StatefulSet	1	按任务并行度和 slot 规划扩缩容
Hazelcast discovery	Headless Service	1	`publishNotReadyAddresses: true`
REST API	ClusterIP Service	1	可按需通过 Ingress 或 LoadBalancer 暴露

创建 ConfigMap

建议按角色拆分 ConfigMap，避免单个 YAML 过长，也便于分别更新 Master、Worker 和客户端配置。生产环境中的访问密钥、密码和 token 应通过 Secret 管理，不应直接写入 ConfigMap。

Master Hazelcast 配置

apiVersion: v1
kind: ConfigMap
metadata:
  name: seatunnel-master-hazelcast-config
data:
  hazelcast-master.yaml: |
    hazelcast:
      cluster-name: seatunnel-cluster
      network:
        rest-api:
          enabled: true
          endpoint-groups:
            CLUSTER_WRITE:
              enabled: true
            DATA:
              enabled: true
        port:
          auto-increment: false
          port: 5801
        join:
          kubernetes:
            enabled: true
            namespace: default
            service-name: seatunnel-cluster
            service-port: 5801
      map:
        engine*:
          map-store:
            enabled: true
            initial-mode: EAGER
            factory-class-name: org.apache.seatunnel.engine.server.persistence.FileMapStoreFactory
            properties:
              type: hdfs
              namespace: /seatunnel/imap
              clusterName: seatunnel-cluster
              storage.type: hdfs
              fs.defaultFS: hdfs://namenode:8020
      properties:
        hazelcast.invocation.max.retry.count: 20
        hazelcast.tcp.join.port.try.count: 30
        hazelcast.logging.type: log4j2
        hazelcast.operation.generic.thread.count: 50
        hazelcast.heartbeat.failuredetector.type: phi-accrual
        hazelcast.heartbeat.interval.seconds: 2
        hazelcast.max.no.heartbeat.seconds: 180
        hazelcast.heartbeat.phiaccrual.failuredetector.threshold: 10
        hazelcast.heartbeat.phiaccrual.failuredetector.sample.size: 200
        hazelcast.heartbeat.phiaccrual.failuredetector.min.std.dev.millis: 100

Worker Hazelcast 配置

apiVersion: v1
kind: ConfigMap
metadata:
  name: seatunnel-worker-hazelcast-config
data:
  hazelcast-worker.yaml: |
    hazelcast:
      cluster-name: seatunnel-cluster
      network:
        rest-api:
          enabled: true
          endpoint-groups:
            CLUSTER_WRITE:
              enabled: true
            DATA:
              enabled: true
        port:
          auto-increment: false
          port: 5801
        join:
          kubernetes:
            enabled: true
            namespace: default
            service-name: seatunnel-cluster
            service-port: 5801
      properties:
        hazelcast.invocation.max.retry.count: 20
        hazelcast.tcp.join.port.try.count: 30
        hazelcast.logging.type: log4j2
        hazelcast.operation.generic.thread.count: 50
        hazelcast.heartbeat.failuredetector.type: phi-accrual
        hazelcast.heartbeat.interval.seconds: 2
        hazelcast.max.no.heartbeat.seconds: 180
        hazelcast.heartbeat.phiaccrual.failuredetector.threshold: 10
        hazelcast.heartbeat.phiaccrual.failuredetector.sample.size: 200
        hazelcast.heartbeat.phiaccrual.failuredetector.min.std.dev.millis: 100
      member-attributes:
        rule:
          type: string
          value: worker

这些示例使用 Hazelcast Kubernetes API 发现。如果希望使用 DNS 发现，并避免 Hazelcast 访问 Kubernetes API，可以将 hazelcast-master.yaml 和 hazelcast-worker.yaml 中的 join.kubernetes 都替换为：

join:
  kubernetes:
    enabled: true
    service-dns: seatunnel-cluster.default.svc.cluster.local
    service-dns-timeout: 10

说明

使用 DNS 发现时，下面的 RBAC 章节不是成员发现所必需的。如果跳过 RBAC 清单，也需要从两个 StatefulSet 中移除 serviceAccountName: seatunnel，或单独创建这个 ServiceAccount。

Hazelcast Client 配置

apiVersion: v1
kind: ConfigMap
metadata:
  name: seatunnel-client-config
data:
  hazelcast-client.yaml: |
    hazelcast-client:
      cluster-name: seatunnel-cluster
      properties:
        hazelcast.logging.type: log4j2
      connection-strategy:
        connection-retry:
          cluster-connect-timeout-millis: 7000
      network:
        cluster-members:
          # 如果 SeaTunnel 部署在其他命名空间，需要将 default 替换为实际命名空间。
          - seatunnel-cluster.default.svc.cluster.local:5801

SeaTunnel Engine 配置

apiVersion: v1
kind: ConfigMap
metadata:
  name: seatunnel-engine-config
data:
  seatunnel.yaml: |
    seatunnel:
      engine:
        backup-count: 1
        history-job-expire-minutes: 1440
        print-execution-info-interval: 300
        classloader-cache-mode: true
        telemetry:
          metric:
            enabled: false
          logs:
            scheduled-deletion-enable: true
        slot-service:
          dynamic-slot: false
          slot-num: 8
        job-schedule-strategy: WAIT
        checkpoint:
          interval: 180000
          timeout: 30000
          storage:
            type: hdfs
            max-retained: 3
            plugin-config:
              storage.type: hdfs
              namespace: /seatunnel/checkpoint/
              fs.defaultFS: hdfs://namenode:8020
        http:
          enable-http: true
          port: 8080

注意

如果使用 S3、OSS、COS、OBS、TOS 等对象存储作为 checkpoint 或 MapStore 后端，需要保证每个 Pod 都具备网络访问能力和对应凭据。不要在 ConfigMap 的 YAML 内容中写 secretKeyRef；需要在部署前渲染最终配置，或使用 Hadoop credential provider、云厂商凭据链、挂载凭据文件等底层文件系统支持的方式。

为 API 发现创建 RBAC

hazelcast-master.yaml 和 hazelcast-worker.yaml 默认使用的 namespace、service-name 和 service-port 属于 Hazelcast Kubernetes API 发现。在启用 RBAC 的集群中，请先创建 ServiceAccount、Role 和 RoleBinding，再启动 StatefulSet。如果改用 service-dns 发现，可以跳过本节：

apiVersion: v1
kind: ServiceAccount
metadata:
  name: seatunnel
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: seatunnel-hazelcast-discovery
rules:
  - apiGroups: [""]
    resources: ["pods", "services", "endpoints"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: seatunnel-hazelcast-discovery
subjects:
  - kind: ServiceAccount
    name: seatunnel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: seatunnel-hazelcast-discovery

创建 Service

seatunnel-cluster 是 Headless Service，用于 Hazelcast 成员发现。Master 和 Worker 都会通过该服务加入同一个集群。

apiVersion: v1
kind: Service
metadata:
  name: seatunnel-cluster
  labels:
    app: seatunnel-cluster
spec:
  clusterIP: None
  publishNotReadyAddresses: true
  ports:
    - name: hazelcast
      port: 5801
      targetPort: 5801
  selector:
    app: seatunnel
---
apiVersion: v1
kind: Service
metadata:
  name: seatunnel-master
  labels:
    app: seatunnel-master
    component: master
spec:
  type: ClusterIP
  ports:
    - name: rest-api
      port: 8080
      targetPort: 8080
    - name: hazelcast
      port: 5801
      targetPort: 5801
  selector:
    app: seatunnel
    component: master
---
apiVersion: v1
kind: Service
metadata:
  name: seatunnel-worker
  labels:
    app: seatunnel-worker
    component: worker
spec:
  type: ClusterIP
  ports:
    - name: rest-api
      port: 8080
      targetPort: 8080
    - name: hazelcast
      port: 5801
      targetPort: 5801
  selector:
    app: seatunnel
    component: worker

创建 Master StatefulSet

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: seatunnel-master
  labels:
    app: seatunnel
    component: master
spec:
  serviceName: seatunnel-cluster
  replicas: 2
  selector:
    matchLabels:
      app: seatunnel
      component: master
  template:
    metadata:
      labels:
        app: seatunnel
        component: master
    spec:
      serviceAccountName: seatunnel
      containers:
        - name: app
          image: seatunnel:3.0.0
          imagePullPolicy: IfNotPresent
          command:
            - /opt/seatunnel/bin/seatunnel-cluster.sh
            - -r
            - master
          env:
            - name: SEATUNNEL_HOME
              value: /opt/seatunnel
            - name: HAZELCAST_CLUSTER_NAME
              value: seatunnel-cluster
          ports:
            - containerPort: 8080
              name: rest-api
            - containerPort: 5801
              name: hazelcast
          resources:
            requests:
              cpu: 500m
              memory: 3Gi
            limits:
              cpu: "1"
              memory: 4Gi
          volumeMounts:
            - name: hazelcast-master-config
              mountPath: /opt/seatunnel/config/hazelcast-master.yaml
              subPath: hazelcast-master.yaml
            - name: client-config
              mountPath: /opt/seatunnel/config/hazelcast-client.yaml
              subPath: hazelcast-client.yaml
            - name: engine-config
              mountPath: /opt/seatunnel/config/seatunnel.yaml
              subPath: seatunnel.yaml
      terminationGracePeriodSeconds: 120
      volumes:
        - name: hazelcast-master-config
          configMap:
            name: seatunnel-master-hazelcast-config
        - name: client-config
          configMap:
            name: seatunnel-client-config
        - name: engine-config
          configMap:
            name: seatunnel-engine-config

创建 Worker StatefulSet

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: seatunnel-worker
  labels:
    app: seatunnel
    component: worker
spec:
  serviceName: seatunnel-cluster
  replicas: 2
  selector:
    matchLabels:
      app: seatunnel
      component: worker
  template:
    metadata:
      labels:
        app: seatunnel
        component: worker
    spec:
      serviceAccountName: seatunnel
      containers:
        - name: app
          image: seatunnel:3.0.0
          imagePullPolicy: IfNotPresent
          command:
            - /opt/seatunnel/bin/seatunnel-cluster.sh
            - -r
            - worker
          env:
            - name: SEATUNNEL_HOME
              value: /opt/seatunnel
            - name: HAZELCAST_CLUSTER_NAME
              value: seatunnel-cluster
          ports:
            - containerPort: 8080
              name: rest-api
            - containerPort: 5801
              name: hazelcast
          resources:
            requests:
              cpu: "1"
              memory: 2Gi
            limits:
              cpu: "2"
              memory: 5Gi
          volumeMounts:
            - name: hazelcast-worker-config
              mountPath: /opt/seatunnel/config/hazelcast-worker.yaml
              subPath: hazelcast-worker.yaml
            - name: client-config
              mountPath: /opt/seatunnel/config/hazelcast-client.yaml
              subPath: hazelcast-client.yaml
            - name: engine-config
              mountPath: /opt/seatunnel/config/seatunnel.yaml
              subPath: seatunnel.yaml
      terminationGracePeriodSeconds: 120
      volumes:
        - name: hazelcast-worker-config
          configMap:
            name: seatunnel-worker-hazelcast-config
        - name: client-config
          configMap:
            name: seatunnel-client-config
        - name: engine-config
          configMap:
            name: seatunnel-engine-config

自定义插件不包含在基础 StatefulSet 中。需要额外插件时，请参考插件加载，通过自定义镜像、initContainer overlay、PersistentVolume 或对象存储单独管理插件。

健康检查和优雅停止

Master 和 Worker 都建议添加以下容器片段。startupProbe 可以避免启动期误杀；preStop 会先调用 SeaTunnel 停止脚本，再等待 SeaTunnelServer 进程退出，配合 terminationGracePeriodSeconds 可以降低滚动更新和节点驱逐对运行任务的影响。

startupProbe:
  tcpSocket:
    port: 5801
  periodSeconds: 10
  failureThreshold: 30
readinessProbe:
  tcpSocket:
    port: 5801
  initialDelaySeconds: 31
  periodSeconds: 30
  timeoutSeconds: 5
  failureThreshold: 3
livenessProbe:
  tcpSocket:
    port: 5801
  initialDelaySeconds: 30
  periodSeconds: 30
  timeoutSeconds: 5
  failureThreshold: 3
lifecycle:
  preStop:
    exec:
      command:
        - /bin/sh
        - -c
        - |
          /opt/seatunnel/bin/stop-seatunnel-cluster.sh
          while kill -0 $(ps -ef | grep SeaTunnelServer | grep -v grep | awk '{print $2}') 2>/dev/null; do
            sleep 1
          done

应用配置

说明

仅在使用 API 发现，或 StatefulSet 保留 serviceAccountName: seatunnel 时应用 seatunnel-rbac.yaml。

kubectl apply -f seatunnel-master-hazelcast-config.yaml
kubectl apply -f seatunnel-worker-hazelcast-config.yaml
kubectl apply -f seatunnel-client-config.yaml
kubectl apply -f seatunnel-engine-config.yaml
kubectl apply -f seatunnel-rbac.yaml
kubectl apply -f seatunnel-services.yaml
kubectl apply -f seatunnel-master.yaml
kubectl apply -f seatunnel-worker.yaml

检查 Pod 状态：

kubectl get pods -l app=seatunnel

访问 REST API：

kubectl port-forward svc/seatunnel-master 8080:8080
curl http://127.0.0.1:8080/system-monitoring-information

提交作业请参考 REST API V2。

扩缩容 Worker

Worker 扩容可以增加可用 slot：

kubectl scale statefulset seatunnel-worker --replicas=4

缩容前应确认被缩容的 Worker 上没有正在运行的关键任务，且剩余 Worker 的 slot 能承载当前作业。建议先停止或迁移作业，再缩容 Worker。

分离集群模式

推荐拓扑​

创建 ConfigMap​

Master Hazelcast 配置​

Worker Hazelcast 配置​

Hazelcast Client 配置​

SeaTunnel Engine 配置​

为 API 发现创建 RBAC​

创建 Service​

创建 Master StatefulSet​

创建 Worker StatefulSet​

健康检查和优雅停止​

应用配置​

扩缩容 Worker​

推荐拓扑