Submitting Jobs to a Remote Zeta Cluster

This guide explains how to submit SeaTunnel jobs to a remote Zeta cluster rather than running locally. It covers Docker single-node setups, multi-node clusters, Kubernetes (K8s) via kubectl port-forward or direct NodePort/LoadBalancer access, and EKS/Helm deployments.

---

1. Prerequisites

Requirement	Details
SeaTunnel client installed	The SeaTunnel home directory is available locally with bin/seatunnel.sh
Cluster reachable	REST API port (default 8080) accessible from the submitting machine
Job configuration file	A HOCON .conf or JSON payload ready

---

2. Submitting from the Local Machine to a Remote Cluster

2.1 --master flag (Zeta)

All SeaTunnel Zeta client commands accept a --master argument that tells the client which cluster to connect to:

bin/seatunnel.sh \
  --config job.conf \
  --master seatunnel://192.168.1.100:5801

The default Zeta cluster-internal port is 5801. This is different from the REST API port (8080). The --master flag is used when running the job via the SeaTunnel client binary, which communicates with the cluster over the Hazelcast member protocol.

2.2 Using the REST API (recommended for automation)

For CI/CD pipelines and scripts, prefer REST API submission over the binary client:

curl -X POST http://192.168.1.100:8080/submit-job \
  -H "Content-Type: application/json" \
  -d @job.json

See REST API v2 for the complete request and response reference.

---

3. Docker: Single-Node Submission

3.1 Start a Zeta container with REST enabled

docker run -d --name seatunnel \
  -p 8080:8080 \
  apache/seatunnel:<version> \
  ./bin/seatunnel-cluster.sh

The published image defines no ENTRYPOINT or CMD, so the cluster startup script must be passed as the container command (the same pattern as the Docker quickstart). Without -r, seatunnel-cluster.sh starts the node with the default role master_and_worker, so this single container is a working one-node cluster and no extra environment variables are needed.

3.2 Submit a job from outside the container

curl -X POST http://localhost:8080/submit-job \
  -H "Content-Type: application/json" \
  -d '{
    "env": { "job.name": "test", "job.mode": "BATCH" },
    "source": [{ "plugin_name": "FakeSource", "plugin_output": "fake",
                 "row.num": 10,
                 "schema": { "fields": { "id": "int", "name": "string" } } }],
    "transform": [],
    "sink": [{ "plugin_name": "Console", "plugin_input": ["fake"] }]
  }'

3.3 Run a quick local smoke test inside the container

docker run -d --name seatunnel \
  -p 8080:8080 \
  -v /path/to/your/jobs:/jobs \
  apache/seatunnel:<version> \
  ./bin/seatunnel-cluster.sh

# Execute inside the container
docker exec seatunnel \
  /opt/seatunnel/bin/seatunnel.sh --config /jobs/my-job.conf --master local

This command is useful only for a container-local smoke test. It does not submit the job to a remote Zeta cluster because --master local starts the job inside that container process.

---

4. Docker: Multi-Node Cluster

4.1 Docker Compose example

The cluster membership is passed to each container with the ST_DOCKER_MEMBER_LIST environment variable (this is the only cluster-discovery environment variable the engine reads). Because the image defines no ENTRYPOINT, each service also starts seatunnel-cluster.sh explicitly with its role, the same as the compose file in the Docker quickstart:

version: "3.8"
services:
  master:
    image: apache/seatunnel:<version>
    container_name: seatunnel-master
    ports:
      - "8080:8080"
      - "5801:5801"
    environment:
      ST_DOCKER_MEMBER_LIST: seatunnel-master:5801,seatunnel-worker:5801
    entrypoint: >
      /bin/sh -c "
      /opt/seatunnel/bin/seatunnel-cluster.sh -r master
      "
    networks:
      - st-net

  worker:
    image: apache/seatunnel:<version>
    container_name: seatunnel-worker
    environment:
      ST_DOCKER_MEMBER_LIST: seatunnel-master:5801,seatunnel-worker:5801
    entrypoint: >
      /bin/sh -c "
      /opt/seatunnel/bin/seatunnel-cluster.sh -r worker
      "
    networks:
      - st-net
    depends_on:
      - master

networks:
  st-net:
    driver: bridge

Start the cluster:

docker-compose up -d

Submit a job to the master:

curl -X POST http://localhost:8080/submit-job \
  -H "Content-Type: application/json" \
  -d @job.json

4.2 Network requirements (Docker)

Port	Protocol	Purpose
5801	TCP	Hazelcast cluster-internal member communication
8080	TCP	REST API (job submission / monitoring)

Ensure the Docker network allows all cluster members to reach each other on port 5801. The REST API only needs to be exposed on the master node(s).

---

5. Kubernetes: Job Submission

5.1 Using kubectl port-forward (development / ad-hoc)

After deploying SeaTunnel to Kubernetes (see Kubernetes Deployment), forward the master pod's REST port to your local machine:

# Find the master pod (the Helm chart deploys master as a Deployment)
MASTER_POD=$(kubectl get pods -n seatunnel \
  -l app.kubernetes.io/name=seatunnel-master \
  -o jsonpath='{.items[0].metadata.name}')

# Forward REST port
kubectl port-forward -n seatunnel \
  pod/${MASTER_POD} 8080:8080

In a second terminal, submit a job:

curl -X POST http://localhost:8080/submit-job \
  -H "Content-Type: application/json" \
  -d @job.json

5.2 Using the master Service (default chart deployment)

The Helm chart only creates a headless Service for the master, so there is no separate REST Service or NodePort out of the box. Forward the master Service's REST port to your local machine:

kubectl port-forward -n seatunnel svc/seatunnel-master 8080:8080

In a second terminal, submit a job:

curl -X POST http://localhost:8080/submit-job \
  -H "Content-Type: application/json" \
  -d @job.json

5.3 Using an Ingress (staging / production)

If you enable the chart's Ingress (ingress.enabled: true, see the Helm Chart Reference), submit jobs through the Ingress host instead:

curl -X POST http://<your-ingress-host>/submit-job \
  -H "Content-Type: application/json" \
  -d @job.json

If you expose the REST API with your own NodePort or LoadBalancer Service, submit to that Service's address the same way.

---

6. Kubernetes: Job Configuration via ConfigMap

When running jobs inside the cluster, mount job configuration files as ConfigMaps rather than baking them into the image:

apiVersion: v1
kind: ConfigMap
metadata:
  name: seatunnel-job-config
  namespace: seatunnel
data:
  cdc-job.conf: |
    env {
      job.name = "cdc-prod"
      job.mode = STREAMING
      checkpoint.interval = 30000
    }
    source {
      MySQL-CDC {
        ...
      }
    }
    sink {
      ...
    }

Mount in the Pod spec:

volumeMounts:
  - name: job-config
    mountPath: /opt/seatunnel/jobs
volumes:
  - name: job-config
    configMap:
      name: seatunnel-job-config

Then submit via kubectl exec (${MASTER_POD} is only defined in section 5.1, so repeat the lookup here if you jumped straight to this section):

MASTER_POD=$(kubectl get pods -n seatunnel \
  -l app.kubernetes.io/name=seatunnel-master \
  -o jsonpath='{.items[0].metadata.name}')

kubectl exec -n seatunnel ${MASTER_POD} -- \
  /opt/seatunnel/bin/seatunnel.sh \
  --config /opt/seatunnel/jobs/cdc-job.conf

---

7. Amazon EKS / Helm Deployment

7.1 Helm install

The SeaTunnel Helm chart is published as an OCI artifact on Docker Hub (there is no https://apache.github.io/seatunnel-helm-charts chart repository):

# Keep the chart version in sync with kubernetes/helm.md
export VERSION=3.1.0
helm pull oci://registry-1.docker.io/apache/seatunnel-helm --version ${VERSION}
tar -xvf seatunnel-helm-${VERSION}.tgz
cd seatunnel-helm

helm install seatunnel . \
  --namespace seatunnel \
  --create-namespace

Scale masters and workers with the master.replicas and worker.replicas values:

helm upgrade seatunnel . --namespace seatunnel \
  --set master.replicas=2 \
  --set worker.replicas=4

7.2 EKS-specific: reaching the REST API

The default chart creates only headless Services, so there is no automatically provisioned load balancer. Either enable the chart Ingress and read its address:

kubectl get ingress -n seatunnel

and use that host as the API endpoint:

curl -X POST http://<your-ingress-host>/submit-job \
  -H "Content-Type: application/json" \
  -d @job.json

or use kubectl port-forward svc/seatunnel-master 8080:8080 as shown in section 5.2.

7.3 Customizing resources via Helm values

# values-prod.yaml
master:
  replicas: "2"
  resources:
    requests:
      memory: "4Gi"
      cpu: "2"
    limits:
      memory: "8Gi"
      cpu: "4"

worker:
  replicas: "8"
  resources:
    requests:
      memory: "8Gi"
      cpu: "4"
    limits:
      memory: "16Gi"
      cpu: "8"

The chart has no seatunnel.config values tree. To customize engine files such as seatunnel.yaml or hazelcast*.yaml, let the chart generate the ConfigMap from the chart's conf/ files (default), or ship your own ConfigMap and set configMap.create: false plus configMap.existingConfigMapName — see the chart's values.yaml and the Helm Chart Reference.

Apply with:

helm upgrade seatunnel . \
  --namespace seatunnel \
  -f values-prod.yaml

---

8. Network and Port Requirements

Port	Protocol	Required By	Notes
5801	TCP	Hazelcast cluster	Member-to-member communication; do NOT expose publicly
8080	TCP	REST API	Expose only via authenticated gateway in production
8090	TCP	Web UI	Optional; only needed for the dashboard

In Kubernetes, set up a NetworkPolicy to restrict port 5801 to the SeaTunnel namespace only:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: seatunnel-internal
  namespace: seatunnel
spec:
  podSelector:
    matchLabels:
      app: seatunnel
  ingress:
    - from:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: seatunnel
      ports:
        - port: 5801

---

9. Troubleshooting

Symptom	Likely Cause	Fix
Connection refused on port 8080	REST API not enabled or wrong port	Set enable-http: true; check port config
Workers not joining the cluster	Firewall blocks port 5801	Open TCP 5801 between all cluster nodes
kubectl port-forward disconnects	Idle timeout or pod restart	Restart port-forward; consider NodePort instead
Job submitted but status always WAITING	No available worker slots	Scale up worker replicas or check resource quotas
EKS Ingress hostname not resolving	DNS propagation delay	Wait 1–2 minutes; verify with nslookup
Helm install fails with pending-install	Previous failed release	Run helm rollback or helm uninstall then retry

---

See Also

• Kubernetes Deployment Guide
• Helm Chart Reference
• Docker Quick Start
• REST API v2
• Zeta Engine Configuration