Skip to main content
Version: 2.3.6

rest-api


RESTful API

SeaTunnel has a monitoring API that can be used to query status and statistics of running jobs, as well as recent completed jobs. The monitoring API is a RESTful API that accepts HTTP requests and responds with JSON data.

Overview​

The monitoring API is backed by a web server that runs as part of the node, each node member can provide RESTful api capability. By default, this server listens at port 5801, which can be configured in hazelcast.yaml like :

network:
rest-api:
enabled: true
endpoint-groups:
CLUSTER_WRITE:
enabled: true
DATA:
enabled: true
join:
tcp-ip:
enabled: true
member-list:
- localhost
port:
auto-increment: true
port-count: 100
port: 5801

API reference​

Returns an overview over the Zeta engine cluster.​

GET /hazelcast/rest/maps/overview?tag1=value1&tag2=value2 (Returns an overview over the Zeta engine cluster.)

Parameters​

nametypedata typedescription
tag_nameoptionalstringthe tags filter, you can add tag filter to get those matched worker count, and slot on those workers

Responses​

{
"projectVersion":"2.3.5-SNAPSHOT",
"gitCommitAbbrev":"DeadD0d0",
"totalSlot":"0",
"unassignedSlot":"0",
"works":"1",
"runningJobs":"0",
"finishedJobs":"0",
"failedJobs":"0",
"cancelledJobs":"0"
}

Notes:

  • If you use dynamic-slot, the totalSlot and unassignedSlot always be 0. when you set it to fix slot number, it will return the correct total and unassigned slot number
  • If the url has tag filter, the works, totalSlot and unassignedSlot will return the result on the matched worker. but the job related metric will always return the cluster level information.

Returns An Overview And State Of All Jobs​

GET /hazelcast/rest/maps/running-jobs (Returns an overview over all jobs and their current state.)

Parameters​

Responses​

[
{
"jobId": "",
"jobName": "",
"jobStatus": "",
"envOptions": {
},
"createTime": "",
"jobDag": {
"vertices": [
],
"edges": [
]
},
"pluginJarsUrls": [
],
"isStartWithSavePoint": false,
"metrics": {
"sourceReceivedCount": "",
"sinkWriteCount": ""
}
}
]

Return Details Of A Job​

GET /hazelcast/rest/maps/job-info/:jobId (Return details of a job. )

Parameters​

nametypedata typedescription
jobIdrequiredlongjob id

Responses​

{
"jobId": "",
"jobName": "",
"jobStatus": "",
"createTime": "",
"jobDag": {
"vertices": [
],
"edges": [
]
},
"metrics": {
"sourceReceivedCount": "",
"sinkWriteCount": ""
},
"finishedTime": "",
"errorMsg": null,
"envOptions": {
},
"pluginJarsUrls": [
],
"isStartWithSavePoint": false
}

jobId, jobName, jobStatus, createTime, jobDag, metrics always be returned. envOptions, pluginJarsUrls, isStartWithSavePoint will return when job is running. finishedTime, errorMsg will return when job is finished.

When we can't get the job info, the response will be:

{
"jobId" : ""
}

Return Details Of A Job​

This API has been deprecated, please use /hazelcast/rest/maps/job-info/:jobId instead

GET /hazelcast/rest/maps/running-job/:jobId (Return details of a job. )

Parameters​

nametypedata typedescription
jobIdrequiredlongjob id

Responses​

{
"jobId": "",
"jobName": "",
"jobStatus": "",
"createTime": "",
"jobDag": {
"vertices": [
],
"edges": [
]
},
"metrics": {
"sourceReceivedCount": "",
"sinkWriteCount": ""
},
"finishedTime": "",
"errorMsg": null,
"envOptions": {
},
"pluginJarsUrls": [
],
"isStartWithSavePoint": false
}

jobId, jobName, jobStatus, createTime, jobDag, metrics always be returned. envOptions, pluginJarsUrls, isStartWithSavePoint will return when job is running. finishedTime, errorMsg will return when job is finished.

When we can't get the job info, the response will be:

{
"jobId" : ""
}

Return All Finished Jobs Info​

GET /hazelcast/rest/maps/finished-jobs/:state (Return all finished Jobs Info.)

Parameters​

nametypedata typedescription
stateoptionalstringfinished job status. FINISHED,CANCELED,FAILED,UNKNOWABLE

Responses​

[
{
"jobId": "",
"jobName": "",
"jobStatus": "",
"errorMsg": null,
"createTime": "",
"finishTime": "",
"jobDag": "",
"metrics": ""
}
]

Returns System Monitoring Information​

GET /hazelcast/rest/maps/system-monitoring-information (Returns system monitoring information.)

Parameters​

Responses​

[
{
"processors":"8",
"physical.memory.total":"16.0G",
"physical.memory.free":"16.3M",
"swap.space.total":"0",
"swap.space.free":"0",
"heap.memory.used":"135.7M",
"heap.memory.free":"440.8M",
"heap.memory.total":"576.5M",
"heap.memory.max":"3.6G",
"heap.memory.used/total":"23.54%",
"heap.memory.used/max":"3.73%",
"minor.gc.count":"6",
"minor.gc.time":"110ms",
"major.gc.count":"2",
"major.gc.time":"73ms",
"load.process":"24.78%",
"load.system":"60.00%",
"load.systemAverage":"2.07",
"thread.count":"117",
"thread.peakCount":"118",
"cluster.timeDiff":"0",
"event.q.size":"0",
"executor.q.async.size":"0",
"executor.q.client.size":"0",
"executor.q.client.query.size":"0",
"executor.q.client.blocking.size":"0",
"executor.q.query.size":"0",
"executor.q.scheduled.size":"0",
"executor.q.io.size":"0",
"executor.q.system.size":"0",
"executor.q.operations.size":"0",
"executor.q.priorityOperation.size":"0",
"operations.completed.count":"10",
"executor.q.mapLoad.size":"0",
"executor.q.mapLoadAllKeys.size":"0",
"executor.q.cluster.size":"0",
"executor.q.response.size":"0",
"operations.running.count":"0",
"operations.pending.invocations.percentage":"0.00%",
"operations.pending.invocations.count":"0",
"proxy.count":"8",
"clientEndpoint.count":"0",
"connection.active.count":"2",
"client.connection.count":"0",
"connection.count":"0"
}
]

Submit A Job​

POST /hazelcast/rest/maps/submit-job (Returns jobId and jobName if job submitted successfully.)

Parameters​

nametypedata typedescription
jobIdoptionalstringjob id
jobNameoptionalstringjob name
isStartWithSavePointoptionalstringif job is started with save point

Body​

{
"env": {
"job.mode": "batch"
},
"source": [
{
"plugin_name": "FakeSource",
"result_table_name": "fake",
"row.num": 100,
"schema": {
"fields": {
"name": "string",
"age": "int",
"card": "int"
}
}
}
],
"transform": [
],
"sink": [
{
"plugin_name": "Console",
"source_table_name": ["fake"]
}
]
}

Responses​

{
"jobId": 733584788375666689,
"jobName": "rest_api_test"
}

Stop A Job​

POST /hazelcast/rest/maps/stop-job (Returns jobId if job stoped successfully.)

Body​

{
"jobId": 733584788375666689,
"isStopWithSavePoint": false # if job is stopped with save point
}

Responses​

{
"jobId": 733584788375666689
}

Encrypt Config​

POST /hazelcast/rest/maps/encrypt-config (Returns the encrypted config if config is encrypted successfully.)
For more information about customize encryption, please refer to the documentation [config-encryption-decryption](/docs/2.3.6/connector-v2/Config-Encryption-Decryption).

Body​

{
"env": {
"parallelism": 1,
"shade.identifier":"base64"
},
"source": [
{
"plugin_name": "MySQL-CDC",
"schema" : {
"fields": {
"name": "string",
"age": "int"
}
},
"result_table_name": "fake",
"parallelism": 1,
"hostname": "127.0.0.1",
"username": "seatunnel",
"password": "seatunnel_password",
"table-name": "inventory_vwyw0n"
}
],
"transform": [
],
"sink": [
{
"plugin_name": "Clickhouse",
"host": "localhost:8123",
"database": "default",
"table": "fake_all",
"username": "seatunnel",
"password": "seatunnel_password"
}
]
}

Responses​

{
"env": {
"parallelism": 1,
"shade.identifier": "base64"
},
"source": [
{
"plugin_name": "MySQL-CDC",
"schema": {
"fields": {
"name": "string",
"age": "int"
}
},
"result_table_name": "fake",
"parallelism": 1,
"hostname": "127.0.0.1",
"username": "c2VhdHVubmVs",
"password": "c2VhdHVubmVsX3Bhc3N3b3Jk",
"table-name": "inventory_vwyw0n"
}
],
"transform": [],
"sink": [
{
"plugin_name": "Clickhouse",
"host": "localhost:8123",
"database": "default",
"table": "fake_all",
"username": "c2VhdHVubmVs",
"password": "c2VhdHVubmVsX3Bhc3N3b3Jk"
}
]
}