Command usage
Command Entrypointâ
- Spark
- Flink
- Spark V2
- Flink V2
bin/start-seatunnel-spark.sh
bin/start-seatunnel-flink.sh
```bash
bin/start-seatunnel-spark-connector-v2.sh
```
```bash
bin/start-seatunnel-flink-connector-v2.sh
```
Optionsâ
- Spark
- Flink
- Spark V2
- Flink V2
bin/start-seatunnel-spark.sh \
-c config-path \
-m master \
-e deploy-mode \
-i city=beijing
Use
-m
or--master
to specify the cluster managerUse
-e
or--deploy-mode
to specify the deployment mode
```bash
bin/start-seatunnel-spark-connector-v2.sh \
-c config-path \
-m master \
-e deploy-mode \
-i city=beijing \
-n spark-test
```
- Use `-m` or `--master` to specify the cluster manager
- Use `-e` or `--deploy-mode` to specify the deployment mode
- Use `-n` or `--name` to specify the app name
bin/start-seatunnel-flink.sh \
-c config-path \
-i key=value \
-r run-application \
[other params]
- Use
-r
or--run-mode
to specify the flink job run mode, you can userun-application
orrun
(default value)
```bash
bin/start-seatunnel-flink-connector-v2.sh \
-c config-path \
-i key=value \
-r run-application \
-n flink-test \
[other params]
```
- Use `-r` or `--run-mode` to specify the flink job run mode, you can use `run-application` or `run` (default value)
- Use `-n` or `--name` to specify the app name
Use
-c
or--config
to specify the path of the configuration fileUse
-i
or--variable
to specify the variables in the configuration file, you can configure multiple
Exampleâ
- Spark
- Flink
# Yarn client mode
./bin/start-seatunnel-spark.sh \
--master yarn \
--deploy-mode client \
--config ./config/application.conf
# Yarn cluster mode
./bin/start-seatunnel-spark.sh \
--master yarn \
--deploy-mode cluster \
--config ./config/application.conf
env {
execution.parallelism = 1
}
source {
FakeSourceStream {
result_table_name = "fake"
field_name = "name,age"
}
}
transform {
sql {
sql = "select name,age from fake where name='"${my_name}"'"
}
}
sink {
ConsoleSink {}
}
Run
bin/start-seatunnel-flink.sh \
-c config-path \
-i my_name=kid-xiong
This designation will replace "${my_name}"
in the configuration file with kid-xiong
All the configurations in the
env
section will be applied to Flink dynamic parameters with the format of-D
, such as-Dexecution.parallelism=1
.
For the rest of the parameters, refer to the original flink parameters. Check the flink parameter method:
bin/flink run -h
. The parameters can be added as needed. For example,-m yarn-cluster
is specified ason yarn
mode.
bin/flink run -h
For example:
-p 2
specifies that the job parallelism is2
bin/start-seatunnel-flink.sh \
-p 2 \
-c config-path
- Configurable parameters of
flink yarn-cluster
For example: -m yarn-cluster -ynm seatunnel
specifies that the job is running on yarn
, and the name of yarn WebUI
is seatunnel
bin/start-seatunnel-flink.sh \
-m yarn-cluster \
-ynm seatunnel \
-c config-path