Command usage

Command Entrypoint

Spark

bin/start-seatunnel-spark.sh

Flink

bin/start-seatunnel-flink.sh  

Options

Spark

bin/start-seatunnel-spark.sh \
    -c config-path \
    -m master \
    -e deploy-mode \
    -i city=beijing

• Use -m or --master to specify the cluster manager

• Use -e or --deploy-mode to specify the deployment mode

Flink

bin/start-seatunnel-flink.sh \
    -c config-path \
    -i key=value \
    -r run-application \
    [other params]

• Use -r or --run-mode to specify the flink job run mode, you can use run-application or run (default value)

• Use -c or --config to specify the path of the configuration file

• Use -i or --variable to specify the variables in the configuration file, you can configure multiple

Example

Spark

# Yarn client mode
./bin/start-seatunnel-spark.sh \
    --master yarn \
    --deploy-mode client \
    --config ./config/application.conf

# Yarn cluster mode
./bin/start-seatunnel-spark.sh \
    --master yarn \
    --deploy-mode cluster \
    --config ./config/application.conf

Flink

env {
    execution.parallelism = 1
}

source {
    FakeSourceStream {
        result_table_name = "fake"
        field_name = "name,age"
    }
}

transform {
    sql {
        sql = "select name,age from fake where name='"${my_name}"'"
    }
}

sink {
    ConsoleSink {}
}

Run

bin/start-seatunnel-flink.sh \
    -c config-path \
    -i my_name=kid-xiong

This designation will replace "${my_name}" in the configuration file with kid-xiong

All the configurations in the env section will be applied to Flink dynamic parameters with the format of -D, such as -Dexecution.parallelism=1 .

For the rest of the parameters, refer to the original flink parameters. Check the flink parameter method: bin/flink run -h . The parameters can be added as needed. For example, -m yarn-cluster is specified as on yarn mode.

bin/flink run -h

For example:

• -p 2 specifies that the job parallelism is 2

bin/start-seatunnel-flink.sh \
    -p 2 \
    -c config-path

• Configurable parameters of flink yarn-cluster

For example: -m yarn-cluster -ynm seatunnel specifies that the job is running on yarn, and the name of yarn WebUI is seatunnel

bin/start-seatunnel-flink.sh \
    -m yarn-cluster \
    -ynm seatunnel \
    -c config-path