跳到主要内容
版本:2.1.0

Quick start

Let's take an application that receives data through a socket , divides the data into multiple fields, and outputs the processing results as an example to quickly show how to use seatunnel .

If you are familiar with Flink or have prepared the Flink operating environment, you can ignore this step. Flink does not require any special configuration.

Please download Flink first, please choose Flink version >= 1.12.0 and version < version < 1.14.x . The download is complete to install Flink

Step 2: Download seatunnel

Enter the seatunnel installation package download page and download the latest version of seatunnel-<version>.zip

Or download the specified version directly (take 2.0.4 as an example):

wget https://github.com/apache/incubator-seatunnel/releases/download/v2.0.4/waterdrop-dist-2.0.4-2.11.8-release.zip -O seatunnel-2.0.4.zip

After downloading, unzip:

unzip seatunnel-<version>.zip
ln -s seatunnel-<version> seatunnel

Step 3: Configure seatunnel

  • Edit config/seatunnel-env.sh , specify the necessary environment configuration such as FLINK_HOME (the directory after Flink downloaded and decompressed in Step 1)

  • Edit config/application.conf , it determines the way and logic of data input, processing, and output after seatunnel is started.

env {
# You can set flink configuration here
execution.parallelism = 1
#execution.checkpoint.interval = 10000
#execution.checkpoint.data-uri = "hdfs://localhost:9000/checkpoint"
}

source {
SocketStream{
result_table_name = "fake"
field_name = "info"
}
}

transform {
Split{
separator = "#"
fields = ["name","age"]
}
sql {
sql = "select * from (select info,split(info) as info_row from fake) t1"
}
}

sink {
ConsoleSink {}
}

Step 4: Start the netcat server to send data

nc -l -p 9999

Step 5: start seatunnel

cd seatunnel
./bin/start-seatunnel-flink.sh \
--config ./config/application.conf

Step 6: Input at the nc terminal

xg#1995

It is printed in the TaskManager Stdout log of flink WebUI:

xg#1995,xg,1995

Summary

If you want to know more seatunnel configuration examples, please refer to:

The above configuration is the default [streaming configuration template] , which can be run directly, the command is as follows:

cd seatunnel
./bin/start-seatunnel-flink.sh \
--config ./config/flink.streaming.conf.template

The above configuration is the default [offline batch configuration template] , which can be run directly, the command is as follows:

cd seatunnel
./bin/start-seatunnel-flink.sh \
--config ./config/flink.batch.conf.template