Skip to main content
Version: Next

Socket

Socket source connector

Support Those Engines​

Spark
Flink
SeaTunnel Zeta

Key features​

Description​

Used to read newline-delimited text data from a socket server. Each line received from the socket becomes one SeaTunnel row of type STRING. In streaming mode the source stays connected to the socket and reads lines as they arrive; in batch mode the reader performs a single read of whatever data is currently available on the socket, emits any complete newline-terminated lines from that read (plus any trailing partial line as a final row), and then finishes — it does not wait for the connection to close and there is no read-timeout setting.

The connector uses a single split (source parallelism is fixed at 1). host and port refer to the server endpoint that SeaTunnel connects to; configure a sink, transformer, or peer like nc -l on the other side.

Data Type Mapping​

Socket source reads each incoming line as a string record.

SeaTunnel Data type
STRING

Options​

NameTypeRequiredDefaultDescription
hostStringYes_socket server host
portIntegerYes_socket server port
common-optionsno-Source plugin common parameters, please refer to Source Common Options for details.
tip

Socket source is mainly used for local debugging and simple text streams. It does not checkpoint socket-server offsets, so it should not be used when replayable, exactly-once reads are required. Each line is treated as one record. Empty lines produce a row with an empty-string payload; they are not skipped.

How to Create a Socket Data Synchronization Jobs​

  • Configuring the SeaTunnel config file

The following example demonstrates how to create a data synchronization job that reads data from Socket and prints it on the local client:

# Set the basic configuration of the task to be performed
env {
  parallelism = 1
  job.mode = "BATCH"
}

# Create a source to connect to socket
source {
    Socket {
        host = "localhost"
        port = 9999
    }
}

# Console printing of the read socket data
sink {
  Console {
    parallelism = 1
  }
}
  • Start a port listening
nc -l 9999

  • Start a SeaTunnel task

  • Socket Source send test data

~ nc -l 9999
test
hello
flink
spark
  • Console Sink print data
[test]
[hello]
[flink]
[spark]

Streaming Mode​

In streaming mode the source keeps the socket open and reads new lines continuously. Pair it with a downstream sink that can buffer events or checkpoint them:

env {
  parallelism = 1
  job.mode = "STREAMING"
  checkpoint.interval = 10000
}

source {
  Socket {
    host = "localhost"
    port = 9999
  }
}

sink {
  Console {
    parallelism = 1
  }
}

Changelog​

Change Log
ChangeCommitVersion
[improve] socket options (#9517)https://github.com/apache/seatunnel/commit/af83a302cf2.3.12
[Feature][Restapi] Allow metrics information to be associated to logical plan nodes (#7786)https://github.com/apache/seatunnel/commit/6b7c53d03c2.3.9
[Improve] Remove use SeaTunnelSink::getConsumedType method and mark it as deprecated (#5755)https://github.com/apache/seatunnel/commit/8de74081002.3.4
[Improve][build] Give the maven module a human readable name (#4114)https://github.com/apache/seatunnel/commit/d7cd6010512.3.1
[Improve][Project] Code format with spotless plugin. (#4101)https://github.com/apache/seatunnel/commit/a2ab1665612.3.1
[Feature][Connector] add get source method to all source connector (#3846)https://github.com/apache/seatunnel/commit/417178fb842.3.1
[Hotfix][OptionRule] Fix option rule about all connectors (#3592)https://github.com/apache/seatunnel/commit/226dc6a1192.3.0
[Improve][Connector-V2][Socket] Unified exception for socket source & sink connector (#3511)https://github.com/apache/seatunnel/commit/581292f2102.3.0
[feature][connector][socket] Add Socket Connector Option Rules (#3317)https://github.com/apache/seatunnel/commit/b85317bcbe2.3.0
[Improve][all] change Log to @Slf4j (#3001)https://github.com/apache/seatunnel/commit/6016100f122.3.0-beta
[DEV][Api] Replace SeaTunnelContext with JobContext and remove singleton pattern (#2706)https://github.com/apache/seatunnel/commit/cbf82f755c2.2.0-beta
[#2606]Dependency management split (#2630)https://github.com/apache/seatunnel/commit/fc047be69b2.2.0-beta
[Feature][Connector-V2] Socket Connector Sink (#2549)https://github.com/apache/seatunnel/commit/94f4600a4e2.2.0-beta
[api-draft][Optimize] Optimize module name (#2062)https://github.com/apache/seatunnel/commit/f79e3112b12.2.0-beta