Skip to main content
Version: 2.1.3

File

Description

Read data from local or hdfs file.

tip

Engine Supported and plugin name

  • Spark: File
  • Flink: File

Options

nametyperequireddefault value
formatstringnojson
pathstringyes-
common-optionsstringyes-
format [string]

Format for reading files, currently supports text, parquet, json, orc, csv.

path [string]
  • If read data from hdfs , the file path should start with hdfs://
  • If read data from local , the file path should start with file://

common options [string]

Source plugin common parameters, please refer to Source Plugin for details

Examples

file {
path = "hdfs:///var/logs"
result_table_name = "access_log"
}
file {
path = "file:///var/logs"
result_table_name = "access_log"
}