Skip to main content
Version: 2.3.0-beta

Hive

Hive source connector

Description​

Get data from hive

tip

Engine Supported and plugin name

  • Spark: Hive
  • Flink

Options​

nametyperequireddefault value
pre_sqlstringyes-
common-optionsstringyes-

pre_sql [string]​

For preprocessed sql , if preprocessing is not required, you can use select * from hive_db.hive_table .

common options [string]​

Source plugin common parameters, please refer to Source Plugin for details

Note: The following configuration must be done to use hive source:

# In the spark section in the seatunnel configuration file:

env {
...
spark.sql.catalogImplementation = "hive"
...
}

Example​

env {
...
spark.sql.catalogImplementation = "hive"
...
}

source {
hive {
pre_sql = "select * from mydb.mytb"
result_table_name = "myTable"
}
}

...

Notes​

It must be ensured that the metastore of hive is in service. Start the command hive --service metastore service default port 9083 cluster , client , local mode, hive-site.xml must be placed in the $HADOOP_CONF directory of the task submission node (or placed under $SPARK_HOME/conf ), IDE local Debug put it in the resources directory