跳到主要内容
版本:2.1.1

Hive

Description

Get data from hive

提示

Engine Supported and plugin name

  • Spark: Hive
  • Flink

Options

nametyperequireddefault value
pre_sqlstringyes-
common-optionsstringyes-

pre_sql [string]

For preprocessed sql , if preprocessing is not required, you can use select * from hive_db.hive_table .

common options [string]

Source plugin common parameters, please refer to Source Plugin for details

Note: The following configuration must be done to use hive source:

# In the spark section in the seatunnel configuration file:

env {
...
spark.sql.catalogImplementation = "hive"
...
}

Example

env {
...
spark.sql.catalogImplementation = "hive"
...
}

source {
hive {
pre_sql = "select * from mydb.mytb"
result_table_name = "myTable"
}
}

...

Notes

It must be ensured that the metastore of hive is in service. Start the command hive --service metastore service default port 9083 cluster , client , local mode, hive-site.xml must be placed in the $HADOOP_CONF directory of the task submission node (or placed under $SPARK_HOME/conf ), IDE local Debug put it in the resources directory