Oracle CDC
Oracle CDC 源连接器
支持这些引擎
SeaTunnel Zeta
Flink
关键特性
描述
Oracle CDC 连接器允许从 Oracle 数据库读取快照数据和增量数据。本文档描述了如何设置 Oracle CDC 连接器以针对 Oracle 数据库运行 SQL 查询。
注意
Debezium Oracle 连接器不依赖于连续挖掘选项。连接器负责检测日志切换并自动调整要挖掘的日志,这是连续挖掘选项为您自动执行的操作。
因此,您不能在 debezium 中设置名为 log.mining.continuous.mine 的此属性。
支持的数据源信息
| 数据源 | 支持的版本 | 驱动程序 | URL | Maven |
|---|---|---|---|---|
| Oracle | 不同的依赖版本有不同的驱动程序类。 | oracle.jdbc.OracleDriver | jdbc:oracle:thin:@datasource01:1523:xe | https://mvnrepository.com/artifact/com.oracle.database.jdbc/ojdbc8 |
数据库依赖
安装 JDBC 驱动程序
对于 Spark/Flink 引擎
- 您需要确保 JDBC 驱动程序 jar 包 已放置在目录
${SEATUNNEL_HOME}/plugins/中。- 为了支持 i18n 字符集,将
orai18n.jar复制到$SEATUNNEL_HOME/plugins/目录。
对于 SeaTunnel Zeta 引擎
- 您需要确保 JDBC 驱动程序 jar 包 已放置在目录
${SEATUNNEL_HOME}/lib/中。- 为了支持 i18n 字符集,将
orai18n.jar复制到$SEATUNNEL_HOME/lib/目录。
启用 Oracle Logminer
要在 Seatunnel 中启用 Oracle CDC(变更数据捕获)使用 Logminer(这是 Oracle 提供的内置工具),请按照以下步骤操作:
在没有 CDB(容器数据库)模式的情况下启用 Logminer。
- 操作系统创建一个空文件目录来存储 Oracle 归档日志和用户表空间。
mkdir -p /opt/oracle/oradata/recovery_area
mkdir -p /opt/oracle/oradata/ORCLCDB
chown -R oracle /opt/oracle/***
- 以管理员身份登录并启用 Oracle 归档日志。
sqlplus /nolog;
connect sys as sysdba;
alter system set db_recovery_file_dest_size = 10G;
alter system set db_recovery_file_dest = '/opt/oracle/oradata/recovery_area' scope=spfile;
shutdown immediate;
startup mount;
alter database archivelog;
alter database open;
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;
archive log list;
- 以管理员身份登录并创建一个名为 logminer_user 的帐户,密码为 "oracle",并授予其读取表和日志的权限。
变更日志
Change Log
| Change | Commit | Version |
|---|---|---|
| [Feature][Core] Add plugin directory support for each connector (#9650) | https://github.com/apache/seatunnel/commit/4beb2b9336 | 2.3.12 |
| [improve] jdbc options (#9541) | https://github.com/apache/seatunnel/commit/d041e5fb32 | 2.3.12 |
| [Feature][Connectors-v2] Optimize the size of CDC JAR Files (#9546) | https://github.com/apache/seatunnel/commit/1dd19c6823 | 2.3.12 |
| [Fix][Connector-V2] Oracle cdc not update transaction commit when LOB enabled (#9412) | https://github.com/apache/seatunnel/commit/2a25bae6f6 | 2.3.12 |
| [Improve][Oracle-CDC] Remove duplicate load table names (#9357) | https://github.com/apache/seatunnel/commit/90e88cafc5 | 2.3.12 |
| [Feature][Connector-JDBC] Supprot read Oracle BLOB data as string instead of bytes (#9305) | https://github.com/apache/seatunnel/commit/454a88f81a | 2.3.11 |
| [Improve][CDC] Filter ddl for snapshot phase (#8911) | https://github.com/apache/seatunnel/commit/641cc72f2f | 2.3.10 |
| [Improve][Oracle-CDC] Support ReadOnlyLogWriterFlushStrategy (#8912) | https://github.com/apache/seatunnel/commit/6aebdc0384 | 2.3.10 |
| [Improve][CDC] Extract duplicate code (#8906) | https://github.com/apache/seatunnel/commit/b922bb90e6 | 2.3.10 |
| [Improve] restruct connector common options (#8634) | https://github.com/apache/seatunnel/commit/f3499a6eeb | 2.3.10 |
| [hotfix][connector-cdc-oracle ] support read partition table (#8265) | https://github.com/apache/seatunnel/commit/91b86b2faf | 2.3.9 |
| [Improve][E2E] improve oracle e2e (#8292) | https://github.com/apache/seatunnel/commit/9f761b9d32 | 2.3.9 |
| [Feature][CDC] Add 'schema-changes.enabled' options (#8285) | https://github.com/apache/seatunnel/commit/8e29ecf54f | 2.3.9 |
| Revert "[Feature][Redis] Flush data when the time reaches checkpoint interval" and "[Feature][CDC] Add 'schema-changes.enabled' options" (#8278) | https://github.com/apache/seatunnel/commit/fcb2938286 | 2.3.9 |
| [Feature][CDC] Add 'schema-changes.enabled' options (#8252) | https://github.com/apache/seatunnel/commit/d783f9447c | 2.3.9 |
| [Improve][dist]add shade check rule (#8136) | https://github.com/apache/seatunnel/commit/51ef800016 | 2.3.9 |
| [Feature][Connector-V2]Jdbc chunk split add snapshotSplitColumn config #7794 (#7840) | https://github.com/apache/seatunnel/commit/b6c6dc0438 | 2.3.9 |
| [Feature][Core] Support cdc task ddl restore for zeta (#7463) | https://github.com/apache/seatunnel/commit/8e322281ed | 2.3.9 |
| [Feature][Connector-v2] Support schema evolution for Oracle connector (#7908) | https://github.com/apache/seatunnel/commit/79406bcc2f | 2.3.9 |
| [Hotfix][CDC] Fix package name spelling mistake (#7415) | https://github.com/apache/seatunnel/commit/469112fa64 | 2.3.8 |
| [Improve][Connector-v2] Optimize the count table rows for jdbc-oracle and oracle-cdc (#7248) | https://github.com/apache/seatunnel/commit/0d08b20061 | 2.3.6 |
| [Improve][CDC] Bump the version of debezium to 1.9.8.Final (#6740) | https://github.com/apache/seatunnel/commit/c3ac953524 | 2.3.6 |
| [Improve][CDC] Close idle subtasks gorup(reader/writer) in increment phase (#6526) | https://github.com/apache/seatunnel/commit/454c339b9c | 2.3.6 |
| [Improve][JDBC Source] Fix Split can not be cancel (#6825) | https://github.com/apache/seatunnel/commit/ee3b7c3723 | 2.3.6 |
| [Fix] Fix ConnectorSpecificationCheckTest failed (#6828) | https://github.com/apache/seatunnel/commit/52d1020eb7 | 2.3.6 |
| [Hotfix][Jdbc/CDC] Fix postgresql uuid type in jdbc read (#6684) | https://github.com/apache/seatunnel/commit/868ba4d7c7 | 2.3.6 |
| [Improve] Improve read table schema in cdc connector (#6702) | https://github.com/apache/seatunnel/commit/a8c6cc6e0c | 2.3.6 |
| [Improve][Jdbc] Add quote identifier for sql (#6669) | https://github.com/apache/seatunnel/commit/849d748d3d | 2.3.5 |
| [Improve][CDC] Optimize split state memory allocation in increment phase (#6554) | https://github.com/apache/seatunnel/commit/fe33422161 | 2.3.5 |
| [Improve][CDC-Connector]Fix CDC option rule. (#6454) | https://github.com/apache/seatunnel/commit/1ea27afa87 | 2.3.5 |
| [Improve][CDC] Optimize memory allocation for snapshot split reading (#6281) | https://github.com/apache/seatunnel/commit/4856645837 | 2.3.5 |
| [Improve][API] Unify type system api(data & type) (#5872) | https://github.com/apache/seatunnel/commit/b38c7edcc9 | 2.3.5 |
| [Fix][Oracle-CDC] Fix invalid split key when no primary key (#6251) | https://github.com/apache/seatunnel/commit/b83c40a6f6 | 2.3.4 |
| [Feature][Oracle-CDC] Support custom table primary key (#6216) | https://github.com/apache/seatunnel/commit/ae4240ca6b | 2.3.4 |
| [Improve][Oracle-CDC] Clean unused code (#6212) | https://github.com/apache/seatunnel/commit/919a91032a | 2.3.4 |
| [Hotfix][Oracle-CDC] Fix state recovery error when switching a single table to multiple tables (#6211) | https://github.com/apache/seatunnel/commit/74cfe1995f | 2.3.4 |
| [Hotfix][Oracle-CDC] Fix jdbc setFetchSize error (#6210) | https://github.com/apache/seatunnel/commit/b7f06ec6d9 | 2.3.4 |
| [Feature][Oracle-CDC] Support read no primary key table (#6209) | https://github.com/apache/seatunnel/commit/3cb34c2b71 | 2.3.4 |
| [Feature][Connector-V2][Oracle-cdc]Support for oracle cdc (#5196) | https://github.com/apache/seatunnel/commit/aaef22b31b | 2.3.4 |