OneSignal
OneSignal source connector
Description
The OneSignal source connector reads data from the OneSignal REST API. It is built on the HTTP source connector and automatically sends password to OneSignal as the Authorization: Basic <token> request header, so you do not need to set Authorization yourself.
Use this connector to ingest OneSignal resources such as apps, players, segments, or notifications as SeaTunnel rows.
Key Features
Source Options
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
| url | String | Yes | - | OneSignal REST API endpoint. Common endpoints include https://onesignal.com/api/v1/apps and https://onesignal.com/api/v1/players. |
| password | String | Yes | - | OneSignal user auth key. The connector sends it as the HTTP Authorization: Basic <password> header. Create one at OneSignal Accounts and Keys. |
| method | String | No | get | HTTP request method. Supported values are GET and POST. |
| headers | Map | No | - | Extra HTTP headers. Note: any Authorization entry configured here is overwritten by the Authorization: Basic <password> header generated from password. |
| params | Map | No | - | HTTP query parameters, such as limit, offset, or other OneSignal API parameters. |
| body | String | No | - | HTTP request body. Useful for endpoints that accept a JSON payload. |
| format | String | No | text | Response format. Use json with schema to read OneSignal JSON as SeaTunnel rows with named fields. Use text to keep the raw response. |
| schema | Config | No | - | Output row structure. Required when format = "json". See Schema Feature. |
| schema.fields | Config | No | - | Field names and SeaTunnel data types used to parse the JSON response. |
| json_field | Config | No | - | Field-level JSONPath mapping. Use it with schema when each output field lives at a different JSON path. |
| content_field | String | No | - | JSONPath expression that selects a JSON fragment before schema parses it. For example, use $.players[*] to flatten a list response. |
| pageing | Config | No | - | HTTP pagination settings inherited from the HTTP source connector. Currently not effective for OneSignal: the reader is created without pagination support, so a job reads only a single response per request. |
| poll_interval_millis | Int | No | - | Request interval in milliseconds for streaming jobs. In batch mode the connector reads once and finishes. |
| retry | Int | No | - | Maximum retry count when an HTTP request fails with IOException. |
| retry_backoff_multiplier_ms | Int | No | 100 | Retry backoff multiplier in milliseconds. |
| retry_backoff_max_ms | Int | No | 10000 | Maximum retry backoff in milliseconds. |
| enable_multi_lines | Boolean | No | false | When true, multiple JSON objects separated by newlines in the response body are treated as separate records. |
| json_filed_missed_return_null | Boolean | No | false | When true, missing JSON fields return null; otherwise a missing field causes an error. |
| common-options | Config | No | - | Source plugin common parameters. See Source Common Options. |
Usage Notes
passwordis sensitive. Avoid hardcoding real keys in shared job files. Use SeaTunnel variable substitution or your deployment secret mechanism.- The connector always adds an
Authorizationheader frompassword. Put other custom headers inheaders. - Set
format = "json"and defineschemawhen you want typed SeaTunnel rows. - Use
content_fieldwhen OneSignal wraps records in a nested array such as$.players[*]. - Use
json_fieldonly when each output field needs its own JSONPath expression. - The OneSignal reader currently ignores
pageing; each job reads a single response. Useparams(for examplelimit) to control how many records that response returns.
Task Examples
Read Apps
env {
parallelism = 1
job.mode = "BATCH"
}
source {
OneSignal {
url = "https://onesignal.com/api/v1/apps"
password = "<onesignal-user-auth-key>"
method = "GET"
format = "json"
schema = {
fields {
id = string
name = string
gcm_key = string
chrome_key = string
site_name = string
created_at = string
updated_at = string
players = int
messageable_players = int
}
}
}
}
sink {
Console {
}
}
Read Players
Query the players endpoint with params. Note that the OneSignal reader currently does not
paginate: a pageing block is accepted but ignored, and the job reads only the single response
returned for the configured params. Use limit to control how many records that response
returns:
env {
parallelism = 1
job.mode = "BATCH"
}
source {
OneSignal {
url = "https://onesignal.com/api/v1/players"
password = "<onesignal-user-auth-key>"
method = "GET"
params = {
app_id = "<your-app-id>"
limit = "50"
}
format = "json"
content_field = "$.players[*]"
schema = {
fields {
id = string
identifier = string
device_type = int
sessions = int
language = string
game_version = string
}
}
}
}
Note: the HTTP source
pageingoption advances the page index by exactly 1 per request, so it cannot express a stride. OneSignal'soffsetis a record offset, not a page number — combining it withpageingwould request overlapping windows (offsets 0, 1, 2, ...), which is why the example above issues a single request instead.
Extract Fields With JSONPath
Use json_field when each output field lives at a different JSON path:
source {
OneSignal {
url = "https://onesignal.com/api/v1/apps"
password = "<onesignal-user-auth-key>"
method = "GET"
format = "json"
json_field = {
id = "$.id"
name = "$.name"
players = "$.players"
site_name = "$.site_name"
}
schema = {
fields {
id = string
name = string
players = int
site_name = string
}
}
}
}
Changelog
Change Log
| Change | Commit | Version |
|---|---|---|
| [improve] http connector options (#8969) | https://github.com/apache/seatunnel/commit/63ff9f910a | 2.3.10 |
| [Feature][Connector-V2] Support TableSourceFactory/TableSinkFactory on http (#5816) | https://github.com/apache/seatunnel/commit/6f49ec6ead | 2.3.4 |
| [Improve][build] Give the maven module a human readable name (#4114) | https://github.com/apache/seatunnel/commit/d7cd601051 | 2.3.1 |
| [Improve][Project] Code format with spotless plugin. (#4101) | https://github.com/apache/seatunnel/commit/a2ab166561 | 2.3.1 |
| [Improve][Connector-V2][Http]Improve json parse option rule for all http connector (#3627) | https://github.com/apache/seatunnel/commit/589e4161ec | 2.3.0 |
| [Improve][Connector-V2][OneSignal]Unified exception for OneSignal connector (#3609) | https://github.com/apache/seatunnel/commit/97cce8c255 | 2.3.0 |
| [Feature][Connector-V2][HTTP] Use json-path parsing (#3510) | https://github.com/apache/seatunnel/commit/1807eb6c95 | 2.3.0 |
| [Hotfix][OptionRule] Fix option rule about all connectors (#3592) | https://github.com/apache/seatunnel/commit/226dc6a119 | 2.3.0 |
| [Feature][Connector-V2][OneSignal]Add OneSignal source conector (#3454) | https://github.com/apache/seatunnel/commit/b318b3166f | 2.3.0 |