Skip to main content
Version: Next

OneSignal

OneSignal source connector

Description​

The OneSignal source connector reads data from the OneSignal REST API. It is built on the HTTP source connector and automatically sends password to OneSignal as the Authorization: Basic <token> request header, so you do not need to set Authorization yourself.

Use this connector to ingest OneSignal resources such as apps, players, segments, or notifications as SeaTunnel rows.

Key Features​

Source Options​

NameTypeRequiredDefaultDescription
urlStringYes-OneSignal REST API endpoint. Common endpoints include https://onesignal.com/api/v1/apps and https://onesignal.com/api/v1/players.
passwordStringYes-OneSignal user auth key. The connector sends it as the HTTP Authorization: Basic <password> header. Create one at OneSignal Accounts and Keys.
methodStringNogetHTTP request method. Supported values are GET and POST.
headersMapNo-Extra HTTP headers. Do not put Authorization here unless you want to override the header generated from password.
paramsMapNo-HTTP query parameters, such as limit, offset, or other OneSignal API parameters.
bodyStringNo-HTTP request body. Useful for endpoints that accept a JSON payload.
formatStringNojsonResponse format. Use json with schema to read OneSignal JSON as SeaTunnel rows with named fields. Use text to keep the raw response.
schemaConfigNo-Output row structure. Required when format = "json". See Schema Feature.
schema.fieldsConfigNo-Field names and SeaTunnel data types used to parse the JSON response.
json_fieldConfigNo-Field-level JSONPath mapping. Use it with schema when each output field lives at a different JSON path.
content_fieldStringNo-JSONPath expression that selects a JSON fragment before schema parses it. For example, use $.players[*] to flatten a list response.
pageingConfigNo-HTTP pagination settings inherited from the HTTP source connector. OneSignal paged endpoints use page / per_page parameters.
poll_interval_millisIntNo-Request interval in milliseconds for streaming jobs. In batch mode the connector reads once and finishes.
retryIntNo-Maximum retry count when an HTTP request fails with IOException.
retry_backoff_multiplier_msIntNo100Retry backoff multiplier in milliseconds.
retry_backoff_max_msIntNo10000Maximum retry backoff in milliseconds.
enable_multi_linesBooleanNofalseWhen true, multiple JSON objects separated by newlines in the response body are treated as separate records.
json_filed_missed_return_nullBooleanNofalseWhen true, missing JSON fields return null; otherwise a missing field causes an error.
common-optionsConfigNo-Source plugin common parameters. See Source Common Options.

Usage Notes​

  • password is sensitive. Avoid hardcoding real keys in shared job files. Use SeaTunnel variable substitution or your deployment secret mechanism.
  • The connector always adds an Authorization header from password. Put other custom headers in headers.
  • Set format = "json" and define schema when you want typed SeaTunnel rows.
  • Use content_field when OneSignal wraps records in a nested array such as $.players[*].
  • Use json_field only when each output field needs its own JSONPath expression.
  • OneSignal paged endpoints accept page and per_page query parameters; configure them through params and pageing.

Task Examples​

Read Apps​

env {
  parallelism = 1
  job.mode = "BATCH"
}

source {
  OneSignal {
    url = "https://onesignal.com/api/v1/apps"
    password = "<onesignal-user-auth-key>"
    method = "GET"
    format = "json"
    schema = {
      fields {
        id = string
        name = string
        gcm_key = string
        chrome_key = string
        site_name = string
        created_at = string
        updated_at = string
        players = int
        messageable_players = int
      }
    }
  }
}

sink {
  Console {
  }
}

Read Players With Pagination​

Use params together with pageing to walk through paged OneSignal endpoints:

env {
  parallelism = 1
  job.mode = "BATCH"
}

source {
  OneSignal {
    url = "https://onesignal.com/api/v1/players"
    password = "<onesignal-user-auth-key>"
    method = "GET"
    params = {
      app_id = "<your-app-id>"
      limit = "50"
      offset = "0"
    }
    pageing = {
      page_field = "offset"
      start_page_number = 0
      page_step = 50
      total_page_size = 10
      use_placeholder_replacement = false
    }
    format = "json"
    content_field = "$.players[*]"
    schema = {
      fields {
        id = string
        identifier = string
        device_type = int
        sessions = int
        language = string
        game_version = string
      }
    }
  }
}

Extract Fields With JSONPath​

Use json_field when each output field lives at a different JSON path:

source {
  OneSignal {
    url = "https://onesignal.com/api/v1/apps"
    password = "<onesignal-user-auth-key>"
    method = "GET"
    format = "json"
    json_field = {
      id = "$.id"
      name = "$.name"
      players = "$.players"
      site_name = "$.site_name"
    }
    schema = {
      fields {
        id = string
        name = string
        players = int
        site_name = string
      }
    }
  }
}

Changelog​

Change Log
ChangeCommitVersion
[improve] http connector options (#8969)https://github.com/apache/seatunnel/commit/63ff9f910a2.3.10
[Feature][Connector-V2] Support TableSourceFactory/TableSinkFactory on http (#5816)https://github.com/apache/seatunnel/commit/6f49ec6ead2.3.4
[Improve][build] Give the maven module a human readable name (#4114)https://github.com/apache/seatunnel/commit/d7cd6010512.3.1
[Improve][Project] Code format with spotless plugin. (#4101)https://github.com/apache/seatunnel/commit/a2ab1665612.3.1
[Improve][Connector-V2][Http]Improve json parse option rule for all http connector (#3627)https://github.com/apache/seatunnel/commit/589e4161ec2.3.0
[Improve][Connector-V2][OneSignal]Unified exception for OneSignal connector (#3609)https://github.com/apache/seatunnel/commit/97cce8c2552.3.0
[Feature][Connector-V2][HTTP] Use json-path parsing (#3510)https://github.com/apache/seatunnel/commit/1807eb6c952.3.0
[Hotfix][OptionRule] Fix option rule about all connectors (#3592)https://github.com/apache/seatunnel/commit/226dc6a1192.3.0
[Feature][Connector-V2][OneSignal]Add OneSignal source conector (#3454)https://github.com/apache/seatunnel/commit/b318b3166f2.3.0