Skip to main content
Version: 2.3.1

Klaviyo

Klaviyo source connector

Description

Used to read data from Klaviyo.

Key features

Options

nametyperequireddefault value
urlStringYes-
private_keyStringYes-
revisionStringYes-
methodStringNoget
schemaConfigNo-
schema.fieldsConfigNo-
formatStringNojson
paramsMapNo-
bodyStringNo-
json_fieldConfigNo-
content_jsonStringNo-
poll_interval_msintNo-
retryintNo-
retry_backoff_multiplier_msintNo100
retry_backoff_max_msintNo10000
common-optionsconfigNo-

url [String]

http request url

private_key [String]

API private key for login, you can get more detail at this link:

https://developers.klaviyo.com/en/docs/retrieve_api_credentials

revision [String]

API endpoint revision (format: YYYY-MM-DD)

method [String]

http request method, only supports GET, POST method

params [Map]

http params

body [String]

http body

poll_interval_ms [int]

request http api interval(millis) in stream mode

retry [int]

The max retry times if request http return to IOException

retry_backoff_multiplier_ms [int]

The retry-backoff times(millis) multiplier if request http failed

retry_backoff_max_ms [int]

The maximum retry-backoff times(millis) if request http failed

format [String]

the format of upstream data, now only support json text, default json.

when you assign format is json, you should also assign schema option, for example:

upstream data is the following:

{
"code": 200,
"data": "get success",
"success": true
}

you should assign schema as the following:

schema {
fields {
code = int
data = string
success = boolean
}
}

connector will generate data as the following:

codedatasuccess
200get successtrue

when you assign format is text, connector will do nothing for upstream data, for example:

upstream data is the following:

{
"code": 200,
"data": "get success",
"success": true
}

connector will generate data as the following:

content
{"code": 200, "data": "get success", "success": true}

schema [Config]

fields [Config]

the schema fields of upstream data

content_json [String]

This parameter can get some json data.If you only need the data in the 'book' section, configure content_field = "$.store.book.*".

If your return data looks something like this.

{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
},
"expensive": 10
}

You can configure content_field = "$.store.book.*" and the result returned looks like this:

[
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
]

Then you can get the desired result with a simpler schema,like

Http {
url = "http://mockserver:1080/contentjson/mock"
method = "GET"
format = "json"
content_field = "$.store.book.*"
schema = {
fields {
category = string
author = string
title = string
price = string
}
}
}

Here is an example:

json_field [Config]

This parameter helps you configure the schema,so this parameter must be used with schema.

If your data looks something like this:

{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
},
"expensive": 10
}

You can get the contents of 'book' by configuring the task as follows:

source {
Http {
url = "http://mockserver:1080/jsonpath/mock"
method = "GET"
format = "json"
json_field = {
category = "$.store.book[*].category"
author = "$.store.book[*].author"
title = "$.store.book[*].title"
price = "$.store.book[*].price"
}
schema = {
fields {
category = string
author = string
title = string
price = string
}
}
}
}

common options

Source plugin common parameters, please refer to Source Common Options for details

Example

Klaviyo {
url = "https://a.klaviyo.com/api/lists/"
private_key = "Seatunnel-test"
revision = "2020-10-17"
method = "GET"
format = "json"
schema = {
fields {
type = string
id = string
attributes = {
name = string
created = string
updated = string
}
links = {
self = string
}
}
}
}

Changelog

next version

  • Add Klaviyo Source Connector
  • [Feature][Connector-V2][HTTP] Use json-path parsing (3510)