Skip to main content
Version: Next

My Hours

My Hours source connector

Support Those Engines​

Spark
Flink
SeaTunnel Zeta

Description​

Used to read data from My Hours through the My Hours REST API. The connector logs in with the configured email and password, obtains an access token, and then sends the configured HTTP request with that token.

Key features​

Supported DataSource Info​

In order to use the My Hours connector, the following dependencies are required. They can be downloaded via install-plugin.sh or from the Maven central repository.

DatasourceSupported VersionsDependency
My HoursuniversalDownload

Source Options​

NameTypeRequiredDefaultDescription
urlStringYes-My Hours API request URL.
emailStringYes-My Hours login email address.
passwordStringYes-My Hours login password.
schemaConfigNo-Required when format is json. For more details, see Schema Feature.
schema.fieldsConfigNo-The schema fields of upstream data.
json_fieldConfigNo-Extract fields from a JSON response by JSONPath. Use it together with schema.
content_fieldStringNo-Extract one part of a JSON response, such as $.store.book.*, before schema parsing.
formatStringNotextResponse format. Supports json and text. Set json when using schema, json_field, or content_field.
methodStringNoGETHTTP request method. Supports GET and POST.
headersMapNo-Extra HTTP headers. The connector adds the My Hours Authorization header after login.
paramsMapNo-HTTP query parameters.
bodyStringNo-HTTP request body.
poll_interval_millisIntNo-Request interval in milliseconds for stream mode.
retryIntNo-Maximum retry times when the request throws IOException.
retry_backoff_multiplier_msIntNo100Retry backoff multiplier in milliseconds.
retry_backoff_max_msIntNo10000Maximum retry backoff in milliseconds.
json_filed_missed_return_nullBooleanNofalseReturn null when a configured JSON field is missing.
common-optionsNo-Source plugin common parameters. See Source Common Options.

How to Create a My Hours Data Synchronization Jobs​

env {
  parallelism = 1
  job.mode = "BATCH"
}

source {
  MyHours {
    url = "https://api2.myhours.com/api/Projects/getAll"
    email = "seatunnel@test.com"
    password = "********"
    method = "GET"
    format = "json"
    schema {
       fields {
         name = string
         archived = boolean
         dateArchived = string
         dateCreated = string
         clientName = string
         budgetAlertPercent = string
         budgetType = int
         totalTimeLogged = double
         budgetValue = double
         totalAmount = double
         totalExpense = double
         laborCost = double
         totalCost = double
         billableTimeLogged = double
         totalBillableAmount = double
         billable = boolean
         roundType = int
         roundInterval = int
         budgetSpentPercentage = double
         budgetTarget = int
         budgetPeriodType = string
         budgetSpent = string
         id = string
       }
    }
  }
}

# Console printing of the read data
sink {
  Console {
    parallelism = 1
  }
}

Parameter Interpretation​

format​

when you assign format is json, you should also assign schema option, for example:

upstream data is the following:

{
  "code": 200,
  "data": "get success",
  "success": true
}

you should assign schema as the following:


schema {
  fields {
    code = int
    data = string
    success = boolean
  }
}

connector will generate data as the following:

codedatasuccess
200get successtrue

when you assign format is text, connector will do nothing for upstream data, for example:

upstream data is the following:

{
  "code": 200,
  "data": "get success",
  "success": true
}

connector will generate data as the following:

content
{"code": 200, "data": "get success", "success": true}

content_field​

This parameter can get some json data.If you only need the data in the 'book' section, configure content_field = "$.store.book.*".

If your return data looks something like this.

{
  "store": {
    "book": [
      {
        "category": "reference",
        "author": "Nigel Rees",
        "title": "Sayings of the Century",
        "price": 8.95
      },
      {
        "category": "fiction",
        "author": "Evelyn Waugh",
        "title": "Sword of Honour",
        "price": 12.99
      }
    ],
    "bicycle": {
      "color": "red",
      "price": 19.95
    }
  },
  "expensive": 10
}

You can configure content_field = "$.store.book.*" and the result returned looks like this:

[
  {
    "category": "reference",
    "author": "Nigel Rees",
    "title": "Sayings of the Century",
    "price": 8.95
  },
  {
    "category": "fiction",
    "author": "Evelyn Waugh",
    "title": "Sword of Honour",
    "price": 12.99
  }
]

Then you can get the desired result with a simpler schema,like

MyHours {
  url = "http://mockserver:1080/contentjson/mock"
  email = "seatunnel@test.com"
  password = "********"
  method = "GET"
  format = "json"
  content_field = "$.store.book.*"
  schema = {
    fields {
      category = string
      author = string
      title = string
      price = string
    }
  }
}

json_field​

This parameter helps you configure the schema,so this parameter must be used with schema.

If your data looks something like this:

{ 
  "store": {
    "book": [
      {
        "category": "reference",
        "author": "Nigel Rees",
        "title": "Sayings of the Century",
        "price": 8.95
      },
      {
        "category": "fiction",
        "author": "Evelyn Waugh",
        "title": "Sword of Honour",
        "price": 12.99
      }
    ],
    "bicycle": {
      "color": "red",
      "price": 19.95
    }
  },
  "expensive": 10
}

You can get the contents of 'book' by configuring the task as follows:

source {
  MyHours {
    url = "http://mockserver:1080/jsonpath/mock"
    email = "seatunnel@test.com"
    password = "********"
    method = "GET"
    format = "json"
    json_field = {
      category = "$.store.book[*].category"
      author = "$.store.book[*].author"
      title = "$.store.book[*].title"
      price = "$.store.book[*].price"
    }
    schema = {
      fields {
        category = string
        author = string
        title = string
        price = string
      }
    }
  }
}

Changelog​

Change Log
ChangeCommitVersion
[Feature][connector-http] Parameters support placeholder replacement (#9184)https://github.com/apache/seatunnel/commit/8617014edc2.3.11
[improve] http connector options (#8969)https://github.com/apache/seatunnel/commit/63ff9f910a2.3.10
[Fix][connector-http] fix when post have param (#8434)https://github.com/apache/seatunnel/commit/c1b2675ab02.3.10
[Improve][dist]add shade check rule (#8136)https://github.com/apache/seatunnel/commit/51ef8000162.3.9
[Feature][Connector-V2] Support TableSourceFactory/TableSinkFactory on http (#5816)https://github.com/apache/seatunnel/commit/6f49ec6ead2.3.4
[Improve][build] Give the maven module a human readable name (#4114)https://github.com/apache/seatunnel/commit/d7cd6010512.3.1
[Improve][Project] Code format with spotless plugin. (#4101)https://github.com/apache/seatunnel/commit/a2ab1665612.3.1
[Improve][Connector-V2][Http]Improve json parse option rule for all http connector (#3627)https://github.com/apache/seatunnel/commit/589e4161ec2.3.0
[Feature][Connector-V2][HTTP] Use json-path parsing (#3510)https://github.com/apache/seatunnel/commit/1807eb6c952.3.0
[Hotfix][OptionRule] Fix option rule about all connectors (#3592)https://github.com/apache/seatunnel/commit/226dc6a1192.3.0
[Improve][Connector-V2][MyHours]Unified exception for MyHours connector (#3538)https://github.com/apache/seatunnel/commit/48ab7c97d52.3.0
[HotFix][Core][API] Fix OptionValidation error code (#3439)https://github.com/apache/seatunnel/commit/ace219f3762.3.0
[Improve][Connector-V2][My Hours]Add http method enum && Improve My Hours connector option rule (#3390)https://github.com/apache/seatunnel/commit/a86c9d90f72.3.0
[Feature][Connector-V2][Http] Add option rules && Improve Myhours sink connector (#3351)https://github.com/apache/seatunnel/commit/cc8bb60c832.3.0
[Feature][Connector-V2][My Hours] Add My Hours Source Connector (#3228)https://github.com/apache/seatunnel/commit/4104a3e30e2.3.0