aaronleggett96/tap-codat

A Singer.io tap to extract data from the Codat API

tap-codat

This is a Singer tap that produces JSON-formatted data following the Singer spec.

This tap:

• Pulls raw data from Codat
• Extracts the following resources:
  • Accounts
  • Companies
  • Balance Sheets
  • Bank Statements
  • Bills
  • Company Info
  • Credit Notes
  • Customers
  • Invoices
  • Payments
  • Profit and Loss
  • Suppliers
• Outputs the schema for each resource

Quick Start

1. Install

   pip install tap-codat

2. Get an API key

   Refer to the Codat documentation here.

3. Create the config file

   You must create a JSON configuration file that looks like this:

   {
     "start_date": "2010-01-01",
     "api_key": "your-api-key",
     "uat_urls": "true"
   }

   The start_date is the date at which the tap will begin pulling data for streams that support this feature. Note that in the initial version of this tap, this date is unused, as all streams replicate all of the data from Codat during every run.

   Replace your-api-key with the API key you received from Codat. If this token is for the UAT environment, change uat_urls to true.

4. Run the Tap in Discovery Mode

   tap-codat -c config.json -d

   See the Singer docs on discovery mode here.

5. Run the Tap in Sync Mode

   tap-codat -c config.json -p catalog-file.json

Data Formatting

For a few endpoints, this tap reformats the structure of "reports" received from Codat. An example is the balance_sheets stream, which returns a structure like this:

"reports": [
    {
        "assets": {
            "name": "Top-level Category",
            "value": 1,
            "items": [
                {"name": "Inner category A", "value": 2},
                {"name": "Inner category B", "value": 3}
            ]
        }
    }
]

Here, assets describes a hierarchical structure. It is recursive in that any of the items can themselves contain an array of items. This is not a structure that easily can fit into a flat, tabular structure. To alleviate this, this tap restructures this data into this format:

"reports": [
    {
        "assets": [
            {
                "name": "Top-level Category",
                "value": 1,
                "name_0": "Top-level Category"
            },
            {
                "name": "Inner category A",
                "value": 2,
                "name_0": "Top-level Category",
                "name_1": "Inner category A"
            },
            {
                "name": "Inner category B",
                "value": 3,
                "name_0": "Top-level Category",
                "name_1": "Inner category B"
            },
        ]
    }
]

The structure is flattened into a single array of objects. The "name" and "value" properties are left as-is for each item, but now each items contains properties "name_X" where X represents the category hierarchy. That is, if your category hierarchy is

A
- B
- - C
- D
E
- F

Then name_0 will always be either A or E, name_1 will always be B, D, or F and name_2 will only ever be C.

---

Copyright © 2017 Stitch