Issue with 'shortcuts' table silimar like in #11

[TomaszE]

Error is repeating every few days, seems like it's not affecting other tables

Error persisting data for table "shortcuts": Record 118 did not conform to schema: #: #: no subschema matched out of the total 2 subschemas #: expected: null, found: JSONObject #: extraneous key [agents] is not permitted

Full log in comment below:

[TomaszE]

Error persisting data for table "shortcuts": Record 114 did not conform to schema:
#: #: no subschema matched out of the total 2 subschemas
#: expected: null, found: JSONObject
#: extraneous key [agents] is not permitted

[TomaszE]

2019-01-30 21:59:25,736Z main - INFO Running tap-zendesk-chat version 0.1.15 and target-stitch version 1.8.1
2019-01-30 21:59:25,795Z main - INFO Starting tap to discover schemas: tap-env/bin/tap-zendesk-chat --config /tmp/tap_discover_config.json --discover
2019-01-30 21:59:26,216Z tap - INFO HTTP request to "departments" endpoint took 0.227s, returned status code 200
2019-01-30 21:59:26,412Z tap - INFO HTTP request to "account" endpoint took 0.196s, returned status code 403
2019-01-30 21:59:26,413Z tap - INFO Ignoring 403 from account endpoint - this must be an integrated Zendesk account. This endpoint will be excluded from discovery.
2019-01-30 21:59:26,443Z main - INFO Tap exited normally.
2019-01-30 21:59:26,444Z main - INFO Saving list of discovered streams
2019-01-30 21:59:26,501Z main - INFO Saving structure of stream agents
2019-01-30 21:59:26,527Z main - INFO Saving structure of stream chats
2019-01-30 21:59:26,555Z main - INFO Saving structure of stream shortcuts
2019-01-30 21:59:26,566Z main - INFO Saving structure of stream triggers
2019-01-30 21:59:26,607Z main - INFO Saving structure of stream bans
2019-01-30 21:59:26,640Z main - INFO Saving structure of stream departments
2019-01-30 21:59:26,650Z main - INFO Saving structure of stream goals
2019-01-30 21:59:26,673Z main - INFO Getting list of selected properties
2019-01-30 21:59:26,762Z main - INFO Starting tap: tap-env/bin/tap-zendesk-chat --config /tmp/tap_config.json --properties /tmp/properties.json --catalog /tmp/catalog.json --state /tmp/tap_state.json
2019-01-30 21:59:26,764Z main - INFO Starting target: target-env/bin/target-stitch --config /tmp/target_config.json
2019-01-30 21:59:26,947Z tap - INFO Using chat_search_interval_days: 14
2019-01-30 21:59:26,996Z target - INFO Using Stitch import URL https://api.stitchdata.com/v2/import/batch
2019-01-30 21:59:27,195Z tap - INFO HTTP request to "chats" endpoint took 0.247s, returned status code 200
2019-01-30 21:59:27,484Z tap - INFO HTTP request to "chats" endpoint took 0.288s, returned status code 200
2019-01-30 21:59:27,500Z tap - INFO replicated 40 records from "chats" endpoint
2019-01-30 21:59:27,724Z tap - INFO HTTP request to "chats" endpoint took 0.223s, returned status code 200
2019-01-30 21:59:27,972Z tap - INFO HTTP request to "chats" endpoint took 0.247s, returned status code 200
2019-01-30 21:59:27,977Z tap - INFO replicated 11 records from "chats" endpoint
2019-01-30 21:59:27,977Z tap - INFO Using chat_search_interval_days: 14
2019-01-30 21:59:28,183Z tap - INFO HTTP request to "chats" endpoint took 0.204s, returned status code 200
2019-01-30 21:59:28,383Z tap - INFO HTTP request to "chats" endpoint took 0.200s, returned status code 200
2019-01-30 21:59:28,384Z tap - INFO replicated 1 records from "chats" endpoint
2019-01-30 21:59:28,385Z target - INFO Sending batch with 52 messages for table chats to https://api.stitchdata.com/v2/import/batch
2019-01-30 21:59:28,751Z main - INFO State update: adding bookmarks.agents.offset.id = null
2019-01-30 21:59:28,751Z main - INFO State update: adding bookmarks.chats.chat.end_timestamp = "2019-01-30T19:49:32Z"
2019-01-30 21:59:28,751Z main - INFO State update: adding bookmarks.chats.offline_msg.timestamp = "2019-01-29T07:17:53Z"
2019-01-30 21:59:28,751Z main - INFO State update: adding bookmarks.chats.offset.chat.next_url = null
2019-01-30 21:59:28,751Z main - INFO State update: adding bookmarks.chats.offset.offline_msg.next_url = null
2019-01-30 21:59:28,751Z main - INFO State update: adding currently_syncing = "chats"
2019-01-30 21:59:29,109Z tap - INFO HTTP request to "shortcuts" endpoint took 0.723s, returned status code 200
2019-01-30 21:59:29,124Z tap - INFO replicated 322 records from "shortcuts" endpoint
2019-01-30 21:59:29,124Z target - INFO Sending batch with 322 messages for table shortcuts to https://api.stitchdata.com/v2/import/batch
2019-01-30 21:59:29,245Z target - CRITICAL Error persisting data for table "shortcuts": Record 114 did not conform to schema:
2019-01-30 21:59:29,245Z target - CRITICAL #: #: no subschema matched out of the total 2 subschemas
2019-01-30 21:59:29,245Z target - CRITICAL #: expected: null, found: JSONObject
2019-01-30 21:59:29,245Z target - CRITICAL #: extraneous key [agents] is not permitted
2019-01-30 21:59:29,331Z tap - INFO HTTP request to "triggers" endpoint took 0.206s, returned status code 200
2019-01-30 21:59:34,275Z main - WARNING Thread tap did not finish within timeout
2019-01-30 21:59:34,276Z main - ERROR Target exited abnormally with status 1. Terminating tap.
2019-01-30 21:59:34,278Z main - INFO Exit status is: Discovery succeeded. Tap failed with code -15. Target failed with code 1 and error message: "Error persisting data for table "shortcuts": Record 114 did not conform to schema:
#: #: no subschema matched out of the total 2 subschemas
#: expected: null, found: JSONObject
#: extraneous key [agents] is not permitted".

[luandy64]

Similar to #11, we should add to the agents.json file a

"scope": {
  "type": [
    "null",
    "string"
  ]
}

in order to address that extraneous key [agents] is not permitted error. Would you be able to open a PR with this change?

[TomaszE]

Hi , it's behind my skill set level i'm afraid.

[luandy64]

I'm going to tag this and I'm sure someone will pick this up because it's relatively straightforward.

[EternalLearner42]

Hi, thanks for the notification @luandy64 . This seems straightforward enough, I'll take it.

[luandy64]

@TomaszE So a change has been made to the tap by @EternalLearner42 - thanks again - which I'm hoping will fix this error you are seeing.

I'm going to leave this issue open until you can confirm that.

[TomaszE]

Error disappeared - i guess all is good , thanks guys

[luandy64]

Great! Open new issues if you need to 👍