TDL-18578 : Errors Observed in Tap Failure Dashboard

[shantanu73]

Description of change

(write a short description or paste a link to JIRA)

1. Added a new schema chat_webpath_array.json for chat webpath which validates array elements.
2. Modified chats.json to include chat_webpath_array.json schema for webpath in chats API.

Manual QA steps

Risks

Rollback steps

• revert this branch

[dbshah1212]

There is no meaning of putting integer after string so do put integer before the string in priority.

[shantanu73]

The schema is able to validate the input json irrespective of the order in which these data types are kept.
The integration tests ran fine and I've validated it through online schema validator as well.
So, wanted to confirm if you're suggesting it as a best practice or does it actually impact any given input ?

[luandy64]

The issue is that the singer.Transformer cares about the order.

Here's an example

$ python3
>>> from singer import Transformer
>>> schema = {
...   "type": "object",
...   "properties": {
...     "string-int": {"type": ["null", "string", "integer"]},
...     "int-string": {"type": ["null", "integer", "string"]}
...   }
... }
>>> records = [
...   {"string-int": 1, "int-string": 1},

...   {"string-int": "2", "int-string": "2"}
... ]
>>> with Transformer() as tm:
...   for record in records:
...     print(tm.transform(record, schema))
...
{'string-int': '1', 'int-string': 1}
{'string-int': '2', 'int-string': 2}

Notice that

1. string-int is a field that prioritizes string
2. int-string is a field that prioritizes integer
3. The transformer forces string-int to be output as a string
4. The transformer forces int-string to be output as an integer

[dbshah1212]

Instead of creating 2 separate files, we can also put anyOf inside the chat_webpath file.

[shantanu73]

Okay, will make this change.