Docs for built-in processors #30
Conversation
Deploying with
|
| Latest commit: |
4b7a9eb
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://fc7ab826.conduit-site.pages.dev |
| Branch Preview URL: | https://haris-processors.conduit-site.pages.dev |
docs/processors/builtin.mdx
Outdated
| - If the key is raw and has a schema attached, extracts the field and uses it to replace the entire key. | ||
| - If the key is raw and has no schema, returns an error (not supported). | ||
| - If the key is structured, extracts the field and uses it to replace the entire key. |
There was a problem hiding this comment.
I copied this from the docs in Conduit, but it doesn't appear it's true: https://github.com/ConduitIO/conduit/blob/71c6737846700e24a0cdb04c4fe1a890794b9276/pkg/processor/procbuiltin/extractfield.go#L72-L76.
Yup, examples came to my mind as well, but having some good examples would take some time. |
docs/processors/builtin.mdx
Outdated
|
|
||
| ### parsejsonkey | ||
|
|
||
| Transforms the record's key into structured data, i.e. a JSON object. |
There was a problem hiding this comment.
I think this processor and parsejsonpayload need more details, maybe something like:
| Transforms the record's key into structured data, i.e. a JSON object. | |
| Transforms the record's raw data key (has to be JSON formatted) into structured data, i.e. a JSON object. |
There was a problem hiding this comment.
@hariso looks like you didn't notice the comment, we need to update the payload processor too 👀
There was a problem hiding this comment.
Ah, sorry about that! I'll get your changes in a separate branch. Thanks for calling this out!
There was a problem hiding this comment.
@maha-hajja Thinking again about this... IIUC, the parsejson pair of processors will parse the JSON even when the data is structured, which means that we don't need to say "transform the record's raw data key/payload.after...". Is my understanding correct? I agree that the note about that it has to be JSON formatted should be there.
There was a problem hiding this comment.
@hariso if the data is already structured then it's just returned as is
Co-authored-by: Maha Hajja <[email protected]>
Co-authored-by: Lovro Mažgon <[email protected]>
|
|
||
| ### parsejsonpayload | ||
|
|
||
| Transforms the record's `Payload.After` into structured data, i.e. a JSON object. |
There was a problem hiding this comment.
forgot to update payload processor:
| Transforms the record's `Payload.After` into structured data, i.e. a JSON object. | |
| Transforms the record's raw data `Payload.After` (has to be JSON formatted) into structured data, i.e. a JSON object. |
docs/processors/builtin.mdx
Outdated
|
|
||
| ### parsejsonkey | ||
|
|
||
| Transforms the record's key into structured data, i.e. a JSON object. |
There was a problem hiding this comment.
@hariso looks like you didn't notice the comment, we need to update the payload processor too 👀
Fixes ConduitIO/conduit#966.
I've used the comments in the
procbuiltina great deal. The docs in Conduit mention raw keys/payloads with schema. While a concept of schema exists in Conduit, it's nowhere used. Because of that, I've removed the mention of "schema" here.