diff --git a/README.md b/README.md index f0aef580d..38956fd96 100644 --- a/README.md +++ b/README.md @@ -221,7 +221,7 @@ The full example in our case would be: ## Connectors For the full list of available connectors, see -the [Connector List](https://conduit.io/docs/connectors/connector-list). If +the [Connector List](https://conduit.io/docs/using/connectors/list). If there's a connector that you're looking for that isn't available in Conduit, please file an [issue](https://github.com/ConduitIO/conduit/issues/new?assignees=&labels=triage&template=3-connector-request.yml&title=Connector%3A+%3Cresource%3E+%5BSource%2FDestination%5D) . @@ -266,15 +266,15 @@ pipeline. It can either change the record (i.e. **transform** it) or **filter** it out based on some criteria. Conduit provides a number of built-in processors, which can be used to manipulate fields, -send requests to HTTP endpoints, and more, check [Builtin processors](https://conduit.io/docs/processors/builtin/) +send requests to HTTP endpoints, and more, check [Builtin processors](https://conduit.io/docs/using/processors/builtin/) for the list of built-in processors and documentations. -Conduit also provides the ability to write your own [Standalone Processor](https://conduit.io/docs/processors/standalone/building), -or you can use the built-in processor [`custom.javascript`](https://conduit.io/docs/processors/builtin/custom.javascript) +Conduit also provides the ability to write your own [Standalone Processor](https://conduit.io/docs/developing/processors/building), +or you can use the built-in processor [`custom.javascript`](https://conduit.io/docs/using/processors/builtin/custom.javascript) to write custom processors in JavaScript. More detailed information as well as examples can be found in -the [Processors documentation](https://conduit.io/docs/processors/getting-started). +the [Processors documentation](https://conduit.io/docs/using/processors/getting-started). ## API @@ -315,13 +315,13 @@ visit [Conduit.io/docs](https://conduit.io/docs). If you are interested in internals of Conduit we have prepared some technical documentation: -- [Pipeline Semantics](https://conduit.io/docs/features/pipeline-semantics) explains the internals of how +- [Pipeline Semantics](https://conduit.io/docs/core-concepts/pipeline-semantics) explains the internals of how a Conduit pipeline works. -- [Pipeline Configuration Files](https://conduit.io/docs/pipeline-configuration-files) +- [Pipeline Configuration Files](https://conduit.io/docs/using/pipelines/configuration-file) explains how you can define pipelines using YAML files. -- [Processors](https://conduit.io/docs/processors/getting-started) contains examples and more information about +- [Processors](https://conduit.io/docs/using/processors/getting-started) contains examples and more information about Conduit processors. -- [Conduit Architecture](https://conduit.io/docs/getting-started/architecture) +- [Conduit Architecture](https://conduit.io/docs/core-concepts/architecture) will give you a high-level overview of Conduit. - [Conduit Metrics](docs/metrics.md) provides more information about how Conduit exposes metrics. diff --git a/docs/architecture-decision-records/20230123-pipeline-configuration-files.md b/docs/architecture-decision-records/20230123-pipeline-configuration-files.md index 9b7439185..e1608f180 100644 --- a/docs/architecture-decision-records/20230123-pipeline-configuration-files.md +++ b/docs/architecture-decision-records/20230123-pipeline-configuration-files.md @@ -65,4 +65,4 @@ can be ( only incompatible changes cause the state to be deleted). ## Related -[Pipeline configuration files documentation](https://conduit.io/docs/pipeline-configuration-files) +[Pipeline configuration files documentation](https://conduit.io/docs/using/pipelines/configuration-file) diff --git a/docs/connectors.md b/docs/connectors.md index b74a167a2..55103f426 100644 --- a/docs/connectors.md +++ b/docs/connectors.md @@ -40,7 +40,7 @@ and it can be run by others, as is, without them having to follow any additional If you want to create your own Conduit binary with a different set of built-in connectors, you can build-in an [existing connector](#the-list). Alternatively, you can also create your own -[using the Conduit connector template or the Conduit Connector SDK](https://conduit.io/docs/connectors/building/). +[using the Conduit connector template or the Conduit Connector SDK](https://conduit.io/docs/developing/connectors). Once you have chosen a connector to be built-in, you can: @@ -80,7 +80,7 @@ In addition to built-in connectors, Conduit can be used together with standalone to enable even more data streaming use cases. The Conduit team and other community developers create and maintain standalone connectors. -Learn more about how you can install [standalone connectors to Conduit here](https://conduit.io/docs/connectors/installing). +Learn more about how you can install [standalone connectors to Conduit here](https://conduit.io/docs/using/connectors/installing). ### Source & Destination @@ -89,4 +89,4 @@ to write to a downstream data store. ### The List -A full list of connectors is hosted here: . +A full list of connectors is hosted here: . diff --git a/docs/design-documents/20231117-better-processors.md b/docs/design-documents/20231117-better-processors.md index fdf254583..72ab9eb01 100644 --- a/docs/design-documents/20231117-better-processors.md +++ b/docs/design-documents/20231117-better-processors.md @@ -180,7 +180,7 @@ plugin. We propose a similar approach to pluggable processors as we already have pluggable connectors. This means that processors would be loaded at startup and added to the processor plugin registry. Processors would be addressable using a similar naming scheme as connectors (more on -[referencing connectors](https://conduit.io/docs/connectors/referencing)). Built-in +[referencing connectors](https://conduit.io/docs/using/connectors/referencing)). Built-in processors would be created in a similar manner as built-in connectors, in that they would implement the same interface and could essentially be extracted into standalone processors (more info on @@ -289,7 +289,7 @@ is to use Go comments for this purpose. We want to take a similar approach like [paramgen](https://github.com/ConduitIO/conduit-connector-sdk/tree/main/cmd/paramgen) in the connector SDK. The developer should be able to write the documentation once in a Go idiomatic way and use a tool to extract it into a format that can be used on the -[documentation website](https://conduit.io/docs/introduction/getting-started/). +[documentation website](https://conduit.io/docs). We can use the same tool to extract the documentation into the connector specification (see [processor lifecycle](#processor-lifecycle)). This way the same documentation can @@ -577,7 +577,7 @@ processors: ### `decode.avro` This processor is the same as the old processor -[`decodewithschema`](https://conduit.io/docs/processors/builtin/#decodewithschemakey), +[`decodewithschema`](https://conduit.io/docs/using/processors/builtin/#decodewithschemakey), except that it focuses solely on decoding Avro values. If we add support for more formats in the future it will be in the shape of new processors. This way the processors will be easier to discover and we can tailor the processor configuration @@ -586,7 +586,7 @@ we can still reuse the same code for most of the processor. #### Configuration -Taken from [existing docs](https://conduit.io/docs/processors/builtin/#configuration). +Taken from [existing docs](https://conduit.io/docs/using/processors/builtin/#configuration). - `url` (required) - URL of the schema registry (e.g. `http://localhost:8085`). - `auth.basic.username` (optional) - Configures the username to use with basic @@ -608,7 +608,7 @@ Taken from [existing docs](https://conduit.io/docs/processors/builtin/#configura #### Example -Taken from [existing docs](https://conduit.io/docs/processors/builtin/#example). +Taken from [existing docs](https://conduit.io/docs/using/processors/builtin/#example). ```yaml processors: @@ -626,7 +626,7 @@ processors: ### `decode.json` This processor replaces the existing -[`parsejson`](https://conduit.io/docs/processors/builtin/#parsejsonkey) processor +[`parsejson`](https://conduit.io/docs/using/processors/builtin/#parsejsonkey) processor and additionally gives the user the ability to specify which field should be decoded. It takes raw data (string) from the target field, parses it as JSON and stores the decoded structured data in the target field. @@ -651,7 +651,7 @@ processors: ### `encode.avro` This processor is the same as the old processor -[`encodewithschema`](https://conduit.io/docs/processors/builtin/#encodewithschemakey), +[`encodewithschema`](https://conduit.io/docs/using/processors/builtin/#encodewithschemakey), except that it focuses solely on encoding Avro values. If we add support for more formats in the future it will be in the shape of new processors. This way the processors will be easier to discover and we can tailor the processor configuration @@ -664,7 +664,7 @@ Differences compared to the old processor: #### Configuration -Taken from [existing docs](https://conduit.io/docs/processors/builtin/#configuration-1). +Taken from [existing docs](https://conduit.io/docs/using/processors/builtin/#configuration-1). - `url` (required) - URL of the schema registry (e.g. `http://localhost:8085`). - `schema.strategy` (required, Enum: `preRegistered`,`autoRegister`) - Specifies which @@ -701,7 +701,7 @@ Taken from [existing docs](https://conduit.io/docs/processors/builtin/#configura #### Example -Taken from [existing docs](https://conduit.io/docs/processors/builtin/#example-1). +Taken from [existing docs](https://conduit.io/docs/using/processors/builtin/#example-1). ```yaml processors: @@ -749,7 +749,7 @@ We are skipping the design of this processor, as it will not be part of the ### `unwrap.kafkaconnect` This processor together with `unwrap.debezium` replaces the existing -[`unwrap`](https://conduit.io/docs/processors/builtin/#unwrap) processor, but works +[`unwrap`](https://conduit.io/docs/using/processors/builtin/#unwrap) processor, but works the same way under the hood. It additionally gives the user the ability to specify which field is the source of the kafkaconnect record. @@ -774,7 +774,7 @@ processors: ### `unwrap.debezium` This processor together with `unwrap.kafkaconnect` replaces the existing -[`unwrap`](https://conduit.io/docs/processors/builtin/#unwrap) processor, but works +[`unwrap`](https://conduit.io/docs/using/processors/builtin/#unwrap) processor, but works the same way under the hood. It additionally gives the user the ability to specify which field is the source of the debezium record. @@ -799,14 +799,14 @@ processors: ### `webhook.http` This processor replaces the existing -[`httprequest`](https://conduit.io/docs/processors/builtin/#httprequest) processor, +[`httprequest`](https://conduit.io/docs/using/processors/builtin/#httprequest) processor, but works the same way under the hood. It additionally gives the user the ability to specify what is sent as the body and where the response body and status are stored (configuration options `request.body`, `response.body` and `response.status`). #### Configuration -Taken from [existing docs](https://conduit.io/docs/processors/builtin/#configuration-5). +Taken from [existing docs](https://conduit.io/docs/using/processors/builtin/#configuration-5). - `request.url` (required) - URL used in the HTTP request. - `request.method` (optional, default=`POST`) - HTTP request method to be used. diff --git a/docs/design-documents/20240812-recover-from-pipeline-errors.md b/docs/design-documents/20240812-recover-from-pipeline-errors.md index 75af9d2f9..205fbce7a 100644 --- a/docs/design-documents/20240812-recover-from-pipeline-errors.md +++ b/docs/design-documents/20240812-recover-from-pipeline-errors.md @@ -81,7 +81,7 @@ reset. ### Dead-letter queue errors -If a DLQ ([Dead-letter queue](https://conduit.io/docs/features/dead-letter-queue)) +If a DLQ ([Dead-letter queue](https://conduit.io/docs/using/other-features/dead-letter-queue)) is configured with a nack threshold greater than 0, the user has configured a DLQ and we should respect that. This means that if the nack threshold gets exceeded and the DLQ returns an error, that error should degrade the pipeline diff --git a/docs/writing-a-connector-guidelines.md b/docs/writing-a-connector-guidelines.md index 3c14473fa..a570553f9 100644 --- a/docs/writing-a-connector-guidelines.md +++ b/docs/writing-a-connector-guidelines.md @@ -83,7 +83,7 @@ destination connector. ### Use built-in connectors as examples -Conduit's [built-in connectors](https://conduit.io/docs/connectors/connector-list/) +Conduit's [built-in connectors](https://conduit.io/docs/using/connectors/list/) are great references that can be used when writing connectors. They are also the first ones to be updated with latest SDK and protocol changes. diff --git a/pkg/plugin/processor/builtin/impl/avro/decode.go b/pkg/plugin/processor/builtin/impl/avro/decode.go index b5c6e760e..94ed0779a 100644 --- a/pkg/plugin/processor/builtin/impl/avro/decode.go +++ b/pkg/plugin/processor/builtin/impl/avro/decode.go @@ -36,7 +36,7 @@ type decoder interface { type decodeConfig struct { // The field that will be decoded. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" default:".Payload.After"` fieldResolver sdk.ReferenceResolver diff --git a/pkg/plugin/processor/builtin/impl/avro/decode_paramgen.go b/pkg/plugin/processor/builtin/impl/avro/decode_paramgen.go index 507842999..58759ab66 100644 --- a/pkg/plugin/processor/builtin/impl/avro/decode_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/avro/decode_paramgen.go @@ -15,7 +15,7 @@ func (decodeConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ decodeConfigField: { Default: ".Payload.After", - Description: "The field that will be decoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "The field that will be decoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{}, }, diff --git a/pkg/plugin/processor/builtin/impl/avro/encode.go b/pkg/plugin/processor/builtin/impl/avro/encode.go index 14e0845e8..cea8adce6 100644 --- a/pkg/plugin/processor/builtin/impl/avro/encode.go +++ b/pkg/plugin/processor/builtin/impl/avro/encode.go @@ -37,7 +37,7 @@ type encoder interface { type encodeConfig struct { // The field that will be encoded. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" default:".Payload.After"` Schema schemaConfig `json:"schema"` diff --git a/pkg/plugin/processor/builtin/impl/avro/encode_paramgen.go b/pkg/plugin/processor/builtin/impl/avro/encode_paramgen.go index a9268b596..13027b1ec 100644 --- a/pkg/plugin/processor/builtin/impl/avro/encode_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/avro/encode_paramgen.go @@ -19,7 +19,7 @@ func (encodeConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ encodeConfigField: { Default: ".Payload.After", - Description: "The field that will be encoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "The field that will be encoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{}, }, diff --git a/pkg/plugin/processor/builtin/impl/base64/decode.go b/pkg/plugin/processor/builtin/impl/base64/decode.go index 40ee9a9b7..43833571e 100644 --- a/pkg/plugin/processor/builtin/impl/base64/decode.go +++ b/pkg/plugin/processor/builtin/impl/base64/decode.go @@ -38,7 +38,7 @@ type decodeConfig struct { // Field is the reference to the target field. Note that it is not allowed to // base64 decode the `.Position` field. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" validate:"required,exclusion=.Position"` } diff --git a/pkg/plugin/processor/builtin/impl/base64/decode_paramgen.go b/pkg/plugin/processor/builtin/impl/base64/decode_paramgen.go index b5e698990..132f94411 100644 --- a/pkg/plugin/processor/builtin/impl/base64/decode_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/base64/decode_paramgen.go @@ -15,7 +15,7 @@ func (decodeConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ decodeConfigField: { Default: "", - Description: "Field is the reference to the target field. Note that it is not allowed to\nbase64 decode the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Field is the reference to the target field. Note that it is not allowed to\nbase64 decode the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{ config.ValidationRequired{}, diff --git a/pkg/plugin/processor/builtin/impl/base64/encode.go b/pkg/plugin/processor/builtin/impl/base64/encode.go index dbcbdfb6b..fbf5640f8 100644 --- a/pkg/plugin/processor/builtin/impl/base64/encode.go +++ b/pkg/plugin/processor/builtin/impl/base64/encode.go @@ -39,7 +39,7 @@ type encodeConfig struct { // Field is a reference to the target field. Note that it is not allowed to // base64 encode the `.Position` field. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" validate:"required,exclusion=.Position"` } diff --git a/pkg/plugin/processor/builtin/impl/base64/encode_paramgen.go b/pkg/plugin/processor/builtin/impl/base64/encode_paramgen.go index b22844786..bb98786a9 100644 --- a/pkg/plugin/processor/builtin/impl/base64/encode_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/base64/encode_paramgen.go @@ -15,7 +15,7 @@ func (encodeConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ encodeConfigField: { Default: "", - Description: "Field is a reference to the target field. Note that it is not allowed to\nbase64 encode the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Field is a reference to the target field. Note that it is not allowed to\nbase64 encode the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{ config.ValidationRequired{}, diff --git a/pkg/plugin/processor/builtin/impl/error.go b/pkg/plugin/processor/builtin/impl/error.go index c6567c63f..ebbae5a34 100644 --- a/pkg/plugin/processor/builtin/impl/error.go +++ b/pkg/plugin/processor/builtin/impl/error.go @@ -55,7 +55,7 @@ func (p *errorProcessor) Specification() (sdk.Specification, error) { Description: `Any time a record is passed to this processor it returns an error, which results in the record being sent to the DLQ if it's configured, or the pipeline stopping. -**Important:** Make sure to add a [condition](https://conduit.io/docs/processors/conditions) +**Important:** Make sure to add a [condition](https://conduit.io/docs/using/processors/conditions) to this processor, otherwise all records will trigger an error.`, Version: "v0.1.0", Author: "Meroxa, Inc.", diff --git a/pkg/plugin/processor/builtin/impl/field/convert.go b/pkg/plugin/processor/builtin/impl/field/convert.go index 86cb00d63..857cc70af 100644 --- a/pkg/plugin/processor/builtin/impl/field/convert.go +++ b/pkg/plugin/processor/builtin/impl/field/convert.go @@ -45,7 +45,7 @@ type convertConfig struct { // Note that you can only convert fields in structured data under `.Key` and // `.Payload`. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" validate:"required,regex=^\\.(Payload|Key).*"` // Type is the target field type after conversion, available options are: `string`, `int`, `float`, `bool`, `time`. Type string `json:"type" validate:"required,inclusion=string|int|float|bool|time"` diff --git a/pkg/plugin/processor/builtin/impl/field/convert_paramgen.go b/pkg/plugin/processor/builtin/impl/field/convert_paramgen.go index 016baae3f..7fa14cc1b 100644 --- a/pkg/plugin/processor/builtin/impl/field/convert_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/field/convert_paramgen.go @@ -18,7 +18,7 @@ func (convertConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ convertConfigField: { Default: "", - Description: "Field is the target field that should be converted.\nNote that you can only convert fields in structured data under `.Key` and\n`.Payload`.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Field is the target field that should be converted.\nNote that you can only convert fields in structured data under `.Key` and\n`.Payload`.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{ config.ValidationRequired{}, diff --git a/pkg/plugin/processor/builtin/impl/field/exclude.go b/pkg/plugin/processor/builtin/impl/field/exclude.go index dff3c7ddb..b72e19648 100644 --- a/pkg/plugin/processor/builtin/impl/field/exclude.go +++ b/pkg/plugin/processor/builtin/impl/field/exclude.go @@ -41,7 +41,7 @@ func NewExcludeProcessor(log.CtxLogger) sdk.Processor { type excludeConfig struct { // Fields is a comma separated list of target fields which should be excluded. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Fields []string `json:"fields" validate:"required"` } diff --git a/pkg/plugin/processor/builtin/impl/field/exclude_paramgen.go b/pkg/plugin/processor/builtin/impl/field/exclude_paramgen.go index 7e46b6d42..810ac8950 100644 --- a/pkg/plugin/processor/builtin/impl/field/exclude_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/field/exclude_paramgen.go @@ -15,7 +15,7 @@ func (excludeConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ excludeConfigFields: { Default: "", - Description: "Fields is a comma separated list of target fields which should be excluded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Fields is a comma separated list of target fields which should be excluded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{ config.ValidationRequired{}, diff --git a/pkg/plugin/processor/builtin/impl/field/rename.go b/pkg/plugin/processor/builtin/impl/field/rename.go index d29b528fb..e0f8e468d 100644 --- a/pkg/plugin/processor/builtin/impl/field/rename.go +++ b/pkg/plugin/processor/builtin/impl/field/rename.go @@ -46,7 +46,7 @@ type renameConfig struct { // // For example: `.Metadata.key:id,.Payload.After.foo:bar`. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Mapping []string `json:"mapping" validate:"required"` } diff --git a/pkg/plugin/processor/builtin/impl/field/rename_paramgen.go b/pkg/plugin/processor/builtin/impl/field/rename_paramgen.go index 0cf1939c3..cf903031a 100644 --- a/pkg/plugin/processor/builtin/impl/field/rename_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/field/rename_paramgen.go @@ -15,7 +15,7 @@ func (renameConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ renameConfigMapping: { Default: "", - Description: "Mapping is a comma separated list of keys and values for fields and their\nnew names (keys and values are separated by colons \":\").\n\nFor example: `.Metadata.key:id,.Payload.After.foo:bar`.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Mapping is a comma separated list of keys and values for fields and their\nnew names (keys and values are separated by colons \":\").\n\nFor example: `.Metadata.key:id,.Payload.After.foo:bar`.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{ config.ValidationRequired{}, diff --git a/pkg/plugin/processor/builtin/impl/field/set.go b/pkg/plugin/processor/builtin/impl/field/set.go index ccdd4a3e2..b6f73e73b 100644 --- a/pkg/plugin/processor/builtin/impl/field/set.go +++ b/pkg/plugin/processor/builtin/impl/field/set.go @@ -44,7 +44,7 @@ type setConfig struct { // Field is the target field that will be set. Note that it is not allowed // to set the `.Position` field. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" validate:"required,exclusion=.Position"` // Value is a Go template expression which will be evaluated and stored in `field` (e.g. `{{ .Payload.After }}`). Value string `json:"value" validate:"required"` diff --git a/pkg/plugin/processor/builtin/impl/field/set_paramgen.go b/pkg/plugin/processor/builtin/impl/field/set_paramgen.go index 0ad1389a5..ef03569e1 100644 --- a/pkg/plugin/processor/builtin/impl/field/set_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/field/set_paramgen.go @@ -16,7 +16,7 @@ func (setConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ setConfigField: { Default: "", - Description: "Field is the target field that will be set. Note that it is not allowed\nto set the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Field is the target field that will be set. Note that it is not allowed\nto set the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{ config.ValidationRequired{}, diff --git a/pkg/plugin/processor/builtin/impl/filter.go b/pkg/plugin/processor/builtin/impl/filter.go index f9b2a3a7d..9e6f0fe9f 100644 --- a/pkg/plugin/processor/builtin/impl/filter.go +++ b/pkg/plugin/processor/builtin/impl/filter.go @@ -40,7 +40,7 @@ func (p *filterProcessor) Specification() (sdk.Specification, error) { the records will be filtered out if the condition provided to the processor is evaluated to ` + "`true`" + `. -**Important:** Make sure to add a [condition](https://conduit.io/docs/processors/conditions) +**Important:** Make sure to add a [condition](https://conduit.io/docs/using/processors/conditions) to this processor, otherwise all records will be filtered out.`, Version: "v0.1.0", Author: "Meroxa, Inc.", diff --git a/pkg/plugin/processor/builtin/impl/json/decode.go b/pkg/plugin/processor/builtin/impl/json/decode.go index 87fd58e39..c46e788cb 100644 --- a/pkg/plugin/processor/builtin/impl/json/decode.go +++ b/pkg/plugin/processor/builtin/impl/json/decode.go @@ -41,7 +41,7 @@ type decodeConfig struct { // Field is a reference to the target field. Only fields that are under // `.Key` and `.Payload` can be decoded. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" validate:"required,regex=^\\.(Payload|Key).*,exclusion=.Payload"` } diff --git a/pkg/plugin/processor/builtin/impl/json/decode_paramgen.go b/pkg/plugin/processor/builtin/impl/json/decode_paramgen.go index eee6b7307..e117a4ac6 100644 --- a/pkg/plugin/processor/builtin/impl/json/decode_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/json/decode_paramgen.go @@ -17,7 +17,7 @@ func (decodeConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ decodeConfigField: { Default: "", - Description: "Field is a reference to the target field. Only fields that are under\n`.Key` and `.Payload` can be decoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Field is a reference to the target field. Only fields that are under\n`.Key` and `.Payload` can be decoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{ config.ValidationRequired{}, diff --git a/pkg/plugin/processor/builtin/impl/json/encode.go b/pkg/plugin/processor/builtin/impl/json/encode.go index e195a172f..233fabdac 100644 --- a/pkg/plugin/processor/builtin/impl/json/encode.go +++ b/pkg/plugin/processor/builtin/impl/json/encode.go @@ -41,7 +41,7 @@ type encodeConfig struct { // Field is a reference to the target field. Only fields that are under // `.Key` and `.Payload` can be encoded. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" validate:"required,regex=^\\.(Payload|Key).*,exclusion=.Payload"` } diff --git a/pkg/plugin/processor/builtin/impl/json/encode_paramgen.go b/pkg/plugin/processor/builtin/impl/json/encode_paramgen.go index ca8430871..ca7177d95 100644 --- a/pkg/plugin/processor/builtin/impl/json/encode_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/json/encode_paramgen.go @@ -17,7 +17,7 @@ func (encodeConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ encodeConfigField: { Default: "", - Description: "Field is a reference to the target field. Only fields that are under\n`.Key` and `.Payload` can be encoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Field is a reference to the target field. Only fields that are under\n`.Key` and `.Payload` can be encoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{ config.ValidationRequired{}, diff --git a/pkg/plugin/processor/builtin/impl/unwrap/debezium.go b/pkg/plugin/processor/builtin/impl/unwrap/debezium.go index 9088d3d6e..65845de34 100644 --- a/pkg/plugin/processor/builtin/impl/unwrap/debezium.go +++ b/pkg/plugin/processor/builtin/impl/unwrap/debezium.go @@ -46,7 +46,7 @@ const ( type debeziumConfig struct { // Field is a reference to the field that contains the Debezium record. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" validate:"regex=^.Payload" default:".Payload.After"` } @@ -64,14 +64,14 @@ func NewDebeziumProcessor(logger log.CtxLogger) sdk.Processor { func (d *debeziumProcessor) Specification() (sdk.Specification, error) { return sdk.Specification{ Name: "unwrap.debezium", - Summary: "Unwraps a Debezium record from the input [OpenCDC record](https://conduit.io/docs/features/opencdc-record).", + Summary: "Unwraps a Debezium record from the input [OpenCDC record](https://conduit.io/docs/using/opencdc-record).", Description: `In this processor, the wrapped (Debezium) record replaces the wrapping record (being processed) completely, except for the position. The Debezium record's metadata and the wrapping record's metadata is merged, with the Debezium metadata having precedence. This is useful in cases where Conduit acts as an intermediary between a Debezium source and a Debezium destination. -In such cases, the Debezium record is set as the [OpenCDC record](https://conduit.io/docs/features/opencdc-record)'s payload,` + +In such cases, the Debezium record is set as the [OpenCDC record](https://conduit.io/docs/using/opencdc-record)'s payload,` + `and needs to be unwrapped for further usage.`, Version: "v0.1.0", Author: "Meroxa, Inc.", diff --git a/pkg/plugin/processor/builtin/impl/unwrap/debezium_paramgen.go b/pkg/plugin/processor/builtin/impl/unwrap/debezium_paramgen.go index 29ad20850..8e7ad635d 100644 --- a/pkg/plugin/processor/builtin/impl/unwrap/debezium_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/unwrap/debezium_paramgen.go @@ -17,7 +17,7 @@ func (debeziumConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ debeziumConfigField: { Default: ".Payload.After", - Description: "Field is a reference to the field that contains the Debezium record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Field is a reference to the field that contains the Debezium record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{ config.ValidationRegex{Regex: regexp.MustCompile("^.Payload")}, diff --git a/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect.go b/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect.go index 792466c09..82a5a0ba5 100644 --- a/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect.go +++ b/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect.go @@ -31,7 +31,7 @@ import ( type kafkaConnectConfig struct { // Field is a reference to the field that contains the Kafka Connect record. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" validate:"regex=^.Payload" default:".Payload.After"` } @@ -48,13 +48,13 @@ func NewKafkaConnectProcessor(log.CtxLogger) sdk.Processor { func (u *kafkaConnectProcessor) Specification() (sdk.Specification, error) { return sdk.Specification{ Name: "unwrap.kafkaconnect", - Summary: "Unwraps a Kafka Connect record from an [OpenCDC record](https://conduit.io/docs/features/opencdc-record).", - Description: `This processor unwraps a Kafka Connect record from the input [OpenCDC record](https://conduit.io/docs/features/opencdc-record). + Summary: "Unwraps a Kafka Connect record from an [OpenCDC record](https://conduit.io/docs/using/opencdc-record).", + Description: `This processor unwraps a Kafka Connect record from the input [OpenCDC record](https://conduit.io/docs/using/opencdc-record). The input record's payload is replaced with the Kafka Connect record. This is useful in cases where Conduit acts as an intermediary between a Debezium source and a Debezium destination. -In such cases, the Debezium record is set as the [OpenCDC record](https://conduit.io/docs/features/opencdc-record)'s payload, and needs to be unwrapped for further usage.`, +In such cases, the Debezium record is set as the [OpenCDC record](https://conduit.io/docs/using/opencdc-record)'s payload, and needs to be unwrapped for further usage.`, Version: "v0.1.0", Author: "Meroxa, Inc.", Parameters: kafkaConnectConfig{}.Parameters(), diff --git a/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect_examples_test.go b/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect_examples_test.go index 48dd34114..f7dd1e330 100644 --- a/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect_examples_test.go +++ b/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect_examples_test.go @@ -31,7 +31,7 @@ func ExampleKafkaConnectProcesor() { Description: `This example shows how to unwrap a Kafka Connect record. The Kafka Connect record is serialized as a JSON string in the ` + "`.Payload.After`" + ` field (raw data). -The Kafka Connect record's payload will replace the [OpenCDC record](https://conduit.io/docs/features/opencdc-record)'s payload. +The Kafka Connect record's payload will replace the [OpenCDC record](https://conduit.io/docs/using/opencdc-record)'s payload. We also see how the key is unwrapped too. In this case, the key comes in as structured data.`, Config: config.Config{}, diff --git a/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect_paramgen.go b/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect_paramgen.go index 223fdec00..d88670081 100644 --- a/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/unwrap/kafka_connect_paramgen.go @@ -17,7 +17,7 @@ func (kafkaConnectConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ kafkaConnectConfigField: { Default: ".Payload.After", - Description: "Field is a reference to the field that contains the Kafka Connect record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Field is a reference to the field that contains the Kafka Connect record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{ config.ValidationRegex{Regex: regexp.MustCompile("^.Payload")}, diff --git a/pkg/plugin/processor/builtin/impl/unwrap/opencdc.go b/pkg/plugin/processor/builtin/impl/unwrap/opencdc.go index 64bd3226d..5a26599a0 100644 --- a/pkg/plugin/processor/builtin/impl/unwrap/opencdc.go +++ b/pkg/plugin/processor/builtin/impl/unwrap/opencdc.go @@ -32,7 +32,7 @@ import ( type openCDCConfig struct { // Field is a reference to the field that contains the OpenCDC record. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). Field string `json:"field" default:".Payload.After"` } @@ -50,12 +50,12 @@ func NewOpenCDCProcessor(logger log.CtxLogger) sdk.Processor { func (u *openCDCProcessor) Specification() (sdk.Specification, error) { return sdk.Specification{ Name: "unwrap.opencdc", - Summary: "Unwraps an [OpenCDC record](https://conduit.io/docs/features/opencdc-record) saved in one of the record's fields.", + Summary: "Unwraps an [OpenCDC record](https://conduit.io/docs/using/opencdc-record) saved in one of the record's fields.", Description: `The ` + "`unwrap.opencdc`" + ` processor is useful in situations where a record goes through intermediate -systems before being written to a final destination. In these cases, the original [OpenCDC record](https://conduit.io/docs/features/opencdc-record) is part of the payload +systems before being written to a final destination. In these cases, the original [OpenCDC record](https://conduit.io/docs/using/opencdc-record) is part of the payload read from the intermediate system and needs to be unwrapped before being written. -Note: if the wrapped [OpenCDC record](https://conduit.io/docs/features/opencdc-record) is not in a structured data field, then it's assumed that it's stored in JSON format.`, +Note: if the wrapped [OpenCDC record](https://conduit.io/docs/using/opencdc-record) is not in a structured data field, then it's assumed that it's stored in JSON format.`, Version: "v0.1.0", Author: "Meroxa, Inc.", Parameters: openCDCConfig{}.Parameters(), diff --git a/pkg/plugin/processor/builtin/impl/unwrap/opencdc_examples_test.go b/pkg/plugin/processor/builtin/impl/unwrap/opencdc_examples_test.go index d055512b3..813194163 100644 --- a/pkg/plugin/processor/builtin/impl/unwrap/opencdc_examples_test.go +++ b/pkg/plugin/processor/builtin/impl/unwrap/opencdc_examples_test.go @@ -27,8 +27,8 @@ func ExampleOpenCDCProcessor() { p := NewOpenCDCProcessor(log.Nop()) exampleutil.RunExample(p, exampleutil.Example{ - Summary: "Unwrap an [OpenCDC record](https://conduit.io/docs/features/opencdc-record)", - Description: "In this example we use the `unwrap.opencdc` processor to unwrap the [OpenCDC record](https://conduit.io/docs/features/opencdc-record) found in the " + + Summary: "Unwrap an [OpenCDC record](https://conduit.io/docs/using/opencdc-record)", + Description: "In this example we use the `unwrap.opencdc` processor to unwrap the [OpenCDC record](https://conduit.io/docs/using/opencdc-record) found in the " + "record's `.Payload.After` field.", Config: config.Config{}, Have: opencdc.Record{ diff --git a/pkg/plugin/processor/builtin/impl/unwrap/opencdc_paramgen.go b/pkg/plugin/processor/builtin/impl/unwrap/opencdc_paramgen.go index 1ba2868e9..4d297c7d3 100644 --- a/pkg/plugin/processor/builtin/impl/unwrap/opencdc_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/unwrap/opencdc_paramgen.go @@ -15,7 +15,7 @@ func (openCDCConfig) Parameters() map[string]config.Parameter { return map[string]config.Parameter{ openCDCConfigField: { Default: ".Payload.After", - Description: "Field is a reference to the field that contains the OpenCDC record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Field is a reference to the field that contains the OpenCDC record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{}, }, diff --git a/pkg/plugin/processor/builtin/impl/webhook/http.go b/pkg/plugin/processor/builtin/impl/webhook/http.go index 5a82ec7ba..ef57a6b9d 100644 --- a/pkg/plugin/processor/builtin/impl/webhook/http.go +++ b/pkg/plugin/processor/builtin/impl/webhook/http.go @@ -67,12 +67,12 @@ type httpConfig struct { RequestBodyTmpl string `json:"request.body"` // Specifies in which field should the response body be saved. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). ResponseBodyRef string `json:"response.body" default:".Payload.After"` // Specifies in which field should the response status be saved. If no value // is set, then the response status will NOT be saved. // - // For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields). + // For more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields). ResponseStatusRef string `json:"response.status"` } diff --git a/pkg/plugin/processor/builtin/impl/webhook/http_paramgen.go b/pkg/plugin/processor/builtin/impl/webhook/http_paramgen.go index b2644c0cf..8c64545b0 100644 --- a/pkg/plugin/processor/builtin/impl/webhook/http_paramgen.go +++ b/pkg/plugin/processor/builtin/impl/webhook/http_paramgen.go @@ -85,13 +85,13 @@ func (httpConfig) Parameters() map[string]config.Parameter { }, httpConfigResponseBody: { Default: ".Payload.After", - Description: "Specifies in which field should the response body be saved.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Specifies in which field should the response body be saved.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{}, }, httpConfigResponseStatus: { Default: "", - Description: "Specifies in which field should the response status be saved. If no value\nis set, then the response status will NOT be saved.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + Description: "Specifies in which field should the response status be saved. If no value\nis set, then the response status will NOT be saved.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", Type: config.ParameterTypeString, Validations: []config.Validation{}, }, diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/avro.decode.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/avro.decode.json index 808375ccd..5ef8b7988 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/avro.decode.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/avro.decode.json @@ -8,7 +8,7 @@ "parameters": { "field": { "default": ".Payload.After", - "description": "The field that will be decoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "The field that will be decoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [] }, diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/avro.encode.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/avro.encode.json index 17d00f6c8..395169326 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/avro.encode.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/avro.encode.json @@ -8,7 +8,7 @@ "parameters": { "field": { "default": ".Payload.After", - "description": "The field that will be encoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "The field that will be encoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [] }, diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/base64.decode.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/base64.decode.json index 1d3db270e..96847434a 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/base64.decode.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/base64.decode.json @@ -8,7 +8,7 @@ "parameters": { "field": { "default": "", - "description": "Field is the reference to the target field. Note that it is not allowed to\nbase64 decode the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Field is the reference to the target field. Note that it is not allowed to\nbase64 decode the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [ { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/base64.encode.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/base64.encode.json index ccc590f5f..7e7870537 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/base64.encode.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/base64.encode.json @@ -8,7 +8,7 @@ "parameters": { "field": { "default": "", - "description": "Field is a reference to the target field. Note that it is not allowed to\nbase64 encode the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Field is a reference to the target field. Note that it is not allowed to\nbase64 encode the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [ { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/error.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/error.json index 9fcab1a97..436b7f7a5 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/error.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/error.json @@ -2,7 +2,7 @@ "specification": { "name": "error", "summary": "Returns an error for all records that get passed to the processor.", - "description": "Any time a record is passed to this processor it returns an error,\nwhich results in the record being sent to the DLQ if it's configured, or the pipeline stopping.\n\n**Important:** Make sure to add a [condition](https://conduit.io/docs/processors/conditions)\nto this processor, otherwise all records will trigger an error.", + "description": "Any time a record is passed to this processor it returns an error,\nwhich results in the record being sent to the DLQ if it's configured, or the pipeline stopping.\n\n**Important:** Make sure to add a [condition](https://conduit.io/docs/using/processors/conditions)\nto this processor, otherwise all records will trigger an error.", "version": "v0.1.0", "author": "Meroxa, Inc.", "parameters": { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.convert.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.convert.json index ed2befcef..312aef481 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.convert.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.convert.json @@ -8,7 +8,7 @@ "parameters": { "field": { "default": "", - "description": "Field is the target field that should be converted.\nNote that you can only convert fields in structured data under `.Key` and\n`.Payload`.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Field is the target field that should be converted.\nNote that you can only convert fields in structured data under `.Key` and\n`.Payload`.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [ { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.exclude.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.exclude.json index a604b6b63..1d75d352d 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.exclude.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.exclude.json @@ -8,7 +8,7 @@ "parameters": { "fields": { "default": "", - "description": "Fields is a comma separated list of target fields which should be excluded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Fields is a comma separated list of target fields which should be excluded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [ { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.rename.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.rename.json index c6f28cc56..8da619965 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.rename.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.rename.json @@ -8,7 +8,7 @@ "parameters": { "mapping": { "default": "", - "description": "Mapping is a comma separated list of keys and values for fields and their\nnew names (keys and values are separated by colons \":\").\n\nFor example: `.Metadata.key:id,.Payload.After.foo:bar`.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Mapping is a comma separated list of keys and values for fields and their\nnew names (keys and values are separated by colons \":\").\n\nFor example: `.Metadata.key:id,.Payload.After.foo:bar`.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [ { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.set.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.set.json index eed55f2db..a0c397881 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.set.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/field.set.json @@ -8,7 +8,7 @@ "parameters": { "field": { "default": "", - "description": "Field is the target field that will be set. Note that it is not allowed\nto set the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Field is the target field that will be set. Note that it is not allowed\nto set the `.Position` field.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [ { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/filter.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/filter.json index 72f9e179c..297f7ad1a 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/filter.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/filter.json @@ -2,7 +2,7 @@ "specification": { "name": "filter", "summary": "Acknowledges all records that get passed to the filter.", - "description": "Acknowledges all records that get passed to the filter, so\nthe records will be filtered out if the condition provided to the processor is\nevaluated to `true`.\n\n**Important:** Make sure to add a [condition](https://conduit.io/docs/processors/conditions)\nto this processor, otherwise all records will be filtered out.", + "description": "Acknowledges all records that get passed to the filter, so\nthe records will be filtered out if the condition provided to the processor is\nevaluated to `true`.\n\n**Important:** Make sure to add a [condition](https://conduit.io/docs/using/processors/conditions)\nto this processor, otherwise all records will be filtered out.", "version": "v0.1.0", "author": "Meroxa, Inc.", "parameters": { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/json.decode.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/json.decode.json index 36f1bce35..98b69f95c 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/json.decode.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/json.decode.json @@ -8,7 +8,7 @@ "parameters": { "field": { "default": "", - "description": "Field is a reference to the target field. Only fields that are under\n`.Key` and `.Payload` can be decoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Field is a reference to the target field. Only fields that are under\n`.Key` and `.Payload` can be decoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [ { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/json.encode.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/json.encode.json index a2705a375..976632a46 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/json.encode.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/json.encode.json @@ -8,7 +8,7 @@ "parameters": { "field": { "default": "", - "description": "Field is a reference to the target field. Only fields that are under\n`.Key` and `.Payload` can be encoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Field is a reference to the target field. Only fields that are under\n`.Key` and `.Payload` can be encoded.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [ { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.debezium.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.debezium.json index 331c75916..96dd5c178 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.debezium.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.debezium.json @@ -1,14 +1,14 @@ { "specification": { "name": "unwrap.debezium", - "summary": "Unwraps a Debezium record from the input [OpenCDC record](https://conduit.io/docs/features/opencdc-record).", - "description": "In this processor, the wrapped (Debezium) record replaces the wrapping record (being processed) \ncompletely, except for the position.\n\nThe Debezium record's metadata and the wrapping record's metadata is merged, with the Debezium metadata having precedence.\n\nThis is useful in cases where Conduit acts as an intermediary between a Debezium source and a Debezium destination. \nIn such cases, the Debezium record is set as the [OpenCDC record](https://conduit.io/docs/features/opencdc-record)'s payload,and needs to be unwrapped for further usage.", + "summary": "Unwraps a Debezium record from the input [OpenCDC record](https://conduit.io/docs/using/opencdc-record).", + "description": "In this processor, the wrapped (Debezium) record replaces the wrapping record (being processed) \ncompletely, except for the position.\n\nThe Debezium record's metadata and the wrapping record's metadata is merged, with the Debezium metadata having precedence.\n\nThis is useful in cases where Conduit acts as an intermediary between a Debezium source and a Debezium destination. \nIn such cases, the Debezium record is set as the [OpenCDC record](https://conduit.io/docs/using/opencdc-record)'s payload,and needs to be unwrapped for further usage.", "version": "v0.1.0", "author": "Meroxa, Inc.", "parameters": { "field": { "default": ".Payload.After", - "description": "Field is a reference to the field that contains the Debezium record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Field is a reference to the field that contains the Debezium record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [ { diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.kafkaconnect.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.kafkaconnect.json index 87afb8700..703925037 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.kafkaconnect.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.kafkaconnect.json @@ -1,14 +1,14 @@ { "specification": { "name": "unwrap.kafkaconnect", - "summary": "Unwraps a Kafka Connect record from an [OpenCDC record](https://conduit.io/docs/features/opencdc-record).", - "description": "This processor unwraps a Kafka Connect record from the input [OpenCDC record](https://conduit.io/docs/features/opencdc-record).\n\nThe input record's payload is replaced with the Kafka Connect record.\n\nThis is useful in cases where Conduit acts as an intermediary between a Debezium source and a Debezium destination. \nIn such cases, the Debezium record is set as the [OpenCDC record](https://conduit.io/docs/features/opencdc-record)'s payload, and needs to be unwrapped for further usage.", + "summary": "Unwraps a Kafka Connect record from an [OpenCDC record](https://conduit.io/docs/using/opencdc-record).", + "description": "This processor unwraps a Kafka Connect record from the input [OpenCDC record](https://conduit.io/docs/using/opencdc-record).\n\nThe input record's payload is replaced with the Kafka Connect record.\n\nThis is useful in cases where Conduit acts as an intermediary between a Debezium source and a Debezium destination. \nIn such cases, the Debezium record is set as the [OpenCDC record](https://conduit.io/docs/using/opencdc-record)'s payload, and needs to be unwrapped for further usage.", "version": "v0.1.0", "author": "Meroxa, Inc.", "parameters": { "field": { "default": ".Payload.After", - "description": "Field is a reference to the field that contains the Kafka Connect record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Field is a reference to the field that contains the Kafka Connect record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [ { @@ -46,7 +46,7 @@ "examples": [ { "summary": "Unwrap a Kafka Connect record", - "description": "This example shows how to unwrap a Kafka Connect record.\n\nThe Kafka Connect record is serialized as a JSON string in the `.Payload.After` field (raw data).\nThe Kafka Connect record's payload will replace the [OpenCDC record](https://conduit.io/docs/features/opencdc-record)'s payload.\n\nWe also see how the key is unwrapped too. In this case, the key comes in as structured data.", + "description": "This example shows how to unwrap a Kafka Connect record.\n\nThe Kafka Connect record is serialized as a JSON string in the `.Payload.After` field (raw data).\nThe Kafka Connect record's payload will replace the [OpenCDC record](https://conduit.io/docs/using/opencdc-record)'s payload.\n\nWe also see how the key is unwrapped too. In this case, the key comes in as structured data.", "config": { "field": ".Payload.After" }, diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.opencdc.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.opencdc.json index ee63ffe25..f348804c1 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.opencdc.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/unwrap.opencdc.json @@ -1,14 +1,14 @@ { "specification": { "name": "unwrap.opencdc", - "summary": "Unwraps an [OpenCDC record](https://conduit.io/docs/features/opencdc-record) saved in one of the record's fields.", - "description": "The `unwrap.opencdc` processor is useful in situations where a record goes through intermediate\nsystems before being written to a final destination. In these cases, the original [OpenCDC record](https://conduit.io/docs/features/opencdc-record) is part of the payload\nread from the intermediate system and needs to be unwrapped before being written.\n\nNote: if the wrapped [OpenCDC record](https://conduit.io/docs/features/opencdc-record) is not in a structured data field, then it's assumed that it's stored in JSON format.", + "summary": "Unwraps an [OpenCDC record](https://conduit.io/docs/using/opencdc-record) saved in one of the record's fields.", + "description": "The `unwrap.opencdc` processor is useful in situations where a record goes through intermediate\nsystems before being written to a final destination. In these cases, the original [OpenCDC record](https://conduit.io/docs/using/opencdc-record) is part of the payload\nread from the intermediate system and needs to be unwrapped before being written.\n\nNote: if the wrapped [OpenCDC record](https://conduit.io/docs/using/opencdc-record) is not in a structured data field, then it's assumed that it's stored in JSON format.", "version": "v0.1.0", "author": "Meroxa, Inc.", "parameters": { "field": { "default": ".Payload.After", - "description": "Field is a reference to the field that contains the OpenCDC record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Field is a reference to the field that contains the OpenCDC record.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [] }, @@ -40,8 +40,8 @@ }, "examples": [ { - "summary": "Unwrap an [OpenCDC record](https://conduit.io/docs/features/opencdc-record)", - "description": "In this example we use the `unwrap.opencdc` processor to unwrap the [OpenCDC record](https://conduit.io/docs/features/opencdc-record) found in the record's `.Payload.After` field.", + "summary": "Unwrap an [OpenCDC record](https://conduit.io/docs/using/opencdc-record)", + "description": "In this example we use the `unwrap.opencdc` processor to unwrap the [OpenCDC record](https://conduit.io/docs/using/opencdc-record) found in the record's `.Payload.After` field.", "config": { "field": ".Payload.After" }, diff --git a/pkg/plugin/processor/builtin/internal/exampleutil/specs/webhook.http.json b/pkg/plugin/processor/builtin/internal/exampleutil/specs/webhook.http.json index 6a167a239..3621efea4 100644 --- a/pkg/plugin/processor/builtin/internal/exampleutil/specs/webhook.http.json +++ b/pkg/plugin/processor/builtin/internal/exampleutil/specs/webhook.http.json @@ -77,13 +77,13 @@ }, "response.body": { "default": ".Payload.After", - "description": "Specifies in which field should the response body be saved.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Specifies in which field should the response body be saved.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [] }, "response.status": { "default": "", - "description": "Specifies in which field should the response status be saved. If no value\nis set, then the response status will NOT be saved.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).", + "description": "Specifies in which field should the response status be saved. If no value\nis set, then the response status will NOT be saved.\n\nFor more information about the format, see [Referencing fields](https://conduit.io/docs/using/processors/referencing-fields).", "type": "string", "validations": [] }, diff --git a/pkg/web/openapi/swagger-ui/api/v1/api.swagger.json b/pkg/web/openapi/swagger-ui/api/v1/api.swagger.json index 128a2ddb3..cc23fbab6 100644 --- a/pkg/web/openapi/swagger-ui/api/v1/api.swagger.json +++ b/pkg/web/openapi/swagger-ui/api/v1/api.swagger.json @@ -1907,7 +1907,7 @@ }, "plugin": { "type": "string", - "title": "Used to reference a plugin. Its format is as follows:\n[PLUGIN-TYPE:]PLUGIN-NAME[@VERSION]\nPLUGIN-TYPE: One of: builtin, standalone or any (default).\nPLUGIN-NAME: The name of the plugin as specified in the plugin specifications.\nVERSION: The plugin version as specified in the plugin specifications or latest (default).\nFor more information, see: https://conduit.io/docs/connectors/referencing/" + "title": "Used to reference a plugin. Its format is as follows:\n[PLUGIN-TYPE:]PLUGIN-NAME[@VERSION]\nPLUGIN-TYPE: One of: builtin, standalone or any (default).\nPLUGIN-NAME: The name of the plugin as specified in the plugin specifications.\nVERSION: The plugin version as specified in the plugin specifications or latest (default).\nFor more information, see: https://conduit.io/docs/using/connectors/referencing" }, "pipelineId": { "type": "string", diff --git a/proto/api/v1/api.pb.go b/proto/api/v1/api.pb.go index 6764febaf..4caaac7f4 100644 --- a/proto/api/v1/api.pb.go +++ b/proto/api/v1/api.pb.go @@ -2019,7 +2019,7 @@ type CreateConnectorRequest struct { // PLUGIN-TYPE: One of: builtin, standalone or any (default). // PLUGIN-NAME: The name of the plugin as specified in the plugin specifications. // VERSION: The plugin version as specified in the plugin specifications or latest (default). - // For more information, see: https://conduit.io/docs/connectors/referencing/ + // For more information, see: https://conduit.io/docs/using/connectors/referencing Plugin string `protobuf:"bytes,2,opt,name=plugin,proto3" json:"plugin,omitempty"` // ID of the pipeline to which the connector will get attached. PipelineId string `protobuf:"bytes,3,opt,name=pipeline_id,json=pipelineId,proto3" json:"pipeline_id,omitempty"` diff --git a/proto/api/v1/api.proto b/proto/api/v1/api.proto index a04196c7e..5361653dc 100644 --- a/proto/api/v1/api.proto +++ b/proto/api/v1/api.proto @@ -775,7 +775,7 @@ message CreateConnectorRequest { // PLUGIN-TYPE: One of: builtin, standalone or any (default). // PLUGIN-NAME: The name of the plugin as specified in the plugin specifications. // VERSION: The plugin version as specified in the plugin specifications or latest (default). - // For more information, see: https://conduit.io/docs/connectors/referencing/ + // For more information, see: https://conduit.io/docs/using/connectors/referencing string plugin = 2; // ID of the pipeline to which the connector will get attached. string pipeline_id = 3; diff --git a/proto/api/v1/api.swagger.json b/proto/api/v1/api.swagger.json index 128a2ddb3..cc23fbab6 100644 --- a/proto/api/v1/api.swagger.json +++ b/proto/api/v1/api.swagger.json @@ -1907,7 +1907,7 @@ }, "plugin": { "type": "string", - "title": "Used to reference a plugin. Its format is as follows:\n[PLUGIN-TYPE:]PLUGIN-NAME[@VERSION]\nPLUGIN-TYPE: One of: builtin, standalone or any (default).\nPLUGIN-NAME: The name of the plugin as specified in the plugin specifications.\nVERSION: The plugin version as specified in the plugin specifications or latest (default).\nFor more information, see: https://conduit.io/docs/connectors/referencing/" + "title": "Used to reference a plugin. Its format is as follows:\n[PLUGIN-TYPE:]PLUGIN-NAME[@VERSION]\nPLUGIN-TYPE: One of: builtin, standalone or any (default).\nPLUGIN-NAME: The name of the plugin as specified in the plugin specifications.\nVERSION: The plugin version as specified in the plugin specifications or latest (default).\nFor more information, see: https://conduit.io/docs/using/connectors/referencing" }, "pipelineId": { "type": "string",