doc: add conduit operator

changelog/2024-10-17-introducing-conduit-operator.md

@@ -0,0 +1,18 @@
+---
+slug: '2024-10-17-conduit-operator'
+title: Introducing Conduit Operator
+draft: false
+tags: [conduit, operator, scale, kubernetes]
+---
+
+We're happy to announce that we recently released [Conduit Operator v0.0.1](https://github.com/ConduitIO/conduit-operator/releases/tag/v0.0.1), a Kubernetes operator that extends the Kubernetes API to enhance the management of Conduit instances on our own [Conduit Platform](https://meroxa.io).
+
+<!--truncate-->
+
+Using this operator, we will be able to significantly reduce the time needed to deploy new connectors, along with many other features.
+
+For **more information**, check out [its documentation page](/docs/scaling/conduit-operator).
+
+:::info
+Conduit Operator is open source and available on [GitHub](https://github.com/ConduitIO/conduit-operator).
+:::

changelog/2024-10-23-conduit-operator-with-schema-support.md

@@ -0,0 +1,16 @@
+---
+slug: '2024-10-23-conduit-operator-with-schema-support'
+title: Conduit Operator with Schema Support
+draft: false
+tags: [conduit, operator, scale, kubernetes]
+---
+
+After the recent announcement of [Conduit Operator v0.0.1](/changelog/2024-10-17-conduit-operator), we're happy to share that [Conduit Operator v0.0.2](https://github.com/ConduitIO/conduit-operator/releases/tag/v0.0.2) is coming with support for a Schema Registry.
+
+<!--truncate-->
+
+For **more information** on how to use it, check out the [Schema Support section](/docs/scaling/conduit-operator#schema-support).
+
+:::info
+Conduit Operator is open source and available on [GitHub](https://github.com/ConduitIO/conduit-operator).
+:::

docs/3-scaling/0-conduit-operator.mdx

@@ -0,0 +1,209 @@
+---
+title: "Conduit Operator"
+toc_max_heading_level: 6
+---
+
+The Conduit Operator is a [Kubernetes operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) designed to simplify the management and orchestration of Conduit instances. The operator extends the [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/) with a [custom resource](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) for Conduit. 
+
+This custom resource allows users to define and manage Conduit instances declaratively, enabling seamless integration with Kubernetes' native features such as scaling, monitoring, and logging.
+
+### Overview
+
+Conduit pipelines are represented as Conduit custom resources, where each pipeline will be provisioned 
+as a distinct Conduit instance with its own lifecycle. 
+
+The Conduit custom resource definition format is very similar to that of [pipeline configurations](/docs/using/pipelines/configuration-file):
+
+```yaml
+apiVersion: operator.conduit.io/v1alpha
+kind: Conduit
+metadata:
+  name: conduit-generator
+spec:
+  running: true
+  name: generator.log
+  description: generator pipeline
+  connectors:
+    - name: source-connector
+      type: source
+      plugin: builtin:generator
+      settings:
+        - name: format.type
+          value: structured
+        - name: format.options.id
+          value: "int"
+        - name: format.options.name
+          value: "string"
+        - name: format.options.company
+          value: "string"
+        - name: format.options.trial
+          value: "bool"
+        - name: recordCount
+          value: "3"
+    - name: destination-connector
+      type: destination
+      plugin: builtin:log
+
+```
+
+### Quickstart
+
+:::info
+To get started with the Conduit Operator, follow the steps described on the [Conduit Operator GitHub repository](https://github.com/ConduitIO/conduit-operator?tab=readme-ov-file#quickstart).
+:::
+
+### Installing standalone connectors
+
+#### `plugin` and `pluginVersion`
+
+A standalone connector can be installed by providing the connector name via `plugin`, and version via `pluginVersion`. 
+
+The operator will then install the connector and create the necessary resources to run it. These connectors need to be referred to by their GitHub `organization/repository` names and are currently limited to the [following organizations](https://github.com/ConduitIO/conduit-operator/blob/7321042d1fa1205f3c1e5faf70775184ebafedba/pkg/conduit/plugin.go#L9-L11):
+
+- [conduitio](https://github.com/conduitio)
+- [conduitio-labs](https://github.com/conduitio-labs)
+- [meroxa](https://github.com/meroxa)
+
+
+:::tip
+Plugin version can optionally be specified or the latest will be used. 
+:::
+
+#### Example
+
+Here's a full example using [`conduitio/conduit-connector-generator`](https://github.com/conduitio/conduit-connector-generator) as `plugin` and `v0.8.0` as `pluginVersion`:
+
+```yaml
+apiVersion: operator.conduit.io/v1alpha
+kind: Conduit
+metadata:
+  name: conduit-generator
+spec:
+  running: true
+  name: generator.log
+  description: generator pipeline
+  connectors:
+    - name: source-connector
+      type: source
+      plugin: conduitio/conduit-connector-generator
+      pluginVersion: v0.8.0
+      settings:
+        - name: format.type
+          value: structured
+        - name: format.options.id
+          value: "int"
+        - name: format.options.name
+          value: "string"
+        - name: format.options.company
+          value: "string"
+        - name: format.options.trial
+          value: "bool"
+        - name: recordCount
+          value: "3"
+    - name: destination-connector
+      type: destination
+      plugin: conduitio/conduit-connector-log
+```
+
+### Schema Support
+
+As of [v0.11.0](/changelog/2024-08-19-conduit-0-11-0-release) Conduit supports the use of a schema registry. 
+
+This allows connectors to automatically extract or use the schema referred to by the [OpenCDC record](/docs/using/opencdc-record) to encode/decode data. 
+
+By default Conduit uses a builtin schema registry, however in certain use cases a schema registry needs to be shared between multiple instances. The Conduit resource allows for schema registry to be defined as of [v0.2.0](https://github.com/ConduitIO/conduit-operator/releases/tag/v0.0.2).
+
+Details are provided directly via the Conduit resource. `basicAuthUser` and `basicAuthPassword` can be provided as secrets but will be copied over into controller-owned secrets for management and ease of use.
+
+#### Example
+
+```yaml
+apiVersion: operator.conduit.io/v1alpha
+kind: Conduit
+metadata:
+  name: conduit-generator-schema-registry
+spec:
+  running: true
+  name: generator.standalone.log
+  description: generator pipeline
+  schemaRegistry:
+    url: http://apicurio:8080/apis/ccompat/v7
+    # basicAuthUser:
+    #   - value: <schemaUser>
+    # basicAuthPassword:
+    #   - secretRef:
+    #     key: schema-registry-password
+    #     name: schema-registry-secret
+  connectors:
+    - name: source-connector
+      type: source
+      plugin: conduitio/conduit-connector-generator
+      settings:
+        - name: format.type
+          value: structured
+        - name: format.options.id
+          value: "int"
+        - name: format.options.name
+          value: "string"
+        - name: format.options.company
+          value: "string"
+        - name: format.options.trial
+          value: "bool"
+        - name: recordCount
+          value: "3"
+    - name: destination-connector
+      type: destination
+      plugin: conduitio/conduit-connector-log
+      pluginVersion: v0.4.0
+```
+
+### Deploying
+
+The operator provides a [helm chart](https://github.com/ConduitIO/conduit-operator/blob/main/charts/conduit-operator) that can be used for deployment on any cluster. Additional metadata can be injected into each provisioned conduit instance via the `controller.conduitMetadata` configuration.
+
+For example, to instruct a prometheus instance to scrape each Conduit instance metrics:
+
+```yaml
+controller:
+  conduitMetadata:
+    podAnnotations:
+      prometheus.io/scrape: true
+      prometheus.io/path: /metrics
+      prometheus.io/port: 8080
+```
+
+For more configuration options see [charts/conduit-operator/values.yaml](https://github.com/ConduitIO/conduit-operator/blob/main/charts/conduit-operator/values.yaml).
+
+Additional metadata can be injected in each provisioned conduit instance via `controller.conduitMetadata` configuration. 
+
+For example, to instruct a prometheus instance to scrape each Conduit instance metrics:
+
+```yaml
+controller:
+  conduitMetadata:
+    podAnnotations:
+      prometheus.io/scrape: true
+      prometheus.io/path: /metrics
+      prometheus.io/port: 8080
+```
+
+For more configuration options see [charts/conduit-operator/values.yaml](https://github.com/ConduitIO/conduit-operator/blob/main/charts/conduit-operator/values.yaml).
+
+### Using the helm chart repository
+
+Alternatively the operator can be deployed via the [Helm repository](https://helm.sh/docs/topics/chart_repository/). To add the repository to your helm repos:
+
+```shell
+helm repo add conduit https://conduitio.github.io/conduit-operator
+```
+
+Install the operator in the `conduit-operator` namespace of your cluster:
+
+```shell
+helm install conduit-operator \
+    conduit/conduit-operator --create-namespace -n conduit-operator
+```
+
+:::tip
+For a low-code experience, and many other Enterprise features, we recommend you use the [Conduit Platform](https://meroxa.io).
+:::

src/components/HomepageNavbar/index.tsx

@@ -11,6 +11,10 @@ const navLinks = [
     label: 'Documentation',
     href: '/docs',
   },
+  {
+    label: 'Changelog',
+    href: '/changelog',
+  },
   {
     label: 'Conduit Platform',
     href: 'https://meroxa.io',

src/sidebars/sidebars.ts

@@ -23,9 +23,8 @@ const sidebars: SidebarsConfig = {
     {type: 'autogenerated', dirName: '2-developing', customProps: { slugBase: 'foo' }},
 
     {type: 'html', value: '<li class="sidebar-header">Scaling Conduit</li>'},
-    {type: "link",label: "Conduit Platform",href: "https://meroxa.io"},
-    {type: "link",label: "Conduit Operator",href: "https://github.com/ConduitIO/conduit-operator"},
     {type: 'autogenerated', dirName: '3-scaling'},
+    {type: "link",label: "Conduit Platform",href: "https://meroxa.io"},
 
     {type: 'html', value: '<li class="sidebar-header">Community</li>'},
     {type: "link", label: "Discord Community",href: "https://discord.meroxa.com"},