InfinyOn Labs is a collection of projects that helps the community accelerate the pace of innovation through experimentation. The projects here use various components, some of which may be custom connectors and smartmodules that InfinyOn does not guarantee to run in production environments.
Smartmodules and Connectors referenced in this repo must have two components:
- binary published in
InfinyOn Hub - public repo in
github
As a member of the Fluvio community, you are encouraged to contribute to these projects or submit new ones.
The InfinyOn team is committed to working with the community and turning the most popular labs projects into certified modules. Certified modules are integrated into the InfinyOn CI/CD pipeline and are safe for production workloads.
Vote Here if you want you want a project to be promoted to certified, and we'll prioritize accordingly.
Configuration files that provision data pipelines using a combination of connectors and smartmodules. The projects are defined by configuration file operated via DSL, and should not require coding.
| Pipeline Project | Description |
|---|---|
| hackernews-reader | Reads Hackernews XML feed and converts each article into a JON record |
| webhook-to-slack | Convert events receives from the InfinyOn webhook API and notify on Slack |
| github-to-slack | Periodically checks a github repo for changes in starts/forks, and notifies on Slack |
| github-to-discord | Checks a github repo for starts/forks, and notifies on Discord |
Smartmodule projects built by labs. You may clone, and enhance:
| Smartmodule Project | Record Type | Description |
|---|---|---|
| labs-rss-json-sm | xml | Parses RSS XML input into JSON format |
| labs-json-formatter-sm | json | Generated a formatted string from JSON values |
| labs-key-gen-json-sm | json | Generates a unique key (digest) from JSON values |
| labs-array-map-json-sm | json | Splits an JSON array into individual records |
| labs-regex-map-json-sm | json | Applies Regex transformations on JSON values |
| labs-stars-forks-changes-sm | json | Detects changes in github stars & forks, and generates an emoji string |
| labs-regex-map-sm | text | Applies Regex transformations on arbitrary text |
| labs-cvs-json-sm | csv | Turns a CVS file into an array of json records |
For additional examples, checkout fluvio/smartmodules.
Connectors in development as incubated labs projects:
| Connector Project | Status | Description |
|---|---|---|
| labs-redis-sink-connector | experimental | Fluvio to Redis |
| labs-s3-sink-connector | roadmap | Fluvio to S3 |
Note, all certified connectors are available for use in labs projects:
| Connector Project | Status | Description |
|---|---|---|
| http-source-connector | certified | HTTP (POST/GET) to fluvio |
| http-sink-connector | certified | fluvio to HTTP (POST) |
| kafka-source-connector | certified | Kafka to fluvio |
| kafka-sink-connector | certified | fluvio to Kafka |
| mqtt-source-connector | certified | mqtt to fluvio |
| sql-sink-connector | certified | fluvio to SQL |
Connector status definition:
roadmap- requested by the community,experimental- under development (compile and run on your own)stable- proven to work, but not yet certified (compile and run on your own)certified- certified and available on InfinyOn Cloud and local environments