Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

drop everything but ceph #1

Merged
merged 2 commits into from
Sep 17, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
50 changes: 0 additions & 50 deletions OWNERS_ALIASES
Original file line number Diff line number Diff line change
Expand Up @@ -9,56 +9,6 @@ aliases:
# These aliases are for OWNERS of the various Source implementations. These
# Are in addition to the repo level OWNERS.

awssqs-approvers:
- lberk
awssqs-reviewers:
- lberk

camel-approvers:
- nicolaferraro
- matzew
camel-reviewers:
- nicolaferraro
- matzew

github-approvers:
- lberk
- mattmoor
github-reviewers:
- lberk
- rootfs

gitlab-approvers:
- sebgoa
- tzununbekov
gitlab-reviewers:
- sebgoa
- tzununbekov

kafka-approvers:
- lberk
- matzew
kafka-reviewers:
- lberk
- matzew

natss-approvers:
- devguyio
natss-reviewers:
- devguyio

couchdb-approvers:
- lionelvillard
couchdb-reviewers:
- lionelvillard

prometheus-approvers:
- syedriko
- matzew
prometheus-reviewers:
- syedriko
- matzew

productivity-approvers:
- chaodaiG
- chizhg
Expand Down
118 changes: 99 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,13 @@
# Knative Eventing Contrib
# Knative Eventing Ceph Source

[![go.dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white)](https://pkg.go.dev/github.com/knative/eventing-contrib)
[![Go Report Card](https://goreportcard.com/badge/knative/eventing-contrib)](https://goreportcard.com/report/knative/eventing-contrib)
[![Releases](https://img.shields.io/github/release-pre/knative/eventing-contrib.svg)](https://github.com/knative/eventing-contrib/releases)
[![LICENSE](https://img.shields.io/github/license/knative/eventing-contrib.svg)](https://github.com/knative/eventing-contrib/blob/master/LICENSE)
[![go.dev reference](https://img.shields.io/badge/go.dev-reference-007d9c?logo=go&logoColor=white)](https://pkg.go.dev/github.com/knative-sandbox/eventing-ceph)
[![Go Report Card](https://goreportcard.com/badge/knative/eventing-contrib)](https://goreportcard.com/report/knative-sandbox/eventing-ceph)
[![Releases](https://img.shields.io/github/release-pre/knative/eventing-contrib.svg)](https://github.com/knative-sandbox/eventing-ceph/releases)
[![LICENSE](https://img.shields.io/github/license/knative/eventing-contrib.svg)](https://github.com/knative-sandbox/eventing-ceph/blob/master/LICENSE)
[![Slack Status](https://img.shields.io/badge/slack-join_chat-white.svg?logo=slack&style=social)](https://knative.slack.com)

The Knative Eventing Contrib project provides source and channel implementations
that:

- Integrate with Apache Camel-K
- Integrate with Apache CouchDB
- Integrate with Apache Kafka
- Integrate with AWS SQS
- Integrate with Ceph
- Integrate with GitHub
- Integrate with GitLab
- Integrate with NATS Streaming
- Integrate with Prometheus
- Integrate with Websockets
- Expose an ingress
The Knative Eventing Ceph project provides source implementation that registers
events for Ceph storage notifications.

For complete documentation about Knative Eventing, see the following repos:

Expand All @@ -30,3 +18,95 @@ For complete documentation about Knative Eventing, see the following repos:

If you are interested in contributing, see [CONTRIBUTING.md](./CONTRIBUTING.md)
and [DEVELOPMENT.md](./DEVELOPMENT.md).

# Ceph to Knative Receive Adapters

The following receive adapters convert bucket notifications from
[Ceph format](https://docs.ceph.com/docs/master/radosgw/notifications/#events)
into CloudEvents format, and inject them into Knative. Adapter logic follow the
one described for
[AWS S3](https://github.com/cloudevents/spec/blob/master/adapters/aws-s3.md)
bucket notifications.

There are 4 different transport options:

## HTTP

This receive adapter expects bucket notifications from Ceph over HTTP transport,
as payload in POST messages.

> Note that this adapted does not assume the CloudEvents HTTP binding in the
> incoming messages.

### Testing

- Create a namespace for the tests:

```
kubectl apply -f samples/ceph-eventing-ns.yaml
```

- Deploy a service for the bucket notification messages coming from Ceph:

```
kubectl apply -f samples/ceph-bucket-notification-svc.yaml -n ceph-eventing
```

- Deploy the ceph-event-display Knative service together with a channel and
subscribe it to the channel:

```
ko apply -f samples/ceph-display-resources.yaml -n ceph-eventing
```

- Build and deploy the Ceph source adapter with SinkBinding to inject the sink:

```
ko apply -f samples/ceph-source.yaml -n ceph-eventing
```

- Deploy a test pod that has cURL installed and a JSON file with bucket
notifications (names `records.json`):

```
kubectl apply -f samples/test-pod.yaml -n ceph-eventing
```

- Execute cURL command on the test pod to send the JSON bucket notifications to
the ceph-bucket-notifications service:

```
kubectl exec test -n ceph-eventing -- curl -d "@records.json" -X POST ceph-bucket-notifications.ceph-eventing.svc.cluster.local
```

- To verify that the events reached the ceph-event-display service, call:

```
kubectl logs -l serving.knative.dev/service=ceph-event-display -n ceph-eventing -c ceph-display-container --tail=100
```

## PubSub (TODO)

This receive adapter is pulling bucket notifications from a special "pubsub"
zone in Ceph, and acking them once they are successfully sent to Knative. More
information about this mechanism could be found
[here](https://docs.ceph.com/docs/master/radosgw/pubsub-module/).

> Note that this adapter needs access to the Ceph cluster to pull the
> notifications and ack them.

## AMQP0.9.1 (TODO)

This receive adapter is subscribed to a list of AMQP topics, on which bucket
notifications are expected.

> Note that this adapted does not assume the CloudEvents AMQP binding in the
> incoming messages.

## Kafka (TODO)

This receive adapter is subscribed to a list of Kafka topics, on which bucket
notifications are expected.

> Note that this adapted does not assume the CloudEvents Kafka binding in the
> incoming messages.
9 changes: 0 additions & 9 deletions awssqs/OWNERS

This file was deleted.

32 changes: 0 additions & 32 deletions awssqs/cmd/controller/main.go

This file was deleted.

75 changes: 0 additions & 75 deletions awssqs/cmd/receive_adapter/main.go

This file was deleted.

20 changes: 0 additions & 20 deletions awssqs/config/100-namespace.yaml

This file was deleted.

21 changes: 0 additions & 21 deletions awssqs/config/200-serviceaccount.yaml

This file was deleted.

Loading