Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for local Avro schema in CONSUMER/PRODUCER of kafka-file #188

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

angelozerr
Copy link
Collaborator

Support for local Avro schema in CONSUMER/PRODUCER of kafka-file

Fixes #114

Signed-off-by: azerr [email protected]

@angelozerr
Copy link
Collaborator Author

The avro support starts working but it requires

  • some work (clean code, manage cache for schema, write test and cocetc)
  • and perhaps more feature (validation, hyperlink, completion for avro file path)

This PR provides :

  • avsc file support (Avro Schema definition written in JSON format). When you create a *.avsc file you should have validation, completion (based on the avro JSON Schema).
  • declare avro file path in key-format, value-format in kafka file PRODUCER,CONSUMER.

Here a sample:

  • create the animals.avsc file:
{
    "type": "record",
    "fields": [
        {
            "name": "kind",
            "type": {
                "name": "animals_type",
                "type": "enum",
                "symbols": [
                    "CAT",
                    "DOG"
                ]
            }
        },
        {
            "name": "name",
            "type": "string"
        }
    ]
}
  • create the animals.kafka file:
PRODUCER json-output
topic: topic_name
value-format: avro(animals.avsc)
{"kind": "CAT", "name": "Albert"}

###

CONSUMER consumer-group-id
topic: topic_name
from: earliest
value-format: avro(animals.avsc)

@angelozerr angelozerr force-pushed the avro branch 2 times, most recently from a67edd3 to 7dd94d2 Compare May 12, 2021 08:11
@fbricon
Copy link
Collaborator

fbricon commented May 12, 2021

@angelozerr angelozerr force-pushed the avro branch 5 times, most recently from e914b35 to 4e7d532 Compare May 12, 2021 14:53
@fbricon
Copy link
Collaborator

fbricon commented May 15, 2021

Please fix the conflict

const currentWorkspace: WorkspaceFolder | undefined = (currentFile && currentFile.uri) ? workspace.getWorkspaceFolder(currentFile.uri) : undefined;
const currentWorkspaceUri: string | undefined = (currentWorkspace && currentWorkspace.uri.fsPath)
|| (workspace.workspaceFolders && workspace.workspaceFolders[0].uri.fsPath);
const resolvedPath = currentWorkspaceUri + '/' + path;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use platform specific separator

@angelozerr angelozerr force-pushed the avro branch 2 times, most recently from 0067688 to eec0f66 Compare May 15, 2021 11:40
@angelozerr
Copy link
Collaborator Author

Please fix the conflict

fixed

I need to write tests and docs yet.

@angelozerr angelozerr force-pushed the avro branch 7 times, most recently from 0678eba to 8babeda Compare May 17, 2021 12:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for local Avro schema in CONSUMER/PRODUCER of kafka-file
2 participants