Skip to content

newrelic/nri-kafka

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

New Relic Open Source community plus project banner.

New Relic integration for Kafka

The New Relic integration for Kafka captures critical performance metrics and inventory reported by Kafka clusters. Data on Brokers, Topics, Java Producers, and Java Consumers is collected.

Inventory data is obtained mainly from Zookeeper nodes and metrics are collected through JMX.

See our documentation web site for more details.

Requirements

JMX is required to be enabled on the following entities in order to be able to collect metrics:

  • Brokers
  • Java Producers
  • Java Consumers

Information on configuring JMX can be found here.

Installation and usage

For installation and usage instructions, see our documentation web site.

Compatibility

  • Supported OS: No limitations
  • Kafka versions: 0.8+

Building

Golang is required to build the integration. We recommend Golang 1.11 or higher.

After cloning this repository, go to the directory of the Kafka integration and build it:

$ make

The command above executes the tests for the Kafka integration and builds an executable file called nri-kafka under the bin directory.

To start the integration, run nri-kafka:

$ ./bin/nri-kafka

If you want to know more about usage of ./bin/nri-kafka, pass the -help parameter:

$ ./bin/nri-kafka -help

External dependencies are managed through the govendor tool. Locking all external dependencies to a specific version (if possible) into the vendor directory is required.

Testing

To run the tests execute:

$ make test

Consumer/Producer container image

A container image for testing purposes to create a simple Java consumer/producer exposing JMX metrics, kafka-consumer-producer.

The consumer/producer will be launched with the client.id set to the name of the hostname for easier matching when setting the nri-kafka config file.

The container can be launched as a consumer with the command:

command: ["consumer","broker:9092","topicA","groupA"]

And as a producer:

command: ["producer","broker:9092","topicA"]

Refer to this example to see how to overwrite the JMX settings if you want to use it (note we set the hostname to not be auto-generated): docker-compose-example.

Support

Should you need assistance with New Relic products, you are in good hands with several support diagnostic tools and support channels.

New Relic offers NRDiag, a client-side diagnostic utility that automatically detects common problems with New Relic agents. If NRDiag detects a problem, it suggests troubleshooting steps. NRDiag can also automatically attach troubleshooting data to a New Relic Support ticket.

If the issue has been confirmed as a bug or is a Feature request, please file a Github issue.

Support Channels

Privacy

At New Relic we take your privacy and the security of your information seriously, and are committed to protecting your information. We must emphasize the importance of not sharing personal data in public forums, and ask all users to scrub logs and diagnostic information for sensitive information, whether personal, proprietary, or otherwise.

We define “Personal Data” as any information relating to an identified or identifiable individual, including, for example, your name, phone number, post code or zip code, Device ID, IP address, and email address.

For more information, review New Relic’s General Data Privacy Notice.

Contribute

We encourage your contributions to improve this project! Keep in mind that when you submit your pull request, you'll need to sign the CLA via the click-through using CLA-Assistant. You only have to sign the CLA one time per project.

If you have any questions, or to execute our corporate CLA (which is required if your contribution is on behalf of a company), drop us an email at [email protected].

A note about vulnerabilities

As noted in our security policy, New Relic is committed to the privacy and security of our customers and their data. We believe that providing coordinated disclosure by security researchers and engaging with the security community are important means to achieve our security goals.

If you believe you have found a security vulnerability in this project or any of New Relic's products or websites, we welcome and greatly appreciate you reporting it to New Relic through HackerOne.

If you would like to contribute to this project, review these guidelines.

To all contributors, we thank you! Without your contribution, this project would not be what it is today.

License

nri-kafka is licensed under the MIT License.