Skip to content

Troubleshooting

Jump to bottom Edit New page
Vadim Chekan edited this page Oct 2, 2015 · 5 revisions

Logger

Kafka4net supports 2 most popular loggers log4net and NLog. Just call

kafka4net.Logger.SetupLog4Net()

or

kafka4net.Logger.SetupNLog()

and kafka4net will start logging into host application configured logger.

Tracing

This is the most powerful way of troubleshooting kafka4net. Kafka4net supports ETW tracing. If you do not have preferred tracing viewer, I would recommend PerfView. It does not have pretty UI but does its job very well and saved me countless hours of debugging.

Register kafka4net ETW event manifest. ETW is super optimized (it is used for drivers development) and that's why it does not embed any metadata into trace files. Not even fields markers. Instead format of the trace data is stored out of band. Find trace manifest file in kafka4net nuget package, packages\kafka4net.0.1.5627.17670\etw-manifest\kafka4net.kafka4net.etwManifest.man. With Administrator privileges, execute wevtutil im provider.man kafka4net.kafka4net.etwManifest.man Without this registration, data you capture in PerfView will be just binary blobs.

Start PerfView and start Collect operation Collect menu image

Uncheck Zip and Merge checkboxes. They are useful only if you want to copy trace to another computer. Type into Additional Provider kafka4net. Pay attention to Circular MB field. If your trace runs over this value, start of your trace will be overwritten, and you will spend some time wondering where did the trace go. Click Start Collection button. Params image

When done, click "Stop collecting" button. It will take some time for PerfView to process the dump. When it's done, open the .etl file you've created and open Events item.

Select name of the process in Process Filter. On this screenshot it is Resharper unit test process. In Filter select "kafka", and select type of events you are interested in and hit "Enter". You can multi-select. Now try to make sense out of your trace. Events image

Event types

  • Fetcher: data fetching command
  • Connection: connection management: close, open, error, etc.
  • Correlation: low-level read/write activity on connection.
  • Recovery Monitor: partition recovery activity.
  • Protocol: kafka commands, protocol level. Requests sent and responses received.
  • Producer: producer errors.
  • Consumer: consumer data flow control.
  • Metadata discovered brokers and partitions, partition state change.

Counters

TODO: think how to expose important events, such as buffers fill ratio and expose them via Metric.Net

Add a custom footer
Add a custom sidebar
Clone this wiki locally