offset & consumer groups

[DEvil0000]

a consumer offset should be stored per topic/partition/consumer group...
as long as broker based offset is not working correct #192 and zookeeper is not integrated #4 it would still be possible to support this feature (local groups only) with file based offsets.

the filename is now constructed from 1) topic name and 2) partition but it should simply include 3) group id.

workaround (local groups only): add consumer group as folder to the path..

ps: i would suggest something like a consumer config instead of a global config

[edenhill]

Yes, you are absolutely right, it should include the groupid indeed.

To not break existing clients that rely on their offset files not being renamed I suggest the following approach:

• if group.id is not configured (it defaults to an empty string) the current functionality with no groupid in the filename remains. (topic-partition.offset)
• if group.id is configured the filename is constructed as: topic-partition-groupid.offset
• an application that explicitly sets offset.store.path to a file will still use that rather than one of the above alternatives.

Comments?

[DEvil0000]

sounds good

[edenhill]

This is fixed now, can you verify on your end aswell?
Thanks.

[edenhill]

Oh, btw, I added group.id to the topic configuration, which overrides the global group.id configuration.

[DEvil0000]

works nice - thx for the fast implementation