KAFKA-3080: Fix ConsoleConsumerTest by checking version when service is started

[ewencp]

The MessageFormatter being used was only introduced as of 0.9.0.0. The Kafka
version in some tests is changed dynamically, sometimes from trunk back to an
earlier version, so this option must be set based on the version used when the
service is started, not when it is created.

[ewencp]

@granders Tracked this down to 9220df9. Verified this fixes the failing test and still passes all of security_rolling_upgrade_test. Maybe @benstopford can sanity check this as well since he was the author on the original patch?

And perhaps @gwenshap for final review + commit?

[ijuma]

Thanks for fixing this @ewencp, makes sense to me.

[granders]

Looks good, passes locally.

Not expecting anything new, but branch builder is running here:
http://jenkins.confluent.io/job/kafka_system_tests_branch_builder/338/

[ijuma]

@granders there were some failures in the branch builder. Are they related?

[granders]

@ijuma
There were 2 failures, but as far as I can tell they seem unrelated:

This failed, but also failed on trunk on 1/17/2016, so I believe it is unrelated

Module: kafkatest.tests.zookeeper_security_upgrade_test
Class:  ZooKeeperSecurityUpgradeTest
Method: test_zk_security_upgrade
Arguments:
{
  "security_protocol": "SASL_PLAINTEXT"
}

This also failed

Module: kafkatest.tests.benchmark_test
Class:  Benchmark
Method: test_consumer_throughput
Arguments:
{
  "security_protocol": "SSL"
}

Failure was due to a parsing error (parsing statistics choked on a NotLeaderForPartitionException), but the same test with "interbroker_security_protocol": "PLAINTEXT", "security_protocol": "SSL" passed. If the failure was due to Ewen's change, I would have expected it to fail on "interbroker_security_protocol": "PLAINTEXT" as well.

[granders]

Running again here:
https://jenkins.confluent.io/job/kafka_system_tests_branch_builder/341/

[granders]

@ewencp @gwenshap All tests passed:
http://confluent-kafka-branch-builder-system-test-results.s3-us-west-2.amazonaws.com/2016-01-18--001.1453165281--ewencp--kafka-3080-system-test-console-consumer-version-failure--0f0d33f/report.html

[ijuma]

Thanks @granders. It would be great to get this in @gwenshap as it fixes a bunch of system test failures.

[granthenke]

LGTM. It would be great to get this in.

[gwenshap]

LGTM