rkafka (version 1.4)

rkafka.createConsumer: Creating KAFKA consumer

Description

This function creates a KAFKA consumer

Usage

rkafka.createConsumer(zookeeperConnect, topicName, 
groupId="test-consumer-group", zookeeperConnectionTimeoutMs="100000", 
consumerTimeoutMs="10000", autoCommitEnable="NULL", 
autoCommitInterval="NULL", autoOffsetReset="NULL")

Value

Returns a consumer

Arguments

zookeeperConnect

Zookeeper connection string comma separated host:port pairs, each corresponding to a zk server. e.g."127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002" Required:Mandatory Type:String default:NONE

topicName

Name of the topic from which to read messages Required:Mandatory Type:String

groupId

consumer group id Required:Mandatory Type:String default:test-consumer-group

zookeeperConnectionTimeoutMs

timeout in ms for connecting to zookeeper Required:Mandatory Type:String default:100000

consumerTimeoutMs

Throw a timeout exception to the consumer if no message is available for consumption after the specified interval Required:Mandatory Type:String default:10000

autoCommitEnable

If true, periodically commit to ZooKeeper the offset of messages already fetched by the consumer. This committed offset will be used when the process fails as the position from which the new consumer will begin. Required:Optional Type:String default:true

autoCommitInterval

The frequency in ms that the consumer offsets are committed to zookeeper. Required:Optional Type:String default:60*1000

autoOffsetReset

smallest : automatically reset the offset to the smallest offset largest : automatically reset the offset to the largest offset anything else: throw exception to the consumer Required:Optional Type:String default:largest

Author

Shruti Gupta

Details

There are two types of KAFKA consumers: High-level and Simple. This functions creates a high level consumer

References

To know when to use simple consumer and when to use High-level Consumer, refer the url below: https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example

To know how to use a high level consumer refer this: https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example

Examples

if (FALSE) {
consumer1=rkafka.createConsumer("127.0.0.1:2181","test123")
consumer2=rkafka.createConsumer("127.0.0.1:2181","test123","test-consumer-group","50000","1000")
}