Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Apache Kafka Consumer don't get newest data from Broker

  1. #1

    Default Apache Kafka Consumer don't get newest data from Broker

    Hi everyone,

    I am using Apache Kafka Consumer in pentaho data integration to read binary message from broker.
    I want to do some real-time dashboard from obtained data.
    So, i need the Apache Kafka Consumer to get the newest data from Broker.
    My problem is that the Apache Kafka Consumer doesn't read newest message from Broker.
    It reads older message.
    Please help me to figure out how to configure the Apache Kafka Consumer to solve this problem.
    Sorry for my bad english.

    Many thanks,
    hoang anh.

  2. #2

    Default

    Hi everyone,

    I want to update some extra information.
    when i set Messages Limit property in Apache Kafka Consumer to a small value (e.g 10), It read older messages.
    But When i set that property to a bigger value (e.g 1000), it can read newest messages.
    I don't understand why this is so.
    Please explain it to me.

    Many thanks,
    Hoang Anh.

  3. #3
    Join Date
    Dec 2013
    Posts
    13

    Default

    Set auto.offset.reset=earliest
    Thanks & Regards
    Karan Arora

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.