Sending data to Apache Kafka Producer

[Lunendl]

Hi guys,

I'm new with the product and would like to know how to send data from a JSON (or other) file into a Kafka Producer.

Thanks

[marabu]

There are a couple of Input steps you might find useful, e.g. JSON Input.
If your producer API is compatible with Java 1.4 (Janino restriction, no generics) you can send messages from a User Defined Java Class.

[Lunendl]

Thank you very much. Although I'm not sure if I'm grasping the JSON Output / Input module. In essence, I have an CSV file that need to go into Kafka.
Thus far, I can do the following.
CSV File contains something like:
ccy | code
JPY | HR00
NZ | HN00

I can transfer the above data into a JSON file that contains:
{"JsonBlockName":[{"ccy":"JPY","code":"HR00"},{"ccy":"NZ","code":"HN00"}]}

My two nodes are CSV file input and Json output. ANY bit of help or ideas will be much appreciated.

[marabu]

Sorry, can't read your screenshot - didn't you check yourself before posting?

[Lunendl]

Sorry, can't read your screenshot - didn't you check yourself before posting?

Hi, sorry. Thought the post would have enabled people to open the original size format. Lesson learnt.

Currently I'm getting an error on the Kafka Producer step. "Input field 'OutputValue' is not binary!"

[Lunendl]

Is there anyone that can help me with this. Please

[Lunendl]

Managed to load the data into kafka by using the following format and setting the "Header" as a binary value (Contributing already!):
Header
{"ccy":"JPY","code":"HR00"}
{"ccy":"NZ","code":"HN00"}

I still sit with a small data manipulation issue when I'm transforming the CSV file to JSON Output. The closest I can get to this format is:
{"":[{"ccy":"JPY","code":"HR00"},{"ccy":"NZ","code":"HN00"}]}

[marabu]

Set the block factor to 1 and extract the row.