how to concatenate 100 output rows into one output row

[chris99]

I have database rows from a table input step that produces one JSON document per row.
What I want to do is to concatenate `100 rows at a time with a comma delimiter and write 1 row out, then
process the next 100 rows, then the remainder if any.

I've been trying UDJC but abandoned that in favor of modified javascript step but am not sure how to achieve this.

[rgardner]

Hi Chris,

You can accomplish this with a sequence, integer division, and a group by:

1. add an incremental sequence to the incoming database rows (you can do this with the "Add Sequence" step or using SQL) - make sure the sequence starts at zero!
2. use the calculator step to
   
   1. define a constant value group_size variable set to 100 (integer data type)
   2. define a json_group variable as sequence / group_size (integer data type - use the A / B calculation)
   
   
3. use Group By step:
   
   1. group by json_group
   2. aggregate your json rows as the "Subject" in the "Aggregates" section. Name the result whatever you like, and use the "Concatenate strings separated by ," as the "Type" selection.

[chris99]

Thanks rgardner, I'll try it out - looks good

[chris99]

Woks great, thanks again