Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Getting data from Web Service - how to stop when there's no more data

  1. #1
    Join Date
    Jun 2007
    Posts
    103

    Default Getting data from Web Service - how to stop when there's no more data

    Here's the use case: our PDI job polls a web service (ws) to retrieve a near-real-time data feed. We call the ws and a number of rows are returned in an xml file. If the number of rows returned equals the maximum allowable batch size, we need to run the job again, until the number of rows returned is less than the batch size. This is the only way to know if we have gotten all the current data.

    How can we loop around a job while a condition is true (rows returned = batch size)?

    Many thanks
    - Russell

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Usually, job entry Simple-Evaluation is quite helpful to implement exit control in a loop design - either based on a result stream or on a variable.
    So long, and thanks for all the fish.

  3. #3
    Join Date
    Sep 2016
    Posts
    16

    Default

    Hi, According to me Parameter Pushdown in dataservice https://help.pentaho.com/Documentati...L0/0Y0/090/020 may help you.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.