Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: stream output to webservice (soap)

  1. #1
    Join Date
    Mar 2011
    Posts
    2

    Default stream output to webservice (soap)

    Is there a way to create an output stream which I can send to a webservice?

    I have a huge amount of data which I want to store in a DB via Webservice(SOAP).
    This Webservice calls some db-prozedures to bring the data into DB.
    Doing this for every datarow is too expensive (it takes approx. 20 - 30 min).
    In my opinion streaming would be a better way.
    I'm using kettle 4.1.0-stable.

  2. #2
    Join Date
    Mar 2008
    Posts
    140

    Default

    My first thought would be to try parallelizing the step.

  3. #3
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    I think that a SOAP request expects a single block of XML. As such I think that streaming in the traditional sense is not possible.
    However, to reduce latency it would probably be a good idea to batch up records in sets of 1000 or even more.
    Obviously that's not going to change the fact that the web service AND the stored procedure are both serious performance bottlenecks.

  4. #4
    Join Date
    Mar 2011
    Posts
    2

    Default

    Thanks a lot! That's what I assumed.
    Well, I have to rethink this approach.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.