Hitachi Vantara Pentaho Community Forums
Results 1 to 8 of 8

Thread: Passing dynamic queries to the Table Input

  1. #1
    Join Date
    Sep 2005
    Posts
    1,403

    Default Passing dynamic queries to the Table Input

    I would like to copy data from a table A on DB1 to table B on DB2 (Table B doesn't exist). So I created a transformation from a Table Input to Table Output.


    But the issue is I have a dynamic WHERE query for Table Input, whose parameters need to be passed during Runtime. Is this possible?

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default RE: Passing dynamic queries to the Table Input

    Yes, there is an example in the Spoon manual (Table Input)

    Matt

  3. #3
    Join Date
    Sep 2005
    Posts
    1,403

    Default RE: Passing dynamic queries to the Table Input

    Actually I don't want to pass System variables, I want to assign values inside a Java class.

  4. #4
    Join Date
    Nov 1999
    Posts
    9,729

    Default RE: Passing dynamic queries to the Table Input

    Whatever the case, you need to get these values over to the TableInput.
    If you have your own Java classes, you can use a plugin or a piece of Javascript to get at them.
    There are a couple of JavaScript examples on that floating around on the forum.

    That's all I can say about it. Depends on how you're using Kettle really.

    All the best,

    Matt

  5. #5
    Join Date
    Sep 2005
    Posts
    1,403

    Default RE: Passing dynamic queries to the Table Input

    Hi Matt,

    Sorry for being a pest. Honestly I didn't find any java script samples that are used in passing arguments to next steps.

    I have been looking at Kettle since 2 weeks. And I really want to use this tool as much as we could. Our intention is to load a text file (could vary in size from 10k to 5MB data) to the database (in the middle we have a matching process to check if the record already exists or not, if exist we have to update or else add). The filename varies all the time which needs to be passed at the run time. Also, we are planning on extracting the existing data depending on a dynamic query from the SQL Server DB to an In memory Db for faster matching process (WHERE clause always changes).

    I was planning on creating different transformations as follows:
    Text Input -------> XML output (File name always differs for Input/Output)
    XML Input --------> Table Output
    Table Input ---------> Table Input

    And then compare the data in two tables in In Memory DB....

    I am able to run transformations from our application (J2EE), but I am really stuck at the point where I have to pass arguments at runtime to the Steps.

    Is KETTLE a right tool to use for this?

    If yes, can you please point me to the right documentation or samples.

    I am using Kettle 2.2.2, JDK 5.0, Jboss 4.0.4

    Regards,

  6. #6
    Join Date
    Sep 2005
    Posts
    1,403

    Default RE: Passing dynamic queries to the Table Input

    Sorry I meant Passing arguments to the Steps. I will try using TransMeta object for this as you suggected.

    Any rough idea when 2.3 will be released?

    I meant using Hypersonic SQL in JBoss as In memory DB for comparing the records (suggested by our lead).

    I wish you had more FAQ or documentation for KETTLE to integrate with other applications.

    Thanks,

  7. #7
    Join Date
    Sep 2005
    Posts
    1,403

    Default RE: Passing dynamic queries to the Table Input

    I really don't understand what you mean by "Passing arguments to next steps". All rows are arguments that are passed to next steps!

    Same for passing "Arguments at runtime": you can grab command-line arguments in "Get System Info" or set these on the TransMeta object.

    In 2.3.0 I'm adding a lot of stuff to allow you to use dynamic variables to make your life even easier.

    > Is KETTLE a right tool to use for this?

    That's only something YOU have to decide as I still have no clue to what you're actually doing ;-)
    However, plenty of folks ARE using it to do similar stuff, even on J2EE.

    Hey, this "In Memory DB": what is that?

    > Sorry for being a pest.

    Well, I'm seriously considering to stop replying to all messages of which I don't understand the content. I simply don't have the time or energy anymore to keep asking additional info time after time. Seems like everyone on this forum expects me to have magical powers or something.

    Thanks,

    Matt

  8. #8
    Join Date
    Nov 1999
    Posts
    9,729

    Default RE: Passing dynamic queries to the Table Input

    > Any rough idea when 2.3 will be released?

    I once did a hike in the north of Thailand and when we asked the guide: "how much further is the camp?", he always replied "9 miles".

    When people ask for the release date of 2.3 I always answer: "in a couple of weeks", it means exactly the same thing: when it's ready, when we have arrived.

    >I wish you had more FAQ or documentation for KETTLE to integrate with other applications.

    We (Pentaho) *are* preparing AI/EII docs and samples. I must say that traditionally Kettle was designed as an ETL tool, so give it some time please ;-)
    That being said, I think there are some examples issued by Pentaho on how to grab the output of a certain step in a transformation, maybe you could simply use that. After all, Pentaho itself runs on JBoss and there *is* a KettleComponent you can use shielding you from all the complexity.

    Finally, if you have figured out a good way to do this, why not post a FAQ or a document about this yourself I'm certainly not going to do it as I'm no J2EE expert by any standard.

    All the best,

    Matt

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.