Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Help Newbie: How to import a zipped xml file via HTTP into MySQL

  1. #1

    Default Help Newbie: How to import a zipped xml file via HTTP into MySQL

    Hi Guys, I am looking to replace a current java solution with an ETL solution since I keep getting memory issues processing large XML files and the bulk import solution seems much quicker (and easier?).

    My thinking was to generate a simple ETL demo to test this using the following steps:
    1) HTTP - to do the XML zip file download
    2) UNZIP - to unzip the zip file and use wildcard to pick up the single XML file
    3) BULKLOAD INTO MYSQL - to import **that single XMLfile ** into MySQL
    (Then maybe use java to process the imported table).

    Initial problem is that I don't know the name of that single XML file to use in Step 3.
    Step 2 unzips using a wildcard expression to extract the file.

    How can I either:
    a) rename that extracted file (wildcard) to a fixed name say x.xml ?
    b) retrieve the wildcard name from the directory for reuse ?

    Kind regards,

    Mark Dathorne

  2. #2

    Default

    Hi mark.dathorne,

    You can/should use variables and parameters to get the name of the file to process into kettle.
    In most text fields you can input a variable (Ctrl+Space) which you see via icon 'red dollar on grey rhomb'.
    SQL parts can also evaluate variables when you check the option.

    Christoph

    EDIT: I should expose there are differences between variables and parameters! Variables can be given via -Dvarname=value (Java) or kettle.properties. The latter is not relevant for you purpose. Parameters are the things after kitchen.sh.
    Last edited by christophWeißenborn; 08-13-2008 at 05:27 AM.

  3. #3
    DEinspanjer Guest

    Default

    Mark,

    If you are extracting the zip into a clean directory, you can use the Get File Names transformation step to find the XML file even if you don't know the name of it using a wildcard. Then you can pass the filename field into a Set Variables step and then use that variable in the bulkload into mysql job entry.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.