Hitachi Vantara Pentaho Community Forums
Results 1 to 10 of 10

Thread: Issue with SET Variable component in a job

  1. #1

    Default Issue with SET Variable component in a job

    Hello All,

    I have started exploring PDI just two days ago.

    Scenario:
    I have many csv files in a directory. I have to pick each csv file one by one and load them in a new table each time (table name should be created dynamically, table name should be same as file name). However, I am facing issue at the very start of my requirement itself.

    I am trying to use a SET VARIABLE component in a job where I created a variable DIR_PATH. I am giving the value as C:\PENTAHO\CDFDataFiles for that variable.

    Now I have placed a transformation where I am using GET VARIABLE where I am looking for the variable DIR_PATH using ctrl+space but i don't see the variable that I had created in the listing. How can I get the filenames?

    If someone can help me out here?

    Regards,
    Sachin

  2. #2
    Join Date
    Apr 2016
    Posts
    5

    Default

    Hey Sachin,

    I've kind of been having the same issues as I'm new to Pentaho too. What I've learnt is the 'get variable' is to retrieve the field of the table/result rather than the value of it, firstly.

    Secondly, the set variable has to be the last process on your transformation after which you can reference it and find it on the table of your next transformation. When you launch the next transformation, you can see the same in the "variables" column.

    Once you can see that it has been passed, you can use the same on a Cntrl + Space anywhere where variables are accepted. Don't use the GET variable, unless you want to import the field name


  3. #3
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by not_that_sachin View Post
    I am looking for the variable DIR_PATH using ctrl+space but i don't see the variable that I had created in the listing.
    Spoon only knows about the variables that are in the session you are working in.
    The fastest way to make those variables known to Spoon while you are doing your design work is to go to the Edit menu, and select "Set Environment Variables" (or simply press Ctrl-Alt-J )
    This will bring up another form where you can tell Spoon (the PDI Designer) what variables you expect to have set when this Transformation runs.

  4. #4
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by Imposter View Post
    Hey Sachin,

    I've kind of been having the same issues as I'm new to Pentaho too. What I've learnt is the 'get variable' is to retrieve the field of the table/result rather than the value of it, firstly.

    Secondly, the set variable has to be the last process on your transformation after which you can reference it and find it on the table of your next transformation. When you launch the next transformation, you can see the same in the "variables" column.

    Once you can see that it has been passed, you can use the same on a Cntrl + Space anywhere where variables are accepted. Don't use the GET variable, unless you want to import the field name

    You're not quite right... You might want to go back and read the page that marabu pointed you to on the other thread.

    Set Variable can happen anywhere in your stream, but it expects only one row.
    Get Variable will get values, and place them into a column in your datastream.

  5. #5

    Default

    Hi Gutlez,

    Thanks for your response. I am able to set the variable BUT when I try to click on show files button in Get Variable step, a pop up shows the path which I had set for the variable DIR_PATH and not the actual file name. I have my source files placed in "C:\PENTAHO\CDFDataFiles", so when I click show file in Get Variable step, it just shows me the path instead of actual file placed in that folder.

    Please let me know if I am missing something here.

    Regards,
    Sachin

  6. #6
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by not_that_sachin View Post
    click on show files button in Get Variable step
    Get Variables step doesn't have a Show Files button, so slow down and take your time doing things.

    Perhaps take a bit of time to read some of the documentation that has already been linked, and some of the user guides that have been written (PacktPub has some really good ones), and then explain what you are doing VERY carefully.

  7. #7

    Default

    I am so sorry that I typed GET Variable instead of GET File Names step. It's an honest mistake, I apologize but trust me I have studied myself to have a basic beginners level of working knowledge of Pentaho . So, I wanted to say about "Show File names" button on GET File Name step. I am attaching few images to explain the problem once again.

    When I set variable DIR_PATH by assigning a value and use this variable in GET file names step and when I click on show file names button, I only see the path and don't see the actual file I want to see. I am not sure what am I missing?

    And my main requirement is to get each file from a location one by one and store its data in a new table every time (table name should be same as the file name from which data is getting loaded into that table)

    Name:  Image1_Set_Var.jpg
Views: 309
Size:  25.4 KBName:  Image2_Get_File_Names.jpg
Views: 418
Size:  33.7 KB

    Also, When I use one more variable DIR_WILDCARD to pick files with .NOR extension (CDF_.*\.NOR) I get the below error saying no files found.

    Name:  Image3_Wildcard.jpg
Views: 301
Size:  25.8 KB
    Last edited by not_that_sachin; 04-26-2016 at 05:50 AM.

  8. #8
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    You forgot to enter a regex to specify the files ... (e.g. .* for all files)
    So long, and thanks for all the fish.

  9. #9

    Default

    Thank you very very much. Made my life.

    Regards,
    Sachin

  10. #10
    Join Date
    Feb 2016
    Posts
    23

    Default

    Quote Originally Posted by not_that_sachin View Post
    I am so sorry that I typed GET Variable instead of GET File Names step. It's an honest mistake, I apologize but trust me I have studied myself to have a basic beginners level of working knowledge of Pentaho . So, I wanted to say about "Show File names" button on GET File Name step. I am attaching few images to explain the problem once again.

    When I set variable DIR_PATH by assigning a value and use this variable in GET file names step and when I click on show file names button, I only see the path and don't see the actual file I want to see. I am not sure what am I missing?

    And my main requirement is to get each file from a location one by one and store its data in a new table every time (table name should be same as the file name from which data is getting loaded into that table)

    Name:  Image1_Set_Var.jpg
Views: 309
Size:  25.4 KBName:  Image2_Get_File_Names.jpg
Views: 418
Size:  33.7 KB

    Also, When I use one more variable DIR_WILDCARD to pick files with .NOR extension (CDF_.*\.NOR) I get the below error saying no files found.

    Name:  Image3_Wildcard.jpg
Views: 301
Size:  25.8 KB
    Hello,

    I think I also have the same problem you had.

    I have one question, if it's possible, Where did you set "DIR_PATH"? where did you put the "Set variables" step?

    Thanks in advance.
    Last edited by Soukaina; 04-26-2016 at 09:44 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.