Hitachi Vantara Pentaho Community Forums
Results 1 to 6 of 6

Thread: Can't read xml input from ${FILENAME}

  1. #1
    Join Date
    Jul 2007
    Posts
    2,498

    Unhappy Can't read xml input from ${FILENAME}

    Hello.

    I'm trying to process all xml files from a dir; Made a job with one transformation that gets all filenames and stores the result. And a second one, processed for each row, that is intended to process that file.

    The problem is that I can't read the file; I always get this error:

    Code:
    XML Input.0 - ERROR (version 2.5.0, build 25002 from 2007/05/04 00:20:04) : No file(s) specified! Stop processing.
    The filename in xml input is ${FILENAME}, and if I take out that xmlinput and just store it to text I see the env. var is correctly grabbed.

    Help please


    Edit: Just noticed that if I make a plain test output to a file called test-${FILENAME}.txt, the output is only test-.txt. I'm def. missing something


    Thanks
    Last edited by pmalves; 08-17-2007 at 07:06 AM.
    Pedro Alves
    Meet us on ##pentaho, a FreeNode irc channel

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    Not all steps support variable sensitivity yet. Make a small test case and attach it.

    Regards,
    Sven

  3. #3
    Join Date
    Jul 2007
    Posts
    2,498

    Default

    My problem was when copying the results. This does not work:



    The "Get List of Files" copies rows to results and "Define FILENAME variable" gets rows from result. But it does not get anything. The only way to get this to work is if I replace the last step with a job and the put that define filename var inside that job; On that case, it works.


    My second problem was accessing that variable; I can't access that variable from the transformation where I set it; I need to make another transformation, link it in the parent job, and only then that variable will be set;


    All of this is in the "samples/jobs/process\ all\ tables/" example, with all those different jobs/steps; But I must confess that I was expecting the simple model in the picture above to work; why so many steps?

    1. Why can't I get results from one transformation to another without a job?

    2. Why can't I access a variable inside the same transformation I define it?


    Thanks
    Pedro Alves
    Meet us on ##pentaho, a FreeNode irc channel

  4. #4
    Join Date
    May 2006
    Posts
    4,882

    Default

    The "Get List of Files" copies rows to results and "Define FILENAME variable" gets rows from result. But it does not get anything. The only way to get this to work is if I replace the last step with a job and the put that define filename var inside that job; On that case, it works.
    Did you switch off clear results.

    All of this is in the "samples/jobs/process\ all\ tables/" example, with all those different jobs/steps; But I must confess that I was expecting the simple model in the picture above to work; why so many steps?

    1. Why can't I get results from one transformation to another without a job?
    You normally should be able to... it may be related to variables in v2.5 ... You do need of course a super job that executes both transformations.

    2. Why can't I access a variable inside the same transformation I define it?
    Steps runs concurrently, you are not sure the variables are defined when you use them. Job entries run in sequence, that's why.

    Regards,
    Sven

  5. #5
    Join Date
    Jul 2007
    Posts
    2,498

    Default

    Did you switch off clear results.
    I was positive that I did; however, I've made some tests and seems to be working


    Steps runs concurrently, you are not sure the variables are defined when you use them. Job entries run in sequence, that's why.
    Even with the hops defining the order? It adds some complexity to the model having an extra transformations (different transformations make things harder to debug) just to store a var.

    Thanks
    Pedro Alves
    Meet us on ##pentaho, a FreeNode irc channel

  6. #6
    Join Date
    May 2006
    Posts
    4,882

    Default

    Even with the hops defining the order? It adds some complexity to the model having an extra transformations (different transformations make things harder to debug) just to store a var.
    Yes, I'm writing some doc on it

    The difference between a job and a transformation qua execution of components is that:
    - a job is a like finite state automaton... only 1 state/job entry is active at any 1 time. You execute a transformation from a job, the job will stay in that job entry until the transformation ends
    - a transformation starts up all steps concurrently for speed. Every step runs in its own thread and will process at its own pace, each hop is a waiting queue between processors.

    For some transformations you could get away with using the variable defined in the same transformation but in 99% of the cases you won't.
    How transformations work is that all steps are first initialized before they are put active, the initialization of a lot of the steps will already require a variable if you use them (e.g. the name of the input file using variables), but a variable is only defined at the moment a certain input row hits a set variable step.
    So this is why you want to define variables in one transformation and use them in another. If you use them in the same transformation, you need the variable value before it's usually defined.

    Another side effect of the concurrent steps (which causes some confusion with people) e.g. is that if e.g. you use a combination of filters with textfile output steps.. all text files will be created in the end although they may not have been triggered by data/rows. All output steps will have initialized (already creating an empty file) and started even before a row needs to "hit" them.

    Regards,
    Sven
    Last edited by sboden; 08-17-2007 at 09:40 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.