Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: unexplainable behavior of a transformation based on the context it's called in

  1. #1

    Default unexplainable behavior of a transformation based on the context it's called in

    Hello!

    I wrote a transformation "t_calculate_checksum" that loads the content of a file into memory, calculates the (MD5) checksum of it, compares it to other checksums in a DB-table and decides whether to go on (checksum not found = new file) or stop the whole thing.
    This transformation gets called in a Job called "j_preparations" that is the place where some - exactly! - preparations are done before the actual load (of data into a DB) is started. This Job then gets called by a "main"-Job, the root, the one controlling the whole flow/load. (so: main --> j_preparations --> t_calculate_checksum)

    So far, so good. The problem now is that t_calculate_checksum only seems to work correctly if I call j_preparations but not if the whole thing is run (by calling main). This is the log I get when calling main:
    2014/03/28 09:15:00 - j_preparations - Starting entry [Log - Start Job]
    2014/03/28 09:15:00 - JOB STARTED - j_preparations - parameter FileName = EH580_FAKTLESK.xml
    2014/03/28 09:15:00 - j_preparations - Starting entry [Calculate Checksum]
    2014/03/28 09:15:00 - Calculate Checksum - Loading transformation from repository [t_calculate_checksum] in directory [/]
    2014/03/28 09:15:01 - t_calculate_checksum - Dispatching started for transformation [t_calculate_checksum]
    2014/03/28 09:15:01 - Store result in database.0 - Connected to database [PG_kdl_owner] (commit=1000)
    2014/03/28 09:15:01 - j_preparations - Starting entry [Success]
    2014/03/28 09:15:01 - j_preparations - Finished job entry [Success] (result=[true])
    2014/03/28 09:15:01 - j_preparations - Finished job entry [Calculate Checksum] (result=[true])
    2014/03/28 09:15:01 - j_preparations - Finished job entry [Log - Start Job] (result=[true])

    And this is what I get when just calling j_preparations (the bold rows are missing in the log above):
    2014/03/28 09:35:38 - j_preparations - Starting entry [Log - Start Job]
    2014/03/28 09:35:38 - JOB STARTED - j_preparations - parameter FileName = EH580_FAKTLESK.xml
    2014/03/28 09:35:38 - j_preparations - Starting entry [Calculate Checksum]
    2014/03/28 09:35:38 - Calculate Checksum - Loading transformation from repository [t_calculate_checksum] in directory [/]
    2014/03/28 09:35:39 - t_calculate_checksum - Dispatching started for transformation [t_calculate_checksum]
    2014/03/28 09:35:39 - Store result in database.0 - Connected to database [PG_kdl_owner] (commit=1000)
    2014/03/28 09:35:39 - Load file content in memory.0 - Finished processing (I=1, O=0, R=0, W=1, U=0, E=0)
    2014/03/28 09:35:39 - Calculate checksum.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    2014/03/28 09:35:39 - Lookup of checksum.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    2014/03/28 09:35:39 - Filter rows.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    2014/03/28 09:35:39 - Get Variables.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    2014/03/28 09:35:39 - Get execution date.0 - Finished processing (I=0, O=0, R=1, W=1, U=0, E=0)
    2014/03/28 09:35:39 - Store result in database.0 - Finished processing (I=0, O=1, R=1, W=1, U=0, E=0)

    2014/03/28 09:35:39 - j_preparations - Starting entry [Success]
    2014/03/28 09:35:39 - j_preparations - Finished job entry [Success] (result=[true])
    2014/03/28 09:35:39 - j_preparations - Finished job entry [Calculate Checksum] (result=[true])
    2014/03/28 09:35:39 - j_preparations - Finished job entry [Log - Start Job] (result=[true])

    So it looks like the transaction gets loaded and the DB connection opened but then nothing happens and control goes back to the Job. Do you have any explanation for this, or any idea what might be wrong? I looked through it several times now, changed a thing here and another there but I don't seem to get somewhere.
    I'll attach a picture of both the Job j_preparations and the transformation t_calculate_checksum. Sorry for not tagging this post but honestly - I wouldn't know what tag would suit it.
    Name:  j_preparations.png
Views: 24
Size:  24.0 KBName:  t_calculate_checksum.jpg
Views: 24
Size:  15.6 KB

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Remove that first step (Write to log) from your transformation and try again.
    So long, and thanks for all the fish.

  3. #3

    Default

    Yes, saw that as well - but that's not it. I moved the "Write to log" step after "Load file content in memory" and it has no effect on the problem.

  4. #4

    Default

    I found it!

    I got the idea to make an exact copy of the "main"-job, so I got to work - copied all the entries and stuff. When copying the parameter being passed on when main calls "j_preparations", I noticed there was a space before the value (see picture, the small blue part). Simply removing this space did the work!
    Thanks to all of you who gave this problem a thought, it's much appreciated. Have a nice weekend everyone (I myself will leave work in less than 30 minutes, now as a happy litte developer)!

    Name:  solution.jpg
Views: 18
Size:  20.5 KB

  5. #5
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Can you strip the database steps from your scenario so that I can run it on my machine in order to reproduce the phenomenon?

    PS: Please ignore, I posted without seeing your previous post.
    So long, and thanks for all the fish.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.