Hitachi Vantara Pentaho Community Forums
Results 1 to 8 of 8

Thread: Question related 'lazy conversion' on 3.0.0GA

  1. #1

    Default Question related 'lazy conversion' on 3.0.0GA

    Hi, kettle users.

    on 'table input' step,
    I tried 'preview' rows after checking 'enable lazy conversion'.
    I can see 'broken' strings.
    but I tried after un-checking 'enable lazy conversion'
    I can see normal strings.

    Is it 'Lazy conversion'??

    how can I get more information about 'Lazy Conversion'?

    thanks.

    regards,
    yw kim

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    In the change log of 3.0 there's a small part on lazy conversion. In short it means that the data is not converted until the need arrives, if e.g. you would write from text file to text file and not change anything to a field it would not "instantiated".

    What you see in the preview are the raw bytes (well a human readable text of the address of the bytes). It's probably a beauty mistake... raise a JIRA.

    Regards,
    Sven

  3. #3
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    You have to love these posts: "I have a certain problem but I'm not going to give you any details about what it is!"

    There should be no difference in previewing data using normal and lazy conversion mode.
    If there is, it's obviously a bug. (if it's a problem at all)

    My bet is that it's a codepage problem.

    Cheers,
    Matt

  4. #4

    Question

    The below explanation given is not clear to me. Can some one help me understand in more deetail as to how this setting works(Lazzy conversion).

    In one of the transformation, upon enabling this setting i get the below errors. And after removing that setting the errors no longer appear.

    Iam really curious to understand how , where and when to use this option("Lazzy Conversion").

    2007/11/28 14:44:32 - Database lookup.0 - ERROR (version 3.0.0, build 500 from 2007/11/14 14:59:11) : Because of an error, this step can't continue:
    2007/11/28 14:44:32 - Database lookup.0 - ERROR (version 3.0.0, build 500 from 2007/11/14 14:59:11) : offending row : [CENTER_ID String(90)<binary-string>]
    2007/11/28 14:44:32 - Database lookup.0 - ERROR (version 3.0.0, build 500 from 2007/11/14 14:59:11) :
    2007/11/28 14:44:32 - Database lookup.0 - ERROR (version 3.0.0, build 500 from 2007/11/14 14:59:11) : Error setting value #1 [CENTER_ID String(90)<binary-string>] on prepared statement (String)
    2007/11/28 14:44:32 - Database lookup.0 - ERROR (version 3.0.0, build 500 from 2007/11/14 14:59:11) : org.pentaho.di.core.exception.KettleValueException:
    2007/11/28 14:44:32 - Database lookup.0 - ERROR (version 3.0.0, build 500 from 2007/11/14 14:59:11) : CENTER_ID String(90)<binary-string> : There was a data type error: the data type of java.lang.String object [015556] does not correspond to value meta [String(90)<binary-string>]
    2007/11/28 14:44:32 - Database lookup.0 - ERROR (version 3.0.0, build 500 from 2007/11/14 14:59:11) :
    2007/11/28 14:44:32 - Database lookup.0 - ERROR (version 3.0.0, build 500 from 2007/11/14 14:59:11) :
    2007/11/28 14:44:32 - Database lookup.0 - ERROR (version 3.0.0, build 500 from 2007/11/14 14:59:11) : CENTER_ID String(90)<binary-string> : There was a data type error: the data type of java.lang.String object [015556] does not correspond to value meta [String(90)<binary-string>]
    2007/11/28 14:44:32 - MJS1.0 - Finished processing (I=0, O=0, R=36, W=35, U=0, E=0)


    Regards,
    Raju

  5. #5
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    I added the "Lazy conversion" option in "Table Input" because even though not all database support it yet, it can be interesting to minimize serialization/de-serialization efforts in certain configurations like:
    - Sorting a lot of rows
    - flat file exports
    - master-to-slave data transfers in a clustered configuration
    - slave-to-slave data transfers in a re-partitioned clustered configuration

    If you don't know what it is good for, I suggest you let it be for the time being ;-)

    Matt

  6. #6

    Default

    Thanks Matt.

    I have a fundamental question regarding the same topic. I was looking at the documentation provided by pentaho, it just has the below two lines and nothing else.

    "Lazy conversion will avoid unnecessary data type conversions and can
    result in a significant performance improvements. Check to enable."

    And it has no mention about the features you mentioned. Am i looking the right place for understanding each of these components or features(is there some thing iam missing?). Is there any other place to get details about such detail aspects. Or is it left to individuals interpretation and exploration abilities.

    (If documentation of these aspects are still not available)While developing these features, you would have executed lot of samples to test the functionality, providing those samples(with the release) would be a great help. doing so, the hard work put in to build a feature, could be appretiated by the entire community. And would help in a great way for newbie's like me

    Neverthless, what you guys are doing for the Open Source community is amazing. Keep the good work going..:-)

    Regards,
    Raju


  7. #7
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    More explanations are here:

    http://www.ibridge.be/?p=63
    http://www.ibridge.be/?p=78

    The information is copied here and there, but it appeared first on my blog.

    Matt

  8. #8

    Default

    Awesome!!! Thanks Matt, for pointing me to the right location. Now things are clear.

    Regards,
    Raju

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.