Hitachi Vantara Pentaho Community Forums
Results 1 to 6 of 6

Thread: Need help with Update Slowly Changing Dimension

  1. #1

    Default Need help with Update Slowly Changing Dimension

    I am trying to use the attached transformation to load the type 2 slowly changing dimension and I can not get pass a java.lang.RuntimeException: Error serializing row to byte array.

    Please take a look at my transformation and attached error log to see if I have it configured incorrectly or if I need to download a recent patch. I am currently using Spoon version 3.0 from the 11/14 build.

    Target Table Definition:
    CREATE TABLE student_dim
    (
    student_id INTEGER NOT NULL,
    source_student_id INTEGER NULL,
    source_username VARCHAR(20) NULL,
    last_name VARCHAR(50) NULL,
    first_name VARCHAR(50) NULL,
    middle_name VARCHAR(50) NULL,
    birthdate_id INTEGER NULL,
    gender VARCHAR(20) NULL,
    primary_ethnicty VARCHAR(50) NULL,
    institution VARCHAR(255) NULL,
    row_version TINYINT NOT NULL DEFAULT 1,
    row_load_date TIMESTAMP NOT NULL DEFAULT now(),
    row_last_update_date TIMESTAMP NULL,
    row_effective_date TIMESTAMP NULL,
    row_expiration_date TIMESTAMP NULL
    )
    ;

    Source Table Definition:
    CREATE TABLE student
    (
    student_id INTEGER NOT NULL,
    source_student_id INTEGER NULL,
    source_username VARCHAR(20) NULL,
    last_name VARCHAR(50) NULL,
    first_name VARCHAR(50) NULL,
    middle_name VARCHAR(50) NULL,
    birthdate DATETIME NULL,
    gender VARCHAR(20) NULL,
    primary_ethnicty VARCHAR(50) NULL,
    institution VARCHAR(255) NULL,
    row_load_date TIMESTAMP NOT NULL DEFAULT now(),
    row_last_update_date TIMESTAMP NULL,
    row_effective_date TIMESTAMP NULL,
    row_expiration_date TIMESTAMP NULL
    )
    ;
    Attached Files Attached Files

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    This has come up before.
    Was there any particular reason you turned on lazy conversion in the "Table Input" step?

    Just curious.

  3. #3

    Talking

    I used lazy conversion option with the hope of obtaining a performance gain.

    I unchecked the option and the transformation is running now.

    Thanks,

    Warren

  4. #4
    Join Date
    Dec 2005
    Posts
    531

    Default

    Hi,

    despite having turned off the lazy conversion I'm not able to run the Dimension Lookup Step and get the following error:

    ERROR 11-12 18:58:44,418 (LogWriter.javarintln:400) -dim_lkp_product_id.0 - java.lang.RuntimeException: Error serializing row to byte array
    at org.pentaho.di.core.row.RowMeta.extractData(RowMeta.java:727)
    at org.pentaho.di.trans.steps.dimensionlookup.DimensionLookup.addToCache(DimensionLookup.java:1012)
    at org.pentaho.di.trans.steps.dimensionlookup.DimensionLookup.lookupValues(DimensionLookup.java:338)
    at org.pentaho.di.trans.steps.dimensionlookup.DimensionLookup.processRow(DimensionLookup.java:182)
    at org.pentaho.di.trans.steps.dimensionlookup.DimensionLookup.run(DimensionLookup.java:1243)
    Caused by: java.lang.ClassCastException: java.lang.Long
    at org.pentaho.di.core.row.ValueMeta.writeData(ValueMeta.java:1936)
    at org.pentaho.di.core.row.RowMeta.writeData(RowMeta.java:434)
    at org.pentaho.di.core.row.RowMeta.extractData(RowMeta.java:720)
    ... 4 more

    Enclosed you will find a simple transformation that fails.

    The same transformation runs in a 2.5.x version and it seems like the error only appears when the step tries to insert a line into the table.

    Thanks for any help,
    Ingo
    Attached Files Attached Files

  5. #5
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    The exception basically means that there is some problem with the caching system.
    You are probably running on a 3.0.0 version so I can't really be certain from the stack trace without looking very hard.

    As Jared will have told you by now you really should try one of our nightly builds to get rid of this problem.

    All the best,

    Matt

  6. #6
    Join Date
    Nov 1999
    Posts
    459

    Default

    I filed a bug with an explanation for Ingos case over here:
    http://jira.pentaho.org/browse/PDI-572
    It is solved by using an Oracle database instead of the Generic database driver.

    Cheers,
    Jens

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.