Hitachi Vantara Pentaho Community Forums
Results 1 to 8 of 8

Thread: Bulk Loader.0 java.lang.NullPointerException

  1. #1
    Join Date
    Feb 2018
    Posts
    5

    Default Bulk Loader.0 java.lang.NullPointerException

    Hi there,

    Container:

    sudo docker run -d -p 5432:5432 --name postgres ioinformatics/transmart-postgres
    sudo docker run -d -p 8080:8080 --link postgresostgres ioinformatics/transmart:1.2.4-distributed

    kettle.config on localhost:
    COMMON_DB_NAME=transmart
    COMMON_DB_PORT=5432
    COMMON_DB_SERVER=127.0.0.1
    PSQL_PATH=/usr/bin/psql


    When running the following line in a script on the host:

    $kitchen -norep=Y -file=$kettle_job -paramATA_LOCATION=$data_location -param:STUDY_ID="$STUDY_ID" -param:TOP_NODE="$study_path$STUDY_ID" -param:SORT_DIR=/tmp -param:COLUMN_MAP_FILE=$filename

    whereas
    $kettle_job = create_clinical_data.kjb
    all other variables are set to valid resources as well

    I get:
    2018/01/30 11:40:36 - Bulk Loader.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : Error in step
    2018/01/30 11:40:36 - Bulk Loader.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : java.lang.NullPointerException
    2018/01/30 11:40:36 - Bulk Loader.0 - at org.pentaho.di.trans.steps.pgbulkloader.PGBulkLoader.processRow(PGBulkLoader.java:201)
    2018/01/30 11:40:36 - Bulk Loader.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
    2018/01/30 11:40:36 - Bulk Loader.0 - at java.lang.Thread.run(Thread.java:748)
    2018/01/30 11:40:36 - Bulk Loader.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=1)
    2018/01/30 11:40:36 - map_data_to_std_format - Transformation detected one or more steps with errors.

    Can anyone point me in the right direction?


    Kind regards,

    /Joe

  2. #2
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    How did you configure the PGBulk loader step?
    From the location reported (PGBulkLoader.java:201 ) it suggests that you didn't set a connection.

  3. #3
    Join Date
    Feb 2018
    Posts
    5

    Default

    thnx gutlez,

    the only bulk loader step I found in map_data_to_std_format (https://github.com/transmart/tranSMA...res/Kettle-ETL) is a postgreSQL bulk loader step. However, as far as I can see, the bulkloader step is properly configured (see attached picture).
    error_bulk_loader_sent.txtName:  map_data_to_std_format.jpg
Views: 385
Size:  30.6 KBmap_data_to_std_format.rar
    If I compare the sequence of steps in map_data_to_format with the steps in the logs, it seems to be this one that causes the error. On the other hand there is a switch called Check LOAD_TYPE just prior to the bulk loader step which should redirect to Insert to lt_src_clinical_data instead as I haven't set the parameter LOAD_TYPE when calling the kitchen script. So maybe that's not it after all?

    2018/02/07 09:06:32 - Remove double quotes in data_label, data_value.0 - Finished processing (I=0, O=0, R=15755, W=15755, U=0, E=0)
    2018/02/07 09:06:32 - Remove any duplicates.0 - Finished processing (I=0, O=0, R=15755, W=15755, U=0, E=0)
    2018/02/07 09:06:32 - seq_col not null.0 - Finished processing (I=0, O=0, R=15755, W=15755, U=0, E=0)
    2018/02/07 09:06:32 - Set standard columns.0 - Finished processing (I=0, O=0, R=15755, W=15755, U=0, E=0)
    2018/02/07 09:06:33 - Check LOAD_TYPE.0 - Finished processing (I=0, O=0, R=15755, W=15755, U=0, E=0)
    2018/02/07 09:06:33 - Bulk Loader.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : Error in step
    2018/02/07 09:06:33 - Bulk Loader.0 - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : java.lang.NullPointerException
    2018/02/07 09:06:33 - Bulk Loader.0 - at org.pentaho.di.trans.steps.pgbulkloader.PGBulkLoader.processRow(PGBulkLoader.java:201)
    2018/02/07 09:06:33 - Bulk Loader.0 - at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
    2018/02/07 09:06:33 - Bulk Loader.0 - at java.lang.Thread.run(Thread.java:748)
    2018/02/07 09:06:33 - Bulk Loader.0 - Finished processing (I=0, O=0, R=0, W=0, U=0, E=1)
    2018/02/07 09:06:33 - Check for Remove any Duplicates step.0 - Finished processing (I=0, O=0, R=7, W=7, U=0, E=0)
    2018/02/07 09:06:33 - map_data_to_std_format - ERROR (version 8.0.0.0-28, build 8.0.0.0-28 from 2017-11-05 07.27.50 by buildguy) : Errors detected!

    What am I missing? Is the PGBulk Loader step something that I need to configure elsewhere? Does PDI 8.0.0.0-28 actually work with java 1.8.0_151?

    Can you shed some light on this? It's the first time I am playing with PDI/transmart.


    /joe
    Last edited by joeblow; 02-07-2018 at 05:53 AM.

  4. #4
    Join Date
    Aug 2016
    Posts
    290

    Default

    Wow that is one messy transformation!

  5. #5
    Join Date
    Feb 2018
    Posts
    5

    Default

    tell me about it, its part of the repository that I downloaded from https://github.com/transmart/tranSMART-ETL

  6. #6
    Join Date
    Feb 2018
    Posts
    5

    Default

    ok, I now got pdi 7.1.0 and it just works. Seems to be a bug in pdi 8.0.0.
    Last edited by joeblow; 02-09-2018 at 08:31 AM.

  7. #7
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Not sure if it is a bug in PDI 8, or if it's that someone made a bad assumption and was relying on a bug that was fixed in PDI8.

    Remember that all steps initialize at the BEGINNING on the transformation, not when data hits, so if LOAD_TYPE is null, that might cause the issue.
    The image you supplied is resized by the forum, so is completely unreadable.

  8. #8
    Join Date
    Feb 2018
    Posts
    5

    Default

    The default value of LOAD_TYPE is "I", right? Yeah I realised that the picture is unreadable, hence I added the rar archive which includes the original picture. As I said I am not sure if its a bug, it just seems to be.

    In any case I am happy pdi 7.1.0 works out-of-the-box with that transmart-etl repository.
    Last edited by joeblow; 02-09-2018 at 08:31 AM. Reason: pdi 7.0.1 -> pdi 7.1.0

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.