Hitachi Vantara Pentaho Community Forums
Results 1 to 8 of 8

Thread: issue with data validation with date conversion mask

  1. #1
    Join Date
    Apr 2012
    Posts
    19

    Default issue with data validation with date conversion mask

    I've got a field with the following format 20120415 yyyyMMdd, but trying to use this as the conversion mask doesn't causes all rows to fail.

    what is the syntax for this field?

  2. #2
    Join Date
    Apr 2008
    Posts
    1,771

    Default

    What is the error message that you have?
    It could be that it fails because some entries are not in the same format.

    Mick

  3. #3
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Maybe it's still a Number of an Integer?

  4. #4
    Join Date
    Apr 2012
    Posts
    19

    Default

    it fails for all records and i tried with a small subset (even a single row) and all the data was formatted correctly

    i even have the "Only numeric date expected" checked and max and min length set to 8 and that processes without error.

    if i enable type validation all i get is
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : Unexpected error
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : org.pentaho.di.core.exception.KettleException:
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : Document Date is invalid
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : Document Date is invalid
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) :
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : at org.pentaho.di.trans.steps.validator.Validator.processRow(Validator.java:155)
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : at org.pentaho.di.trans.step.RunThread.run(RunThread.java:40)
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : at java.lang.Thread.run(Thread.java:722)
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : Caused by: org.pentaho.di.trans.steps.validator.KettleValidatorException: Document Date is invalid
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : at org.pentaho.di.trans.steps.validator.Validator.validateFields(Validator.java:263)
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : at org.pentaho.di.trans.steps.validator.Validator.processRow(Validator.java:127)
    2012/04/18 10:26:04 - Data Validator.0 - ERROR (version 4.2.0-RC1, build 15366 from 2011-06-14 12.04.35 by buildguy) : ... 2 more

    setting logging to row detail doesn't give me any more error information.

    I'm running kettle 4.2.0 RC1

    I can send a simple test case.
    Last edited by tecronin; 04-18-2012 at 11:39 AM.

  5. #5
    Join Date
    Apr 2007
    Posts
    2,009

    Default

    try doing the same but with a select values step. that will give a better error message I think.

  6. #6
    Join Date
    Apr 2012
    Posts
    19

    Default

    the select values doesn't error but doesn't seem to convert the value to a date.

    i configured the meta-data tab, selected my field typed it as date and used the same formatting value and renamed it, but in preview it shows as the original string value...

  7. #7
    Join Date
    Apr 2008
    Posts
    4,689

    Default

    Preview doesn't always show everything...

    Try outputting to a text file, and format the date when it's written to the text file. (eg. yyyy-MM-dd HH:mm:ss)
    I'll bet you get something different.

  8. #8
    Join Date
    Apr 2007
    Posts
    2,009

    Default

    if preview worked, and showed the data, and the field was defined as a date, that means that select values successfully converted those values to a date.
    therefore you need to find which row(s) dont convert. So preview more rows, or just run the whole transformation - it doesnt have to output to a file, it'll bomb once it gets to the failure point.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.