Hitachi Vantara Pentaho Community Forums
Results 1 to 7 of 7

Thread: Truncate String

  1. #1

    Default Truncate String

    hi everybody

    i have a little problem while i do a transformation



    i have a string field named DESCRIZ string(40) and with SelectedValues step i rename it into ART_DESCRIZIONE1 string(30), but i see that some records are not trunced

    why this happens?

    if i specify a lesser length into SelectValues step, doesnt Kettle truncate the field? how can i truncate the string i need to be shorter?



    regards
    Antonio

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    Ooh... one of my favorite questions Have a look at http://wiki.pentaho.org/display/EAI/Advanced+Users+FAQ for the answer.

    Kettle does not limit the values to the size of the metadata, conversion is only done at the very last stages.

    If you really want to cut... you can use e.g. javascript step or use the trimcut step that can be downloaded at http://wiki.pentaho.org/display/EAI/...ation+Plug-Ins (trimcut only if you're using v3 of Kettle)

    Regards,
    Sven

  3. #3

    Default

    i checked some rows he wrote in the destination DB

    i see that the strings are trunced in automatic




    any way... there is a problem now..

    the trasformation is ok, when i VERIFY it, there is no error...

    but after somer rows it crashes


    the input database is a MDBaccessDB

    the output database is a ACUCoboldDB, to which i refer through an access file, made with linked tables


    in both connection option i set
    Database type MSAccess - useCursorFetch - true
    Database type MSAccess - defaultFetchSize - 10


    in table output step i set Commit Size = 10






    this is the text of the error log file


    #
    # An unexpected error has been detected by Java Runtime Environment:
    #
    # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x1af03fd8, pid=3360, tid=2544
    #
    # Java VM: Java HotSpot(TM) Client VM (1.6.0_03-b05 mixed mode)
    # Problematic frame:
    # C [acuv32.dll+0x3fd8]
    #
    # If you would like to submit a bug report, please visit:
    # http://java.sun.com/webapps/bugreport/crash.jsp
    #

    --------------- T H R E A D ---------------

    Current thread (0x17afa000): JavaThread "Table output articoli.0 (Thread-11)" [_thread_in_native, id=2544]

    siginfo: ExceptionCode=0xc0000005, reading address 0x1741d000

    Registers:
    EAX=0x000000ff, EBX=0x1a4fa4b0, ECX=0x00000027, EDX=0x00000000
    ESP=0x1fb0f838, EBP=0x1a4fa4c0, ESI=0x1741d000, EDI=0x1a4f6ad8
    EIP=0x1af03fd8, EFLAGS=0x00010207

    Top of Stack: (sp=0x1fb0f838)
    0x1fb0f838: 0000000d 1a4b7ee0 1fb0f8d8 1a4b7f38
    0x1fb0f848: 00000000 000000ff ffffffff 7c9206eb
    0x1fb0f858: 7c3416b3 00290000 00000000 7c3416b8
    0x1fb0f868: 1fb0f8a8 1af75230 ffffffff 1af32897
    0x1fb0f878: 1a4b7ee0 00000000 1a4b7ee0 1a2e30f0
    0x1fb0f888: 7c34240d 7c37a2a8 74615a16 745e1367
    0x1fb0f898: 1af0a40b 00000000 1a4b7ee0 1a2e30f0
    0x1fb0f8a8: 1fb0fbe4 1af79998 ffffffff 1af0aba8

    Instructions: (pc=0x1af03fd8)
    0x1af03fc8: 38 e8 d2 13 02 00 66 8b c6 eb 35 8b c1 c1 e9 02
    0x1af03fd8: f3 a5 8b c8 83 e1 03 f3 a4 eb 91 66 83 ff 01 75


    Stack: [0x1fac0000,0x1fb10000), sp=0x1fb0f838, free space=318k
    Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
    C [acuv32.dll+0x3fd8]

    Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
    j sun.jdbc.odbc.JdbcOdbc.execute(J[B)V+0
    j sun.jdbc.odbc.JdbcOdbc.SQLExecute(J)Z+46
    j sun.jdbc.odbc.JdbcOdbcPreparedStatement.execute()Z+47
    j sun.jdbc.odbc.JdbcOdbcPreparedStatement.executeUpdate()I+28
    j be.ibridge.kettle.core.database.Database.insertRow(Ljava/sql/PreparedStatement;Z)Z+90
    j be.ibridge.kettle.trans.step.tableoutput.TableOutput.writeToTable(Lbe/ibridge/kettle/core/RowZ+656
    j be.ibridge.kettle.trans.step.tableoutput.TableOutput.processRow(Lbe/ibridge/kettle/trans/step/StepMetaInterface;Lbe/ibridge/kettle/trans/step/StepDataInterfaceZ+29
    j be.ibridge.kettle.trans.step.tableoutput.TableOutput.run()V+15
    v ~StubRoutines::call_stub

    --------------- P R O C E S S ---------------

    Java Threads: ( => current thread )
    =>0x17afa000 JavaThread "Table output articoli.0 (Thread-11)" [_thread_in_native, id=2544]
    0x172e8800 JavaThread "Select values.0 (Thread-10)" [_thread_blocked, id=720]
    0x17aff000 JavaThread "Modified Java Script Value.0 (Thread-9)" [_thread_blocked, id=2636]
    0x17b00000 JavaThread "int2string.0 (Thread-8)" [_thread_blocked, id=2012]
    0x16bca400 JavaThread "rinomina campi.0 (Thread-7)" [_thread_blocked, id=3136]
    0x16bcd000 JavaThread "Table input articoli.0 (Thread-6)" [_thread_blocked, id=3132]
    0x17ac1c00 JavaThread "Timer-0" [_thread_blocked, id=3748]
    0x17ad0800 JavaThread "org.apache.commons.vfs.cache.SoftRefFilesCache$SoftRefReleaseThread" daemon [_thread_blocked, id=1252]
    0x16e9e000 JavaThread "Thread-1" [_thread_blocked, id=2280]
    0x16b7e400 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=972]
    0x16b79400 JavaThread "CompilerThread0" daemon [_thread_blocked, id=568]
    0x16b78400 JavaThread "Attach Listener" daemon [_thread_blocked, id=3268]
    0x16b77400 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=3664]
    0x16b68c00 JavaThread "Finalizer" daemon [_thread_blocked, id=1052]
    0x16b64800 JavaThread "Reference Handler" daemon [_thread_blocked, id=3556]
    0x00298400 JavaThread "main" [_thread_in_native, id=3936]

    Other Threads:
    0x16b60000 VMThread [id=1348]
    0x16ba0c00 WatcherThread [id=2004]

    VM state:not at safepoint (normal execution)

    VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])
    [0x00297138/0x000006d0] Heap_lock - owner thread: 0x172e8800

    Heap
    def new generation total 960K, used 95K [0x02990000, 0x02a90000, 0x03d40000)
    eden space 896K, 3% used [0x02990000, 0x02997ed8, 0x02a70000)
    from space 64K, 100% used [0x02a80000, 0x02a90000, 0x02a90000)
    to space 64K, 0% used [0x02a70000, 0x02a70000, 0x02a80000)
    tenured generation total 7340K, used 6649K [0x03d40000, 0x0446b000, 0x12990000)
    the space 7340K, 90% used [0x03d40000, 0x043be588, 0x043be600, 0x0446b000)
    compacting perm gen total 15616K, used 15423K [0x12990000, 0x138d0000, 0x16990000)
    the space 15616K, 98% used [0x12990000, 0x1389fe20, 0x138a0000, 0x138d0000)
    No shared spaces configured.

    Dynamic libraries:
    0x00400000 - 0x00423000 c:\windows\system32\java.exe
    0x7c910000 - 0x7c9c6000 C:\WINDOWS\system32\ntdll.dll
    0x7c800000 - 0x7c900000 C:\WINDOWS\system32\kernel32.dll
    0x77f40000 - 0x77feb000 C:\WINDOWS\system32\ADVAPI32.dll
    0x77da0000 - 0x77e32000 C:\WINDOWS\system32\RPCRT4.dll
    0x77f10000 - 0x77f21000 C:\WINDOWS\system32\Secur32.dll
    0x7c340000 - 0x7c396000 C:\Programmi\Java\jre1.6.0_03\bin\msvcr71.dll
    0x6d7c0000 - 0x6da0a000 C:\Programmi\Java\jre1.6.0_03\bin\client\jvm.dll
    0x7e390000 - 0x7e420000 C:\WINDOWS\system32\USER32.dll
    0x77e40000 - 0x77e87000 C:\WINDOWS\system32\GDI32.dll
    0x76b00000 - 0x76b2e000 c:\windows\system32\WINMM.dll
    0x6d310000 - 0x6d318000 C:\Programmi\Java\jre1.6.0_03\bin\hpi.dll
    0x76bb0000 - 0x76bbb000 c:\windows\system32\PSAPI.DLL
    0x6d770000 - 0x6d77c000 C:\Programmi\Java\jre1.6.0_03\bin\verify.dll
    0x6d3b0000 - 0x6d3cf000 C:\Programmi\Java\jre1.6.0_03\bin\java.dll
    0x6d7b0000 - 0x6d7bf000 C:\Programmi\Java\jre1.6.0_03\bin\zip.dll
    0x17470000 - 0x174bd000 C:\Documents and Settings\Administrator\Impostazioni locali\Temp\swt-win32-3333.dll
    0x774b0000 - 0x775ed000 C:\WINDOWS\system32\ole32.dll
    0x77be0000 - 0x77c38000 C:\WINDOWS\system32\msvcrt.dll
    0x773a0000 - 0x774a3000 C:\WINDOWS\WinSxS\X86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.2982_x-ww_ac3f9c03\COMCTL32.dll
    0x77e90000 - 0x77f06000 C:\WINDOWS\system32\SHLWAPI.dll
    0x76360000 - 0x763aa000 C:\WINDOWS\system32\comdlg32.dll
    0x7c9d0000 - 0x7d1ed000 C:\WINDOWS\system32\SHELL32.dll
    0x770f0000 - 0x7717b000 C:\WINDOWS\system32\OLEAUT32.dll
    0x76340000 - 0x7635d000 c:\windows\system32\IMM32.dll
    0x74d20000 - 0x74d8b000 c:\windows\system32\USP10.dll
    0x77180000 - 0x77227000 C:\WINDOWS\system32\WININET.dll
    0x77a50000 - 0x77ae5000 C:\WINDOWS\system32\CRYPT32.dll
    0x77af0000 - 0x77b02000 C:\WINDOWS\system32\MSASN1.dll
    0x5d4d0000 - 0x5d56a000 C:\WINDOWS\system32\comctl32.dll
    0x5b180000 - 0x5b1b8000 c:\windows\system32\uxtheme.dll
    0x76330000 - 0x76335000 c:\windows\system32\msimg32.dll
    0x18620000 - 0x18633000 C:\Documents and Settings\Administrator\Impostazioni locali\Temp\swt-gdip-win32-3333.dll
    0x4ebd0000 - 0x4ed73000 C:\WINDOWS\WinSxS\x86_Microsoft.Windows.GdiPlus_6595b64144ccf1df_1.0.2600.2180_x-ww_522f9f82\gdiplus.dll
    0x74c10000 - 0x74c3c000 c:\windows\system32\oleacc.dll
    0x76030000 - 0x76095000 c:\windows\system32\MSVCP60.dll
    0x76f90000 - 0x7700f000 c:\windows\system32\CLBCATQ.DLL
    0x77010000 - 0x770e2000 c:\windows\system32\COMRes.dll
    0x77bd0000 - 0x77bd8000 C:\WINDOWS\system32\VERSION.dll
    0x7e210000 - 0x7e37f000 C:\WINDOWS\system32\shdocvw.dll
    0x76890000 - 0x76913000 C:\WINDOWS\system32\CRYPTUI.dll
    0x76bf0000 - 0x76c1e000 C:\WINDOWS\system32\WINTRUST.dll
    0x76c50000 - 0x76c78000 C:\WINDOWS\system32\IMAGEHLP.dll
    0x5bc70000 - 0x5bcc4000 C:\WINDOWS\system32\NETAPI32.dll
    0x76f20000 - 0x76f4d000 C:\WINDOWS\system32\WLDAP32.dll
    0x77b10000 - 0x77b32000 C:\WINDOWS\system32\appHelp.dll
    0x75e40000 - 0x75ef0000 c:\windows\system32\SXS.DLL
    0x20000000 - 0x202d5000 c:\windows\system32\xpsp2res.dll
    0x7d1f0000 - 0x7d4ae000 c:\windows\system32\msi.dll
    0x778f0000 - 0x779e7000 c:\windows\system32\SETUPAPI.dll
    0x7df20000 - 0x7dfc0000 C:\WINDOWS\system32\urlmon.dll
    0x19aa0000 - 0x19b2c000 C:\WINDOWS\system32\shdoclc.dll
    0x75d50000 - 0x75de1000 c:\windows\system32\mlang.dll
    0x7dbe0000 - 0x7ded6000 C:\WINDOWS\system32\mshtml.dll
    0x74650000 - 0x74677000 C:\WINDOWS\system32\msls31.dll
    0x66d50000 - 0x66d5c000 c:\windows\system32\ImgUtil.dll
    0x5e750000 - 0x5e75c000 C:\WINDOWS\system32\pngfilt.dll
    0x6d3f0000 - 0x6d3fd000 C:\Programmi\Java\jre1.6.0_03\bin\JdbcOdbc.dll
    0x745e0000 - 0x7461d000 c:\windows\system32\ODBC32.dll
    0x1a210000 - 0x1a228000 c:\windows\system32\odbcint.dll
    0x71a30000 - 0x71a47000 C:\WINDOWS\system32\WS2_32.dll
    0x71a20000 - 0x71a28000 C:\WINDOWS\system32\WS2HELP.dll
    0x62190000 - 0x621ab000 c:\windows\system32\odbccp32.dll
    0x4dda0000 - 0x4dde4000 C:\WINDOWS\system32\odbcjt32.dll
    0x1b000000 - 0x1b170000 C:\WINDOWS\system32\msjet40.dll
    0x1b5d0000 - 0x1b665000 C:\WINDOWS\system32\mswstr10.dll
    0x5d570000 - 0x5d57f000 C:\WINDOWS\system32\odbcji32.dll
    0x1a490000 - 0x1a49d000 C:\WINDOWS\system32\msjter40.dll
    0x1a620000 - 0x1a64c000 C:\WINDOWS\system32\MSJINT40.DLL
    0x1af00000 - 0x1af9d000 c:\g2z_cln\bin\acuv32.dll
    0x1a5d0000 - 0x1a5f7000 c:\g2z_cln\bin\skeylink.dll
    0x71a50000 - 0x71a5a000 C:\WINDOWS\system32\WSOCK32.dll
    0x7c3e0000 - 0x7c45a000 c:\g2z_cln\bin\acuv.dll
    0x1fb10000 - 0x1fb4a000 C:\WINDOWS\system32\msjtes40.dll
    0x1a650000 - 0x1a65b000 c:\windows\system32\VBAJET32.DLL
    0x1ff50000 - 0x1ffb2000 c:\windows\system32\expsrv.dll

    VM Arguments:
    jvm_args: -Xmx256m -Djava.library.path=libswt\win32\ -DKETTLE_HOME= -DKETTLE_REPOSITORY= -DKETTLE_USER= -DKETTLE_PASSWORD=
    java_command: be.ibridge.kettle.spoon.Spoon
    Launcher Type: SUN_STANDARD

    Environment Variables:
    JAVA_HOME=C:\Programmi\Java\jre1.6.0_03
    CLASSPATH=.;lib\kettle.jar;libext\CacheDB.jar;libext\edtftpj-1.5.4.jar;libext\jackcess-1.1.5.jar;libext\jakarta-oro-2.0.8.jar;libext\javadbf.jar;libext\js.jar;
    libext\jsch-0.1.24.jar;libext\jug-lgpl-2.0.0.jar;libext\jxl.jar;libext\log4j-1.2.8.jar;libext\mail.jar;libext\JDBC\db2jcc.jar;libext\JDBC\db2jcc_license_cu.jar;
    libext\JDBC\derbyclient.jar;libext\JDBC\h2.jar;libext\JDBC\hsqldb.jar;libext\JDBC\ifxjdbc.jar;libext\JDBC\iijdbc.jar;libext\JDBC\interclient.jar;
    libext\JDBC\jaybird-full-2.1.0.jar;libext\JDBC\jconn2.jar;libext\JDBC\jt400.jar;libext\JDBC\jtds-1.2.jar;libext\JDBC\mysql-connector-java-3.1.14-bin.jar;
    libext\JDBC\nzjdbc.jar;libext\JDBC\ojdbc14.jar;libext\JDBC\orai18n.jar;libext\JDBC\postgresql-8.1-407.jdbc3.jar;libext\JDBC\rdbthin.jar;libext\JDBC\sapdbc.jar;
    libext\JDBC\SQLBaseJDBC.jar;libext\JDBC\sqlitejdbc-v032-nested.jar;libext\JDBC\unijdbc.jar;libext\JDBC\xdbjdbc.jar;libext\webservices\jsr173_api.jar;
    libext\webservices\qname.jar;libext\webservices\wsdl4j.jar;libext\webservices\wstx-asl-3.2.0.jar;libext\commons\commons-codec-1.3.jar;
    libext\commons\commons-collections-3.1.jar;libext\commons\commons-dbcp-1.2.1.jar;libext\commons\commons-fileupload-1.0.jar;libext\commons\commons-httpclient-3.0.1.jar;
    libext\commons\commons-lang-2.2.jar;libext\commons\commons-logging-1.1.jar;libext\commons\commons-net-1.4.1.jar;libext\commons\commons-pool-1.3.jar;
    libext\commons\commons-vfs-1.0.jar;libext\web\activation.jar;libext\web\jetty-6.1.2rc1.jar;libext\web\jetty-util-6.1.2rc1.jar;libext\web\servlet-api-2.5-6.0.2.jar;
    libext\web\simple-jndi-0.11.1.jar;libext\pentaho\jcommon-1.0.8.jar;libext\pentaho\libformula-0.1.3.jar;libext\pentaho\pentaho-1.2.0.jar;libswt\runtime.jar;libswt\jface.jar;
    libswt\common.jar;libswt\commands.jar;libswt\win32\swt.jar
    PATH=C:\Programmi\QuickTime\QTSystem\;c:\windows\system32
    USERNAME=Administrator
    OS=Windows_NT
    PROCESSOR_IDENTIFIER=x86 Family 6 Model 13 Stepping 8, GenuineIntel



    --------------- S Y S T E M ---------------

    OS: Windows XP Build 2600 Service Pack 2

    CPU:total 1 (1 cores per cpu, 1 threads per core) family 6 model 13 stepping 8, cmov, cx8, fxsr, mmx, sse, sse2

    Memory: 4k page, physical 1038768k(404388k free), swap 2499760k(1959948k free)

    vm_info: Java HotSpot(TM) Client VM (1.6.0_03-b05) for windows-x86, built on Sep 24 2007 22:24:33 by "java_re" with unknown MS VC++:1310




    please help me, i'm goin crazy...

    thanks
    antonio
    Last edited by mrtyler; 12-12-2007 at 07:54 AM.

  4. #4
    Join Date
    May 2006
    Posts
    4,882

    Default

    This is not a Kettle problem... it's a driver problem while outputting the rows it seems.
    Hardly anyone gets out of those besides upgrading the driver or moving to another way of accessing the database.

    Regards,
    Sven

  5. #5

    Default

    i solved the crash using another DNS


    another question i have is
    the destination DB has 226 fields

    but the fields i need to insert into are just 11

    i made ModifiedJavaScript create the other fields

    but this slow down the trasformation


    it is possible to generate query with TableOutput that insert only the fields that i need?

    i cannot drop the other fields because the app needs the table as it is...


    thanks
    antonio

  6. #6
    Join Date
    May 2006
    Posts
    4,882

    Default

    extra columns can't be dropped yet... there's a JIRA open for it (already since a year or so).

    Regards,
    Sven

  7. #7
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    I'm sympathetic to this "Table Output" cause, but we just didn't find the time yet.
    Same for the "Table Input" case of doing explicit and strict tables, field names, data types and join specifications for the suspicious people among us.

    Matt

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.