Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Working with KettleVFS in custom step plugin

  1. #1

    Default Working with KettleVFS in custom step plugin

    Hello,

    basically I'm trying to write a custom step plugin which gets a byte array as input (e.g. via the load file content in memory step) and writes that into a file on disc. While I've the plumbing of the code based on the "Serialize to file" step somehow ready, I can't find a way to work with KettleVFS to create an output stream for file writing.

    E.g., I have this:

    ...
    import org.pentaho.di.core.vfs.KettleVFS;
    ...
    data.fos = KettleVFS.getOutputStream(filename, getTransMeta(), false);


    data.fos is of type java.io.OutputStream

    Ecplise shows me this error:
    The type org.apache.commons.vfs.FileObject cannot be resolved. It is indirectly referenced from required .class

    Any ideas? Thanks.

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    We use a slightly modified version of Apache Commons VFS. The library is called libext/pentaho/kettle-vfs-20100924.jar

    You need it in your classpath.

    Matt

  3. #3

    Default

    Thanks! It compiles now. Although I have another problem, namely the resulting file generated by plugin step produces a file which is 6 bytes larger.

    I have a "Load file content into memory" step followed by my plugin step. The actual writing is done with the following method.

    Code:
        private synchronized boolean writeRowToFile(Object[] r)
        {
            try
            {    
                   // Write data to the cube file...
                data.outputRowMeta.writeData(data.dos, r);
            }
            catch(Exception e)
            {
                logError(BaseMessages.getString(PKG, "CubeOutput.Log.ErrorWritingLine")+e.toString()); //$NON-NLS-1$
                return false;
            }
    
            incrementLinesOutput();
            
            return true;
        }
    Where data.dos is of type DataOutputStream, which is instantiated by:

    Code:
                data.fos = KettleVFS.getOutputStream(filename, getTransMeta(), false);
                data.dos = new DataOutputStream(data.fos);
    Comparing the source (a .PNG file actually) and the generated file, I see 5 additonal bytes at the beginning and one additional byte at the end of the generated file.

    Any ideas? Do I have to access the binary content in the row only etc ...? If so, how? I read Chapter 23 of the Pentaho Kettle Solutions book. While I can follow the chapter, my scenario is a bit different.

    Thanks!

    Thomas

  4. #4
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    RowMetaInterface.writeData() is used to write a complete row include length and null state to disk, not just the binary value. So that's where the extra bytes come from.

    try:

    Code:
    data.dos.write(data.outputRowMeta.getBinary(r, index);
    HTH,
    Matt

  5. #5

    Default

    Matt,

    cool! Thanks a lot for your help. Slowly adopting the plugin development stuff.

    Thanks again!

    Thomas

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.