Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Cobol Input Step

  1. #1
    rpm2 Guest

    Default Cobol Input Step

    Hello kettle-developers - first post

    I have been working on a Cobol Input step for the last few months and
    aside from minor issues I have it working -granted it still needs a
    lot of work before I would think it's worthy for others.

    It's basically is a tool for reading a flat file or dataset that is
    generated on a z/OS IBM mainframe based on a Cobol copybook. I think
    it works on AS/400 computers as well. Or maybe any environment that
    creates a file from a Cobol program.

    I have one minor issue. I created a feature where I can save the
    original native data (binary record) as a byte field in Kettle.
    Basically I save the original record in the metadata stream as a field
    so I can rewrite the field later as a record. Its kind of lazy
    conversion (sort of). I wanted a way to filter millions of records to
    create test data for myself. (I'm still working on a filter feature)

    My problem is if the data I'm reading in byte[0] contains x'00' null
    the entire byte array field is treated as null by Kettle. In my
    dataset the x'00' is just the high order byte in an integer or packed
    decimal.

    Is there a work around for this? I proved this problem by making the
    first byte not x'00' and it worked. The byte record was maintained and
    not treated as null. Also if my first field is Alpha it also works
    fine.

    Another request or question I have is how should I name my packages. I
    would like to store this plugin in a SVN project. Should I use Google
    Code repository, Sourceforge. Any recommendations.

    I would even like to give this to the Pentaho project if any interest.
    I feel no ownership for it. I just wanted to learn how to build a
    plugin and this Cobol Input step is something I needed for work.

    Believe me I got my payback with Kettle many times over!
    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

  2. #2
    DEinspanjer Guest

    Default Re: Cobol Input Step

    svn://source.pentaho.org/svnkettleroot/plugins/ is a good place if you
    don't already have a home for it. I believe you can get commit access
    to that directory fairly easily, but I'm not positive.

    I suggest making sure your plugin package name starts with "plugin."
    and that you use the plugin annotation on your Meta class. That will
    make it very easy for someone to just drop your jar into their
    classpath and have it work.

    -Daniel

    On Feb 2, 11:47

  3. #3
    rpm2 Guest

    Default Re: Cobol Input Step

    I wanted to reply to my own message.

    I wonder if what I'm seeing with my x'00' null issue is just Kettle
    trying to display the byte as a String when I see it in the Preview
    window.

    I'm going to retest this issue by outputing the byte array untouched
    to an output file which is what I wanted to do anyway.


    On Feb 2, 10:47

  4. #4
    Matt Casters Guest

    Default Re: Cobol Input Step

    For persistence (XML) and visualization, try to encode the binary data in an encoding like Base64.

    Good luck!
    Matt
    ____________________________________________
    Matt Casters
    Chief Data Integration - Kettle founder
    Pentaho, Open Source Business Intelligence
    http://www.pentaho.org -- mcasters (AT) pentaho (DOT) org
    Tel. +32 (0) 486 97 29 37

    On Tuesday 03 February 2009 14:12:37 rpm2 wrote:
    >
    > I wanted to reply to my own message.
    >
    > I wonder if what I'm seeing with my x'00' null issue is just Kettle
    > trying to display the byte as a String when I see it in the Preview
    > window.
    >
    > I'm going to retest this issue by outputing the byte array untouched
    > to an output file which is what I wanted to do anyway.
    >
    >
    > On Feb 2, 10:47 pm, rpm2 <rmoo... (AT) wi (DOT) rr.com> wrote:
    > > Hello kettle-developers - first post
    > >
    > > I have been working on a Cobol Input step for the last few months and
    > > aside from minor issues I have it working -granted it still needs a
    > > lot of work before I would think it's worthy for others.
    > >
    > > It's basically is a tool for reading a flat file or dataset that is
    > > generated on a z/OS IBM mainframe based on a Cobol copybook. I think
    > > it works on AS/400 computers as well. Or maybe any environment that
    > > creates a file from a Cobol program.
    > >
    > > I have one minor issue. I created a feature where I can save the
    > > original native data (binary record) as a byte field in Kettle.
    > > Basically I save the original record in the metadata stream as a field
    > > so I can rewrite the field later as a record. Its kind of lazy
    > > conversion (sort of). I wanted a way to filter millions of records to
    > > create test data for myself. (I'm still working on a filter feature)
    > >
    > > My problem is if the data I'm reading in byte[0] contains x'00' null
    > > the entire byte array field is treated as null by Kettle. In my
    > > dataset the x'00' is just the high order byte in an integer or packed
    > > decimal.
    > >
    > > Is there a work around for this? I proved this problem by making the
    > > first byte not x'00' and it worked. The byte record was maintained and
    > > not treated as null. Also if my first field is Alpha it also works
    > > fine.
    > >
    > > Another request or question I have is how should I name my packages. I
    > > would like to store this plugin in a SVN project. Should I use Google
    > > Code repository, Sourceforge. Any recommendations.
    > >
    > > I would even like to give this to the Pentaho project if any interest.
    > > I feel no ownership for it. I just wanted to learn how to build a
    > > plugin and this Cobol Input step is something I needed for work.
    > >
    > > Believe me I got my payback with Kettle many times over!
    > >

    >
    >



    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.