Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Kettle log errors - Step works nonetheless

  1. #1
    Join Date
    Oct 2006
    Posts
    18

    Question Kettle log errors - Step works nonetheless

    A Shell script step calls the program 'osmosis' which loads an (OpenStreetMap) .osm (format XML) file to postgresql (version 11.x).
    It works - the result is exactly what I want.
    However, when run, the log just shows a series of ERROR messages at this step. (See below, in the middle of the log lines.) Also, subsequent job steps provide my desired results without errors.
    I am ignoring this, but any idea what might be causing it?
    Thanks.
    RCD
    =================== LOG ================================
    2019/05/19 23:00:22 - Spoon - Starting job...
    2019/05/19 23:00:22 - Load Larabanga OSM - Start of job execution
    2019/05/19 23:00:22 - Load Larabanga OSM - Starting entry [DATA_HOME=/opt/pentaho/laradata]
    2019/05/19 23:00:22 - Load Larabanga OSM - Starting entry [Get Geofabrik Ghana]
    2019/05/19 23:00:22 - Get Geofabrik Ghana - Start of HTTP job entry.
    2019/05/19 23:00:22 - Get Geofabrik Ghana - Connecting to URL: http://download.geofabrik.de/africa/...latest.osm.pbf
    2019/05/19 23:00:22 - Get Geofabrik Ghana - Resource type: application/octet-stream, last modified on: Sun May 19 01:01:58 CEST 2019.
    2019/05/19 23:00:26 - Get Geofabrik Ghana - Finished writing 41922383 bytes to result file [/opt/pentaho/laratmp/ghana.osm.pbf]
    2019/05/19 23:00:26 - Load Larabanga OSM - Starting entry [..........]
    2019/05/19 23:00:26 - Ghana pbf to Larabanga osm - Running on platform : Linux
    2019/05/19 23:00:26 - Ghana pbf to Larabanga osm - Executing command : /tmp/kettle_205b0c0b-7a79-11e9-a189-3bd5352ed62bshell
    2019/05/19 23:00:27 - Load Larabanga OSM - Starting entry [......... OSM exists]
    2019/05/19 23:00:27 - Load Larabanga OSM - Starting entry [Wait for file creation]
    2019/05/19 23:00:30 - Load Larabanga OSM - Starting entry [osm to geojson]
    2019/05/19 23:00:30 - osm to geojson - Running on platform : Linux
    2019/05/19 23:00:30 - osm to geojson - Executing command : /tmp/kettle_2281bc9c-7a79-11e9-a189-3bd5352ed62bshell
    2019/05/19 23:00:31 - Load Larabanga OSM - Starting entry [OSMosis init SQL]
    2019/05/19 23:00:31 - Load Larabanga OSM - Starting entry [SQL Trunc or Drop tables]
    2019/05/19 23:00:31 - Load Larabanga OSM - Starting entry [Wait for SQL init]
    2019/05/19 23:00:34 - Load Larabanga OSM - Starting entry [OSMosis to Posgtres DB]
    2019/05/19 23:00:34 - OSMosis to Posgtres DB - Running on platform : Linux
    2019/05/19 23:00:34 - OSMosis to Posgtres DB - Executing command : /tmp/kettle_24f1d23d-7a79-11e9-a189-3bd5352ed62bshell
    2019/05/19 23:00:34 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) May 19, 2019 11:00:34 PM org.openstreetmap.osmosis.core.Osmosis run
    2019/05/19 23:00:34 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) INFO: Osmosis Version 0.46
    2019/05/19 23:00:34 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) May 19, 2019 11:00:34 PM org.openstreetmap.osmosis.core.Osmosis run
    2019/05/19 23:00:34 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) INFO: Preparing pipeline.
    2019/05/19 23:00:34 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) May 19, 2019 11:00:34 PM org.openstreetmap.osmosis.core.Osmosis run
    2019/05/19 23:00:34 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) INFO: Launching pipeline execution.
    2019/05/19 23:00:34 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) May 19, 2019 11:00:34 PM org.openstreetmap.osmosis.core.Osmosis run
    2019/05/19 23:00:34 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) INFO: Pipeline executing, waiting for completion.
    2019/05/19 23:00:36 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) May 19, 2019 11:00:36 PM org.openstreetmap.osmosis.core.Osmosis run
    2019/05/19 23:00:36 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) INFO: Pipeline complete.
    2019/05/19 23:00:36 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) May 19, 2019 11:00:36 PM org.openstreetmap.osmosis.core.Osmosis run
    2019/05/19 23:00:36 - OSMosis to Posgtres DB - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : (stderr) INFO: Total execution time: 2353 milliseconds.
    2019/05/19 23:00:36 - Load Larabanga OSM - Starting entry [SQL for ways tags]
    2019/05/19 23:00:36 - Load Larabanga OSM - Starting entry [SQL for nodes tags]
    2019/05/19 23:00:37 - Load Larabanga OSM - Starting entry [SQL for relations tags]
    2019/05/19 23:00:37 - Load Larabanga OSM - Starting entry [Delete old zip file]
    2019/05/19 23:00:37 - Delete old zip file - File [$DATA_HOME/........_osm.zip] already deleted.
    2019/05/19 23:00:37 - Load Larabanga OSM - Starting entry [Zip OSM input data]
    2019/05/19 23:00:37 - Load Larabanga OSM - Starting entry [Store loaded OSM data]
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [Store loaded OSM data] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [Zip OSM input data] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [Delete old zip file] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [SQL for relations tags] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [SQL for nodes tags] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [SQL for ways tags] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [OSMosis to Posgtres DB] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [Wait for SQL init] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [SQL Trunc or Drop tables] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [OSMosis init SQL] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [osm to geojson] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [Wait for file creation] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [........ OSM exists] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [Ghana pbf to ......... osm] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [Get Geofabrik Ghana] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Finished job entry [DATA_HOME=/opt/pentaho/laradata] (result=[true])
    2019/05/19 23:00:37 - Load Larabanga OSM - Job execution finished
    2019/05/19 23:00:37 - Spoon - Job has ended.
    2019/05/19 23:17:45 - Spoon - Spoon

  2. #2
    Join Date
    Jan 2015
    Posts
    107

    Default

    Pentaho marks those lines as ERROR lines simply because they come from stderr, which is meant for errors.
    The Osmosis executable apparently returns extra information on stderr instead of stdout, probably to keep the stdout clean for more important messages.
    Curl does the same with download stats and they also get marked as errors in Pentaho. So you can keep ignoring this.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.