PDA

View Full Version : Spoon could not connect to remote node of cassandra cluster



yajun
06-12-2012, 04:06 AM
Hi list



we have a cassandra cluster with 4 nodes. as a requirement, we should transfer data from hadoop to Cassandra, so I follow the user guide of http://wiki.pentaho.com/display/BAD/Write+Data+To+Cassandra?atl_token=vrytBNjwWD , and on step 10, a KettleException was occurred.

I use the Spoon with Stable Release - 4.3.0 with cassandra 1.1.1.

Does anyone have this kind of problem before? And any explanation? Thank you in advance.

partial exception:

2012/06/12 16:04:21 - Cassandra Output.0 - Connecting to Cassandra node at 'echadoop01:9160' using keyspace 'pentaho'...
2012/06/12 16:04:21 - Cassandra Output.0 - Getting meta data for column family 'PageSuccessions'
2012/06/12 16:04:21 - Cassandra Output.0 - Closing connection...
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Unexpected error
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : org.pentaho.di.core.exception.KettleException:
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : null
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at java.lang.Thread.run (Thread.java:619)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.di.trans.step.RunThread.run (RunThread.java:50)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.di.trans.steps.cassandraoutput.CassandraOutput.processRow (CassandraOutput.java:176)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.cassandra.CassandraColumnMetaData.<init> (CassandraColumnMetaData.java:118)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.cassandra.CassandraColumnMetaData.refresh (CassandraColumnMetaData.java:175)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.di.trans.steps.cassandraoutput.CassandraOutput.processRow(CassandraOutput.java:185)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.di.trans.step.RunThread.run(RunThread.java:50)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at java.lang.Thread.run(Thread.java:619)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Caused by: java.lang.NullPointerException
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.cassandra.CassandraColumnMetaData.refresh(CassandraColumnMetaData.java:175)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.cassandra.CassandraColumnMetaData.<init>(CassandraColumnMetaData.java:118)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.di.trans.steps.cassandraoutput.CassandraOutput.processRow(CassandraOutput.java:176)
2012/06/12 16:04:21 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : ... 2 more
2012/06/12 16:04:21 - Cassandra Output.0 - Finished processing (I=0, O=0, R=1, W=0, U=0, E=1)
2012/06/12 16:04:21 - Spoon - The transformation has finished!!
2012/06/12 16:04:21 - populate_cassandra_page_successions - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Errors detected!
2012/06/12 16:04:21 - populate_cassandra_page_successions - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Errors detected!
2012/06/12 16:04:21 - populate_cassandra_page_successions - populate_cassandra_page_successions
2012/06/12 16:04:21 - populate_cassandra_page_successions - populate_cassandra_page_successions
2012/06/12 16:04:21 - populate_cassandra_page_successions - Looking at step: text file input
2012/06/12 16:04:21 - populate_cassandra_page_successions - Looking at step: Cassandra Output

Mark
06-12-2012, 05:13 AM
Hi,

The Cassandra steps that ship with 4.3.0 are compatible with Cassandra 1.0.x. There have been some small API changes between 1.0.8 and 1.1.0. You could try a snapshot build of the big data plugin available from:

http://ci.pentaho.com/job/pentaho-big-data-plugin/

Just replace the big data plugin in your PDI installation with the one from CI. Also replace the apache-cassandra-1.0.0.jar and apache-cassandra-thrift-1.0.0.jar in the lib directory of the big data plugin with the corresponding jars from your Cassandra 1.1.1 installation.

Cheers,
Mark.

yajun
06-12-2012, 11:57 PM
Thanks for the reply!

I just downgrade the cassandra to an early version(1.0.0), and Spoon could connect to the remote cassandra cluster, but I faced another problem with "Show Schema" button from the "Cassandra Output" window.

Show Schema produces an error "Column names are neither UTF-8 or ASCII!"

the output of the Logging:

2012/06/13 11:16:13 - Cassandra Output.0 - Connecting to Cassandra node at 'xxx.xxx.xxx.xxx:9160' using keyspace 'Demo'...
2012/06/13 11:16:13 - Cassandra Output.0 - Getting meta data for column family 'PageSuccessions'
2012/06/13 11:16:13 - Cassandra Output.0 - Closing connection...
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Unexpected error
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : org.pentaho.di.core.exception.KettleException:
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Column names are neither UTF-8 or ASCII!
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Column names are neither UTF-8 or ASCII!
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.di.trans.steps.cassandraoutput.CassandraOutput.processRow(CassandraOutput.java:185)
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.di.trans.step.RunThread.run(RunThread.java:50)
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at java.lang.Thread.run(Thread.java:619)
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Caused by: java.lang.Exception: Column names are neither UTF-8 or ASCII!
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.cassandra.CassandraColumnMetaData.refresh(CassandraColumnMetaData.java:194)
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.cassandra.CassandraColumnMetaData.<init>(CassandraColumnMetaData.java:118)
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : at org.pentaho.di.trans.steps.cassandraoutput.CassandraOutput.processRow(CassandraOutput.java:176)
2012/06/13 11:16:13 - Cassandra Output.0 - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : ... 2 more
2012/06/13 11:16:13 - Cassandra Output.0 - Finished processing (I=0, O=0, R=1, W=0, U=0, E=1)
2012/06/13 11:16:13 - Spoon - The transformation has finished!!
2012/06/13 11:16:13 - populate_cassandra_page_successions - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Errors detected!
2012/06/13 11:16:13 - populate_cassandra_page_successions - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Errors detected!

Mark
06-14-2012, 10:55 PM
There will be a new release of the big-data-plugin in a few weeks time. A number of improvements to the Cassandra input step have been made. See:

http://jira.pentaho.com/browse/PDI-7722
http://jira.pentaho.com/browse/PDI-7963
http://jira.pentaho.com/browse/PDI-7964

Cheers,
Mark.

jganoff
06-15-2012, 01:53 PM
In the mean time you could try the latest build of the plugin from here:

pentaho-big-data-plugin-1.3-SNAPSHOT.zip (http://ci.pentaho.com/job/BRANCH_pentaho-big-data-plugin/lastSuccessfulBuild/artifact/dist/pentaho-big-data-plugin-1.3-SNAPSHOT.zip)
pentaho-big-data-plugin-1.3-SNAPSHOT.tar.gz (http://ci.pentaho.com/job/BRANCH_pentaho-big-data-plugin/lastSuccessfulBuild/artifact/dist/pentaho-big-data-plugin-1.3-SNAPSHOT.tar.gz)

The fixes Mark mentioned should be included in that build.

yajun
06-19-2012, 10:56 PM
In the mean time you could try the latest build of the plugin from here:

pentaho-big-data-plugin-1.3-SNAPSHOT.zip (http://ci.pentaho.com/job/BRANCH_pentaho-big-data-plugin/lastSuccessfulBuild/artifact/dist/pentaho-big-data-plugin-1.3-SNAPSHOT.zip)
pentaho-big-data-plugin-1.3-SNAPSHOT.tar.gz (http://ci.pentaho.com/job/BRANCH_pentaho-big-data-plugin/lastSuccessfulBuild/artifact/dist/pentaho-big-data-plugin-1.3-SNAPSHOT.tar.gz)

The fixes Mark mentioned should be included in that build.


Hi

Thanks for the replies, I just fixed my problem. And before I create column family(BytesType by default with comparator and validation_class) first via command line, than ran the job on Spoon, it could not work. Now I found out the Spoon seems to only support UTF8Type column_type. so, I just specify a new column family which does not exist in keyspace on Cassandra Output window, than checked the "Create column family", it works.

Thanks.