Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Timeseries forecast failing in junit test

  1. #1
    Join Date
    Oct 2016
    Posts
    3

    Default Timeseries forecast failing in junit test

    Im running the similar to the example code, and found the exact same code runs ok if I run from main thread, but failed in junit test.
    environment: java8, weka3.9, junit4.12


    stacktrace:


    java.lang.AssertionError
    at weka.core.expressionlanguage.weka.InstancesHelper.setInstance(InstancesHelper.java:159)
    at weka.filters.unsupervised.attribute.AddExpression.input(AddExpression.java:409)
    at weka.filters.supervised.attribute.TSLagMaker.processInstance(TSLagMaker.java:2810)
    at weka.filters.supervised.attribute.TSLagMaker.processInstancePreview(TSLagMaker.java:2692)
    at weka.classifiers.timeseries.WekaForecaster.forecast(WekaForecaster.java:1671)
    at weka.classifiers.timeseries.WekaForecaster.forecast(WekaForecaster.java:1485)
    at com.eniro.content.social.insight.InsightEngine.predict(InsightEngine.java:96)
    at com.eniro.content.social.insight.InsightEngineTest.name(InsightEngineTest.java:34)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)


    I dig into the exception and found one instances have following attributes
    0 = {Attribute@2467} "@attribute DATE date yyyy-MM-dd"
    1 = {Attribute@2468} "@attribute POPULARITY numeric"
    2 = {Attribute@2469} "@attribute Month {jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec}"
    3 = {Attribute@2470} "@attribute Quarter {Q1,Q2,Q3,Q4}"
    4 = {Attribute@2471} "@attribute DATE-remapped numeric"
    5 = {Attribute@2472} "@attribute Lag_POPULARITY-1 numeric"
    6 = {Attribute@2473} "@attribute Lag_POPULARITY-2 numeric"
    7 = {Attribute@2474} "@attribute Lag_POPULARITY-3 numeric"
    8 = {Attribute@2475} "@attribute Lag_POPULARITY-4 numeric"
    9 = {Attribute@2476} "@attribute Lag_POPULARITY-5 numeric"
    10 = {Attribute@2477} "@attribute Lag_POPULARITY-6 numeric"
    11 = {Attribute@2478} "@attribute Lag_POPULARITY-7 numeric"
    12 = {Attribute@2479} "@attribute Lag_POPULARITY-8 numeric"
    13 = {Attribute@2480} "@attribute Lag_POPULARITY-9 numeric"
    14 = {Attribute@2481} "@attribute Lag_POPULARITY-10 numeric"
    15 = {Attribute@2482} "@attribute Lag_POPULARITY-11 numeric"
    16 = {Attribute@2483} "@attribute Lag_POPULARITY numeric"

    The other one have following
    0 = {Attribute@2467} "@attribute DATE date yyyy-MM-dd"
    1 = {Attribute@2468} "@attribute POPULARITY numeric"
    2 = {Attribute@2469} "@attribute Month {jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec}"
    3 = {Attribute@2470} "@attribute Quarter {Q1,Q2,Q3,Q4}"
    4 = {Attribute@2471} "@attribute DATE-remapped numeric"
    5 = {Attribute@2472} "@attribute Lag_POPULARITY-1 numeric"
    6 = {Attribute@2473} "@attribute Lag_POPULARITY-2 numeric"
    7 = {Attribute@2474} "@attribute Lag_POPULARITY-3 numeric"
    8 = {Attribute@2475} "@attribute Lag_POPULARITY-4 numeric"
    9 = {Attribute@2476} "@attribute Lag_POPULARITY-5 numeric"
    10 = {Attribute@2477} "@attribute Lag_POPULARITY-6 numeric"
    11 = {Attribute@2478} "@attribute Lag_POPULARITY-7 numeric"
    12 = {Attribute@2479} "@attribute Lag_POPULARITY-8 numeric"
    13 = {Attribute@2480} "@attribute Lag_POPULARITY-9 numeric"
    14 = {Attribute@2481} "@attribute Lag_POPULARITY-10 numeric"
    15 = {Attribute@2482} "@attribute Lag_POPULARITY-11 numeric"
    16 = {Attribute@2608} "@attribute Lag_POPULARITY-12 numeric"


    Those attributes have to be the same to continue on weka.core.expressionlanguage.weka.InstancesHelper.setInstance method.

    The only difference that i can see from maven test environment is the junit dependency, but even if i removed the <scope>test</scope> main thread still runs ok.
    Can someone take a look at this? thanks


    Best


    Andrew

  2. #2
    Join Date
    Aug 2006
    Posts
    1,741

    Default

    Java assertions are turned off by default at runtime (and turned on by Maven's test environment). Your instances structure does not match - why is the Lag_POPULARITY-12 attribute named Lag_POPULARITY in the first instance? In this case it has not adverse affect because the order and meaning of the attributes is the same in both cases. Weka does not check normally check structure consistency at prediction time for schemes/filters for speed reasons.

    Cheers,
    Mark.

  3. #3
    Join Date
    Oct 2016
    Posts
    3

    Default

    Hi Mark

    Thanks for your clarification! I read that part of the code in weka (instances checking headers) the problem is that my code have no control of that structure, it is generated while using timeseriesforecast library in the forecast method.
    I failed to run the example code in junit with same exception, maybe someone can verify this bug on your end?


    Thanks!


    Andrew
    Quote Originally Posted by Mark View Post
    Java assertions are turned off by default at runtime (and turned on by Maven's test environment). Your instances structure does not match - why is the Lag_POPULARITY-12 attribute named Lag_POPULARITY in the first instance? In this case it has not adverse affect because the order and meaning of the attributes is the same in both cases. Weka does not check normally check structure consistency at prediction time for schemes/filters for speed reasons.

    Cheers,
    Mark.

  4. #4
    Join Date
    Aug 2006
    Posts
    1,741

    Default

    Yes, you are correct. There was a bug in filter being used to transform data. I've fixed this and made a new release of the timeseriesForecasting package. Refresh your repository cache and install the new version.

    Thanks for the bug report!

    Cheers,
    Mark.

  5. #5
    Join Date
    Oct 2016
    Posts
    3

    Default

    Thanks! problem solved.

    Quote Originally Posted by Mark View Post
    Yes, you are correct. There was a bug in filter being used to transform data. I've fixed this and made a new release of the timeseriesForecasting package. Refresh your repository cache and install the new version.

    Thanks for the bug report!

    Cheers,
    Mark.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.