I wrote a simple Java script to build a Logit Model Tree. It works with tiny data sets, like Fischer's Iris data. However if I pass it a production data set with around 100,000 rows and 5 columns the RAM usage always causes a crash.

I ran it on a server and gave it 50GB of RAM, but that still wasn't enough. I could do the same thing in R and only use a couple of GB's; I would've expected Java / WEKA to be more efficient.

Is there anything I can do to troubleshoot or reduce RAM usage? Is this normal?

Thanks in advance.