PDA

View Full Version : Error: Too many open files



theeren
09-27-2006, 11:25 AM
Hi-
I have been getting the following error:

ERROR: [PoolTcpEndpoint] Endpoint ServerSocket...java.net.SocketException: Too many open files

then later on in the logs...

ERROR .. FileNotFoundException...(Too many open files)...This will be on a file that should be found...

I have one report that has 9 frames, each with 3 dial charts. The browser then refreshes every 3 minutes. There are also various other reports, some html, some pdf. I estimate that that 10 people are accessing these reports.

Has anyone else seen this? I did a search on google and found i can do ulimit -n to change the default limit of files open which is 1024. Is this really necessary though? Have I allowed a leak to occur or what is causing this?

Any help is much appreciated!!!
Thanks,
Tiffani

mbatchelor
09-27-2006, 01:52 PM
I don't think it's anything you're specifically doing.

In addition to the posts you found, you may want to look at this:

http://www.mail-archive.com/tomcat-user@jakarta.apache.org/msg03609.html

Good luck,

Marc

theeren
10-02-2006, 11:47 AM
Hi Marc.

Thanks for your reply, unfortunately that did not seem to be my problem. I am wondering if it might have to do with the Birt Logging. It seems to be creating about 3 log files about every minute. I am not sure how to turn this feature off. Oddly enough, these files are blank. If there are any Birt experts out there that happen to know how to Turn this off...I could really use the help. My system ends up crashing about ever 5 hours at this point.

Thanks,
Tiffani

mbatchelor
10-02-2006, 08:54 PM
Hi Tiffani,

Are you doing anything with BIRT? If not, perhaps removing the BIRT system listener would help. Open the pentaho.xml, and remove all BIRT references.

Let me know if that helps at all.

Take care,

Marc

theeren
10-03-2006, 05:56 AM
Hi Marc-
Well unfortunately I have three reports that use Birt. I believe only one of them seems to be the problem however and it is one I can just redo in JFree. What is happening is that this particular report is being refreshed every 3-5 minutes and apparently Birt creates a blank log file everytime. I have done a search and found others with a similar problem but no solution as of yet. Birt puts these log files in "system/logs/BIRT" directory and so I removed this BIRT directory and now it just complains that it can't find the BIRT directory to put the log file in but my system isn't crashing at least. :) This isn't really a clean fix though so I think it will be much easier to just convert this into a JFree report. If anyone stumbles upon this post in the future and knows a fix for this Birt log mess, please let me know. :)

Thanks,
Tiffani

Taqua
10-03-2006, 06:45 AM
Hi,

the logging behaviour of Birt is hardcoded inside the class "org.eclipse.birt.report.engine.api.impl.EngineLogger". As far as I can see from the code, they dont use any configuration for the logging here, so there is no way to turn it off.

To me, this either looks like the work of the pure evil. I would count the current behaviour as bug or design error - or both. The log system gets reinitalized all the time, and this causes the file-handle exhaustion.

Transforming that particular implementation into a singleton - by introducing a plain boolean flag for whether the logging has been initialized - should solve the problem.

Maybe the programmer who wrote that had a particulary bad day and wanted to share the pain he felt :)

Regards,
Thomas

dmoran
10-03-2006, 08:36 AM
Tiffani,

I did a little Googling and testing and it looks like the BIRT 2.1 implementation will work better. It will still create a log file but will only create a new file when the Platform is restarted and only one (maybe 2 since there is a .lck file) will be open at any time.

Try any build after 420. If you are using Pro, call Tech support for download instructions.

It looks like the logging is only semi evil. No matter what logging level is set (including OFF), BIRT will still log report generation info if it has been given a log file path.

Since your solution is refreshing multiple browsers every few minutes, that is still a lot of overhead. I have attached a new .class file for you to try that will stop all BIRT logging. It still requires a build after 420 in order to work. Put it in:

<pentaho-demo>/jboss/server/default/deploy/pentaho.war/WEB-INF/classes/org/pentaho/plugin/eclipsebirt

Let me know how it goes.

Doug

PS - Given the real-time nature of your reporting, you still may be better off with jFreeReports. I recommend building the same report in jFree and see if performance/memory usage is better. http://forums.pentaho.org/archived_att/files/BirtSystemListener.class

Post edited by: dmoran, at: 2006/10/03 12:40

theeren
10-16-2006, 05:56 AM
Hi Doug-
Thanks for looking into this for me. I think I will take your advice on this one and redo this report in JFree. Its a lesson learned I guess, hopefully someone else will benefit from this thread. Birt must be better suited for lengthy reports that aren't regenerated every couple of minutes. :)

Thanks again..

Tiffani