« Properties of tomcat and apache on strip1 | Main | Analysis of Apache/Catalina AgEcon Records for »

Why UDC crashed ... "pool exhausted" error Jason, Basically DSPACE does not properly close connections to the SQL server. When the pool is exhausted it generates error messages. This may be more of a problem now because OAI is available (climbing down t

Jason,

Basically DSPACE does not properly close connections to the SQL server. When the pool is exhausted it generates error messages. This may be more of a problem now because OAI is available (climbing down the tree will hit the DB a lot) or there is another SQL-Injection attack, or UDC just may be more popular. I did not explore the probable increased load.

A more detailed explanation of the error is given below, with a possible fix. To step up the fix given on the web I need some privileges on strip3. I have asked CCO for them.

Jeff

1) Problem Indicated in the Logs

Starting at
2008-09-17 08:23:08,769
and ending at
2008-09-17 08:53:29,729 (When Bill restarted DSPACE).
There were 330 error messages of the type:
2008-09-17 08:53:29,729 WARN org.dspace.app.webui.servlet.DSpaceServlet @ anonymous:no_context:database_error:org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool exhausted


An error messages of this sort would generate an error screen.

2) Fix from University of Michigan

In dspace-tech the University of Michigan team addresses this problem by closing prossess that have the phrase 'idle in transaction' when displayed by ps.

see
http://www.mail-archive.com/dspace-tech@lists.sourceforge.net/msg01057.html

3) Confirmation that our problem matches University of Michigan's
I have checked strip3 (where the postgres database lives) and between 08:54 (When Bill restarted DSPACE) and 11:22, there have been 45 processes created that have the form:

postgres 15047 0.0 0.0 86304 4964 ? S 10:58 0:00 postgres: dspace_ir dspace_ir 134.84.135.19 idle

So we are building up these "idle" processes on the DB side and it is likely that the system will crash again, unless we put in the Michigan fix

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)