too many clients already
Locked
 
|
Hi miners,
The similar error has been reported previously: http://gmod.827538.n3.nabble.com/ratmine-data-loading-errors-td4029257.html However, even I using project_build script. The error remain. 1) I have set the "db.production.datasource.maxConnections=150" in the default.intermine.integrate.properties file. During the running of the project_build, "ps -ef |grep postgres | grep idle |wc -l" return as many as 167 processes. When I load the source one-by-one, although one source was failed loaded during to "too many clients" error, the post-process and the web deployment were successful. 2) I have also tested set the "db.production.datasource.maxConnections=200". But the error message suggest this number should not be as larger as 200. Further I found that many of the process of postgres were "idle". Is it mean that these processes could be killed? But, I have tested to kill the idle process, which resulted in the termination of the building process. 3) I have tested to restart postgres server and then run project_build again. Having tested many times, each time was stopped at different steps with the same error "too many clients already" =================== error message============================ BUILD FAILED /home/pengchy/soft/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line: /home/pengchy/soft/intermine/imbuild/source.xml:201: Exception in XmlDataLoaderTask at org.intermine.dataloader.XmlDataLoaderTask.execute(XmlDataLoaderTask.java:170) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:437) at org.apache.tools.ant.Target.performTasks(Target.java:458) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36) at org.apache.tools.ant.Project.executeTargets(Project.java:1260) at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442) at org.intermine.task.Integrate.performAction(Integrate.java:219) at org.intermine.task.Integrate.performAction(Integrate.java:131) at org.intermine.task.Integrate.execute(Integrate.java:123) at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) at org.apache.tools.ant.Task.perform(Task.java:348) at org.apache.tools.ant.Target.execute(Target.java:437) at org.apache.tools.ant.Target.performTasks(Target.java:458) at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) at org.apache.tools.ant.Project.executeTarget(Project.java:1376) at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) at org.apache.tools.ant.Project.executeTargets(Project.java:1260) at org.apache.tools.ant.Main.runBuild(Main.java:857) at org.apache.tools.ant.Main.startAnt(Main.java:236) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:286) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:112) Caused by: org.intermine.objectstore.ObjectStoreException: org.intermine.objectstore.ObjectStoreException: Unable to get database for InterMine ObjectStore at org.intermine.objectstore.ObjectStoreWriterFactory.getObjectStoreWriter(ObjectStoreWriterFactory.java:71) at org.intermine.dataloader.IntegrationWriterDataTrackingImpl.getInstance(IntegrationWriterDataTrackingImpl.java:120) at org.intermine.dataloader.IntegrationWriterDataTrackingImpl.getInstance(IntegrationWriterDataTrackingImpl.java:80) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.intermine.dataloader.IntegrationWriterFactory.getIntegrationWriter(IntegrationWriterFactory.java:65) at org.intermine.dataloader.XmlDataLoaderTask.execute(XmlDataLoaderTask.java:123) ... 31 more Caused by: org.intermine.objectstore.ObjectStoreException: Unable to get database for InterMine ObjectStore at org.intermine.objectstore.intermine.ObjectStoreInterMineImpl.getInstance(ObjectStoreInterMineImpl.java:358) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.intermine.objectstore.ObjectStoreFactory.getObjectStore(ObjectStoreFactory.java:63) at org.intermine.objectstore.ObjectStoreWriterFactory.getObjectStoreWriter(ObjectStoreWriterFactory.java:69) ... 39 more Caused by: java.lang.RuntimeException: Failed to initialise db.production at org.intermine.sql.DatabaseFactory.getDatabase(DatabaseFactory.java:58) at org.intermine.objectstore.intermine.ObjectStoreInterMineImpl.getInstance(ObjectStoreInterMineImpl.java:356) ... 45 more Caused by: com.zaxxer.hikari.pool.PoolInitializationException: Exception during pool initialization at com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:544) at com.zaxxer.hikari.pool.BaseHikariPool.<init>(BaseHikariPool.java:171) at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:60) at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:48) at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:80) at org.intermine.sql.Database.<init>(Database.java:122) at org.intermine.sql.DatabaseFactory.getDatabase(DatabaseFactory.java:56) ... 46 more Caused by: org.postgresql.util.PSQLException: FATAL: sorry, too many clients already at org.postgresql.Driver$ConnectThread.getResult(Driver.java:355) at org.postgresql.Driver.connect(Driver.java:279) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:247) at org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:86) at org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:71) at com.zaxxer.hikari.pool.BaseHikariPool.addConnection(BaseHikariPool.java:438) at com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:542) ... 52 more Total time: 22 seconds Fri Apr 6 09:55:48 CST 2018 finished ERROR: 256 failed with exit code 0: ant -v -Dsource=entrez-organism Best, Pengcheng _______________________________________________ dev mailing list [hidden email] https://lists.intermine.org/mailman/listinfo/dev |
|
|
Hi，
I have resolved this problem by set "max_connections = 500" in /var/lib/pgsql/data/postgresql.conf file. Some useful links are: [1] https://stackoverflow.com/questions/2757549/org-postgresql-util-psqlexception-fatal-sorry-too-many-clients-already [2] https://stackoverflow.com/questions/30778015/how-to-increase-the-max-connections-in-postgres Best, Pengcheng On 2018-4-6 10:20, Pengcheng Yang wrote: > Hi miners, > > The similar error has been reported previously: > http://gmod.827538.n3.nabble.com/ratmine-data-loading-errors-td4029257.html > > However, even I using project_build script. The error remain. > > 1) I have set the "db.production.datasource.maxConnections=150" in the > default.intermine.integrate.properties file. During the running of the > project_build, "ps -ef |grep postgres | grep idle |wc -l" return as > many as 167 processes. When I load the source one-by-one, although one > source was failed loaded during to "too many clients" error, the > post-process and the web deployment were successful. > > 2) I have also tested set the > "db.production.datasource.maxConnections=200". But the error message > suggest this number should not be as larger as 200. > > Further I found that many of the process of postgres were "idle". Is > it mean that these processes could be killed? But, I have tested to > kill the idle process, which resulted in the termination of the > building process. > > 3) I have tested to restart postgres server and then run project_build > again. Having tested many times, each time was stopped at different > steps with the same error "too many clients already" > > =================== error message============================ > > BUILD FAILED > /home/pengchy/soft/intermine/imbuild/integrate.xml:54: The following > error occurred while executing this line: > /home/pengchy/soft/intermine/imbuild/source.xml:201: Exception in > XmlDataLoaderTask > at > org.intermine.dataloader.XmlDataLoaderTask.execute(XmlDataLoaderTask.java:170) > at > org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) > at org.apache.tools.ant.Task.perform(Task.java:348) > at org.apache.tools.ant.Target.execute(Target.java:437) > at org.apache.tools.ant.Target.performTasks(Target.java:458) > at > org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) > at > org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36) > at org.apache.tools.ant.Project.executeTargets(Project.java:1260) > at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442) > at org.intermine.task.Integrate.performAction(Integrate.java:219) > at org.intermine.task.Integrate.performAction(Integrate.java:131) > at org.intermine.task.Integrate.execute(Integrate.java:123) > at > org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) > at org.apache.tools.ant.Task.perform(Task.java:348) > at org.apache.tools.ant.Target.execute(Target.java:437) > at org.apache.tools.ant.Target.performTasks(Target.java:458) > at > org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) > at org.apache.tools.ant.Project.executeTarget(Project.java:1376) > at > org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) > at org.apache.tools.ant.Project.executeTargets(Project.java:1260) > at org.apache.tools.ant.Main.runBuild(Main.java:857) > at org.apache.tools.ant.Main.startAnt(Main.java:236) > at org.apache.tools.ant.launch.Launcher.run(Launcher.java:286) > at org.apache.tools.ant.launch.Launcher.main(Launcher.java:112) > Caused by: org.intermine.objectstore.ObjectStoreException: > org.intermine.objectstore.ObjectStoreException: Unable to get database > for InterMine ObjectStore > at > org.intermine.objectstore.ObjectStoreWriterFactory.getObjectStoreWriter(ObjectStoreWriterFactory.java:71) > at > org.intermine.dataloader.IntegrationWriterDataTrackingImpl.getInstance(IntegrationWriterDataTrackingImpl.java:120) > at > org.intermine.dataloader.IntegrationWriterDataTrackingImpl.getInstance(IntegrationWriterDataTrackingImpl.java:80) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.intermine.dataloader.IntegrationWriterFactory.getIntegrationWriter(IntegrationWriterFactory.java:65) > at > org.intermine.dataloader.XmlDataLoaderTask.execute(XmlDataLoaderTask.java:123) > ... 31 more > Caused by: org.intermine.objectstore.ObjectStoreException: Unable to > get database for InterMine ObjectStore > at > org.intermine.objectstore.intermine.ObjectStoreInterMineImpl.getInstance(ObjectStoreInterMineImpl.java:358) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.intermine.objectstore.ObjectStoreFactory.getObjectStore(ObjectStoreFactory.java:63) > at > org.intermine.objectstore.ObjectStoreWriterFactory.getObjectStoreWriter(ObjectStoreWriterFactory.java:69) > ... 39 more > Caused by: java.lang.RuntimeException: Failed to initialise db.production > at > org.intermine.sql.DatabaseFactory.getDatabase(DatabaseFactory.java:58) > at > org.intermine.objectstore.intermine.ObjectStoreInterMineImpl.getInstance(ObjectStoreInterMineImpl.java:356) > ... 45 more > Caused by: com.zaxxer.hikari.pool.PoolInitializationException: > Exception during pool initialization > at > com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:544) > at > com.zaxxer.hikari.pool.BaseHikariPool.<init>(BaseHikariPool.java:171) > at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:60) > at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:48) > at > com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:80) > at org.intermine.sql.Database.<init>(Database.java:122) > at > org.intermine.sql.DatabaseFactory.getDatabase(DatabaseFactory.java:56) > ... 46 more > Caused by: org.postgresql.util.PSQLException: FATAL: sorry, too many > clients already > at org.postgresql.Driver$ConnectThread.getResult(Driver.java:355) > at org.postgresql.Driver.connect(Driver.java:279) > at java.sql.DriverManager.getConnection(DriverManager.java:664) > at java.sql.DriverManager.getConnection(DriverManager.java:247) > at > org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:86) > at > org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:71) > at > com.zaxxer.hikari.pool.BaseHikariPool.addConnection(BaseHikariPool.java:438) > at > com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:542) > ... 52 more > > Total time: 22 seconds > Fri Apr 6 09:55:48 CST 2018 > > finished > > > ERROR: 256 > failed with exit code 0: ant -v -Dsource=entrez-organism > > > Best, > > Pengcheng > > > _______________________________________________ > dev mailing list > [hidden email] > https://lists.intermine.org/mailman/listinfo/dev _______________________________________________ dev mailing list [hidden email] https://lists.intermine.org/mailman/listinfo/dev |
Re: too many clients already
|
|
You shouldn't need 500 connections. Human and flymine build fine with 20.
1. Is your code on github? Can I take a look? Specifically I am interested in what your data sources are doing. 2. Which version of postgres are you on? I recommend 9.3 but I am using 9.5 without any issue. 3. When this happens, when you see this error, what is the output of `ps aux | grep postgres`. 4. Do you see any hanging queries? `select * from pg_stat_activity` will tell you which queries are running. No you shouldn't need to kill the postgres processes. I would have thought that your earlier errors might have caused the database connections to not close properly. This would account for the error message you saw. However this would have been fixed by a postgres restart - which you did. On 04/06/2018 08:48 AM, Pengcheng Yang wrote: > Hi， > > I have resolved this problem by set "max_connections = 500" in > /var/lib/pgsql/data/postgresql.conf file. > > Some useful links are: > > [1] > https://stackoverflow.com/questions/2757549/org-postgresql-util-psqlexception-fatal-sorry-too-many-clients-already > > > [2] > https://stackoverflow.com/questions/30778015/how-to-increase-the-max-connections-in-postgres > > > Best, > > Pengcheng > > > On 2018-4-6 10:20, Pengcheng Yang wrote: >> Hi miners, >> >> The similar error has been reported previously: >> http://gmod.827538.n3.nabble.com/ratmine-data-loading-errors-td4029257.html >> >> >> However, even I using project_build script. The error remain. >> >> 1) I have set the "db.production.datasource.maxConnections=150" in the >> default.intermine.integrate.properties file. During the running of the >> project_build, "ps -ef |grep postgres | grep idle |wc -l" return as >> many as 167 processes. When I load the source one-by-one, although one >> source was failed loaded during to "too many clients" error, the >> post-process and the web deployment were successful. >> >> 2) I have also tested set the >> "db.production.datasource.maxConnections=200". But the error message >> suggest this number should not be as larger as 200. >> >> Further I found that many of the process of postgres were "idle". Is >> it mean that these processes could be killed? But, I have tested to >> kill the idle process, which resulted in the termination of the >> building process. >> >> 3) I have tested to restart postgres server and then run project_build >> again. Having tested many times, each time was stopped at different >> steps with the same error "too many clients already" >> >> =================== error message============================ >> >> BUILD FAILED >> /home/pengchy/soft/intermine/imbuild/integrate.xml:54: The following >> error occurred while executing this line: >> /home/pengchy/soft/intermine/imbuild/source.xml:201: Exception in >> XmlDataLoaderTask >> at >> org.intermine.dataloader.XmlDataLoaderTask.execute(XmlDataLoaderTask.java:170) >> >> at >> org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) >> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:498) >> at >> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) >> >> at org.apache.tools.ant.Task.perform(Task.java:348) >> at org.apache.tools.ant.Target.execute(Target.java:437) >> at org.apache.tools.ant.Target.performTasks(Target.java:458) >> at >> org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) >> at >> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36) >> >> at org.apache.tools.ant.Project.executeTargets(Project.java:1260) >> at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442) >> at org.intermine.task.Integrate.performAction(Integrate.java:219) >> at org.intermine.task.Integrate.performAction(Integrate.java:131) >> at org.intermine.task.Integrate.execute(Integrate.java:123) >> at >> org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) >> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:498) >> at >> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) >> >> at org.apache.tools.ant.Task.perform(Task.java:348) >> at org.apache.tools.ant.Target.execute(Target.java:437) >> at org.apache.tools.ant.Target.performTasks(Target.java:458) >> at >> org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) >> at org.apache.tools.ant.Project.executeTarget(Project.java:1376) >> at >> org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) >> >> at org.apache.tools.ant.Project.executeTargets(Project.java:1260) >> at org.apache.tools.ant.Main.runBuild(Main.java:857) >> at org.apache.tools.ant.Main.startAnt(Main.java:236) >> at org.apache.tools.ant.launch.Launcher.run(Launcher.java:286) >> at org.apache.tools.ant.launch.Launcher.main(Launcher.java:112) >> Caused by: org.intermine.objectstore.ObjectStoreException: >> org.intermine.objectstore.ObjectStoreException: Unable to get database >> for InterMine ObjectStore >> at >> org.intermine.objectstore.ObjectStoreWriterFactory.getObjectStoreWriter(ObjectStoreWriterFactory.java:71) >> >> at >> org.intermine.dataloader.IntegrationWriterDataTrackingImpl.getInstance(IntegrationWriterDataTrackingImpl.java:120) >> >> at >> org.intermine.dataloader.IntegrationWriterDataTrackingImpl.getInstance(IntegrationWriterDataTrackingImpl.java:80) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:498) >> at >> org.intermine.dataloader.IntegrationWriterFactory.getIntegrationWriter(IntegrationWriterFactory.java:65) >> >> at >> org.intermine.dataloader.XmlDataLoaderTask.execute(XmlDataLoaderTask.java:123) >> >> ... 31 more >> Caused by: org.intermine.objectstore.ObjectStoreException: Unable to >> get database for InterMine ObjectStore >> at >> org.intermine.objectstore.intermine.ObjectStoreInterMineImpl.getInstance(ObjectStoreInterMineImpl.java:358) >> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> at java.lang.reflect.Method.invoke(Method.java:498) >> at >> org.intermine.objectstore.ObjectStoreFactory.getObjectStore(ObjectStoreFactory.java:63) >> >> at >> org.intermine.objectstore.ObjectStoreWriterFactory.getObjectStoreWriter(ObjectStoreWriterFactory.java:69) >> >> ... 39 more >> Caused by: java.lang.RuntimeException: Failed to initialise db.production >> at >> org.intermine.sql.DatabaseFactory.getDatabase(DatabaseFactory.java:58) >> at >> org.intermine.objectstore.intermine.ObjectStoreInterMineImpl.getInstance(ObjectStoreInterMineImpl.java:356) >> >> ... 45 more >> Caused by: com.zaxxer.hikari.pool.PoolInitializationException: >> Exception during pool initialization >> at >> com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:544) >> >> at >> com.zaxxer.hikari.pool.BaseHikariPool.<init>(BaseHikariPool.java:171) >> at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:60) >> at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:48) >> at >> com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:80) >> at org.intermine.sql.Database.<init>(Database.java:122) >> at >> org.intermine.sql.DatabaseFactory.getDatabase(DatabaseFactory.java:56) >> ... 46 more >> Caused by: org.postgresql.util.PSQLException: FATAL: sorry, too many >> clients already >> at org.postgresql.Driver$ConnectThread.getResult(Driver.java:355) >> at org.postgresql.Driver.connect(Driver.java:279) >> at java.sql.DriverManager.getConnection(DriverManager.java:664) >> at java.sql.DriverManager.getConnection(DriverManager.java:247) >> at >> org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:86) >> >> at >> org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:71) >> >> at >> com.zaxxer.hikari.pool.BaseHikariPool.addConnection(BaseHikariPool.java:438) >> >> at >> com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:542) >> >> ... 52 more >> >> Total time: 22 seconds >> Fri Apr 6 09:55:48 CST 2018 >> >> finished >> >> >> ERROR: 256 >> failed with exit code 0: ant -v -Dsource=entrez-organism >> >> >> Best, >> >> Pengcheng >> >> >> _______________________________________________ >> dev mailing list >> [hidden email] >> https://lists.intermine.org/mailman/listinfo/dev > > _______________________________________________ > dev mailing list > [hidden email] > https://lists.intermine.org/mailman/listinfo/dev dev mailing list [hidden email] https://lists.intermine.org/mailman/listinfo/dev |
|
|
Hi Julie,
Thank you for the helpful suggestions. 1. I have just uploaded my source code to github at: https://github.com/pengchy/intermine/tree/master/testmine for your checking. 2. I deploy the intermine on CentOS 7, PostgreSQL 9.2.3. 3. When this happens, the `ps aux|grep postgres` output the following message with about >100 postgres process with idle state. postgres 2035 0.0 0.0 3561652 8792 ? Ss 15:36 0:00 postgres: postgres userprofile-testmodel 10.4.0.17(47445) idle postgres 2038 0.0 0.0 3561772 8312 ? Ss 15:36 0:00 postgres: postgres userprofile-testmodel 10.4.0.17(47446) idle postgres 2508 0.0 0.0 3561484 8052 ? Ss 15:37 0:00 postgres: postgres testmine 10.4.0.17(47551) idle postgres 2579 0.0 0.0 3561640 8532 ? Ss 15:37 0:00 postgres: postgres userprofile-testmine 10.4.0.17(47584) idle postgres 2585 0.0 0.0 3561764 8568 ? Ss 15:37 0:00 postgres: postgres userprofile-testmine 10.4.0.17(47587) idle postgres 13382 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48353) idle postgres 13384 0.0 0.0 3560392 4476 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48354) idle postgres 13386 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48355) idle postgres 13388 0.0 0.0 3560392 4468 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48356) idle postgres 13390 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48357) idle postgres 13392 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48358) idle postgres 13394 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48359) idle postgres 13396 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48360) idle postgres 13398 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48361) idle postgres 13400 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48362) idle postgres 13402 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48363) idle postgres 13404 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48364) idle postgres 13406 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48365) idle postgres 13408 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48366) idle postgres 13410 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48367) idle postgres 13412 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48368) idle postgres 13414 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48369) idle postgres 13416 0.0 0.0 3560392 4476 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48370) idle postgres 13418 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48371) idle postgres 13420 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48372) idle postgres 13422 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48373) idle postgres 13424 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48374) idle postgres 13426 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48375) idle postgres 13428 0.0 0.0 3560392 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48376) idle postgres 13430 0.0 0.0 3560396 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48377) idle postgres 13432 0.0 0.0 3560396 4476 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48378) idle postgres 13434 0.0 0.0 3560396 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48379) idle postgres 13436 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48380) idle postgres 13438 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48381) idle postgres 13440 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48382) idle postgres 13442 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48383) idle postgres 13444 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48384) idle postgres 13446 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48385) idle postgres 13448 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48386) idle postgres 13450 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48387) idle postgres 13452 0.0 0.0 3560396 4472 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48388) idle postgres 13456 0.0 0.0 3560396 4464 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48389) idle postgres 13458 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48390) idle postgres 13460 0.0 0.0 3560396 4464 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48391) idle postgres 13462 0.0 0.0 3560396 4460 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48392) idle postgres 13464 0.0 0.0 3560396 4460 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48393) idle postgres 13466 0.0 0.0 3560396 4460 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48394) idle postgres 13468 0.0 0.0 3560396 4464 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48395) idle postgres 13470 0.0 0.0 3560396 4460 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48396) idle postgres 13472 0.0 0.0 3560396 4460 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48397) idle postgres 13474 0.0 0.0 3560396 4460 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48398) idle postgres 13476 0.0 0.0 3560396 4464 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48399) idle postgres 13478 0.0 0.0 3560396 4464 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48400) idle postgres 13480 0.0 0.0 3560396 4464 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48401) idle postgres 13482 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48402) idle postgres 13484 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48403) idle postgres 13486 0.0 0.0 3560396 4464 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48404) idle postgres 13507 0.0 0.0 3560396 4468 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48405) idle postgres 13509 0.0 0.0 3560396 4476 ? Ss 17:33 0:00 postgres: postgres testmine 10.4.0.17(48406) idle postgres 13513 0.0 0.0 3560396 4464 ? Ss 17:33 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48407) idle postgres 13619 0.0 0.0 3560396 4472 ? Ss 17:35 0:00 postgres: postgres testmine 10.4.0.17(48410) idle postgres 13722 0.0 0.0 3560396 4472 ? Ss 17:38 0:00 postgres: postgres userprofile-testmine 10.4.0.17(48411) idle postgres 13848 0.0 0.0 3560396 4468 ? Ss 17:42 0:00 postgres: postgres unittest 10.4.0.17(48412) idle postgres 13850 0.0 0.0 3560396 4472 ? Ss 17:42 0:00 postgres: postgres unittest 10.4.0.17(48413) idle postgres 13852 0.0 0.0 3560396 4468 ? Ss 17:42 0:00 postgres: postgres unittest 10.4.0.17(48414) idle postgres 13854 0.0 0.0 3560396 4472 ? Ss 17:42 0:00 postgres: postgres unittest 10.4.0.17(48415) idle postgres 13856 0.0 0.0 3560396 4472 ? Ss 17:42 0:00 postgres: postgres unittest 10.4.0.17(48416) idle postgres 13858 0.0 0.0 3560396 4472 ? Ss 17:42 0:00 postgres: postgres unittest 10.4.0.17(48417) idle postgres 13860 0.0 0.0 3560396 4472 ? Ss 17:42 0:00 postgres: postgres unittest 10.4.0.17(48418) idle postgres 13862 0.0 0.0 3560396 4476 ? Ss 17:42 0:00 postgres: postgres unittest 10.4.0.17(48419) idle postgres 13864 0.0 0.0 3560396 4472 ? Ss 17:42 0:00 postgres: postgres unittest 10.4.0.17(48420) idle postgres 13866 0.0 0.0 3560396 4472 ? Ss 17:42 0:00 postgres: postgres unittest 10.4.0.17(48421) idle postgres 13870 0.0 0.0 3560396 4472 ? Ss 17:42 0:00 postgres: postgres userprofile-testmodel 10.4.0.17(48422) idle postgres 13872 0.0 0.0 3560396 4476 ? Ss 17:42 0:00 postgres: postgres userprofile-testmodel 10.4.0.17(48423) idle postgres 13874 0.0 0.0 3560396 4472 ? Ss 17:42 0:00 postgres: postgres userprofile-testmodel 10.4.0.17(48424) idle root 13877 0.0 0.0 112664 980 pts/2 R+ 17:42 0:00 grep --color=auto postgres postgres 20268 0.0 0.6 3557504 221940 ? S 15:26 0:04 /usr/bin/postgres -D /var/lib/pgsql/data -p 5432 postgres 20269 0.0 0.0 190472 1520 ? Ss 15:26 0:00 postgres: logger process postgres 20271 0.0 2.3 3559456 775564 ? Ss 15:26 0:01 postgres: checkpointer process postgres 20272 0.0 0.0 3559008 21280 ? Ss 15:26 0:00 postgres: writer process postgres 20273 0.0 0.0 3559008 18296 ? Ss 15:26 0:01 postgres: wal writer process postgres 20274 0.0 0.0 3560536 3680 ? Ss 15:26 0:00 postgres: autovacuum launcher process postgres 20275 0.0 0.0 193300 2160 ? Ss 15:26 0:02 postgres: stats collector process 4. After successfully building the InterMine, now, `select * from pg_stat_activity` returns 76 records as follows. postgres=# select * from pg_stat_activity; datid | datname | pid | usesysid | usename | application_name | client_addr | client_hostname | client_port | backend_start | xact_start | query_start | state_change | waiting | state | query -------+-----------------------+-------+----------+----------+------------------+-------------+-----------------+-------------+------------------------ -------+-------------------------------+-------------------------------+-------------------------------+---------+--------+---------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------------------------------- 18438 | testmine | 2508 | 10 | postgres | | 10.4.0.17 | | 47551 | 2018-04-06 15:37:45.155 604+08 | | 2018-04-06 15:37:45.25924+08 | 2018-04-06 15:37:45.260696+08 | f | idle | SELECT NULL AS TABLE_CAT, n .nspname AS TABLE_SCHEM, c.relname AS TABLE_NAME, CASE n.nspname ~ '^pg_' OR n.nspname = 'information_schema' WHEN true THEN CASE WHEN n.nspname = ' pg_catalog' OR n.nspname = 'information_schema' THEN CASE c.relkind WHEN 'r' THEN 'SYSTEM TABLE' WHEN 'v' THEN 'SYSTEM VIEW' WHEN 'i' THEN 'SYSTE M INDEX' ELSE NULL END WHEN n.nspname = 'pg_toast' THEN CASE c.relkind WHEN 'r' THEN 'SYSTEM TOAST TABLE' WHEN 'i' THEN 'SYSTEM TOAST INDEX' ELSE NULL END ELSE CASE c.relkind WHEN 'r' THEN 'TEMPORARY TABLE' WHEN 'i' THEN 'TEMPORARY INDEX' WHEN 'S' THEN 'TEMPORARY SEQUENCE' WHEN ' v' THEN 'TEMPORARY VIEW' ELSE NULL END END WHEN false THEN CASE c.relkind WHEN 'r' THEN 'TABLE' WHEN 'i' THEN 'INDEX' WHEN 'S' THEN 'SEQUENCE' WHEN 'v' THEN 'VIEW' WHEN 'c' THEN 'TYPE' WHEN 'f' THEN 'FOREIGN TABLE' WHEN 'm' THEN 'MATERIALIZED VIEW' ELSE NULL END ELSE NULL END AS TABL E_TYPE, d.description AS REMARKS FROM pg_catalog.pg_namespace n, pg_catalog.pg_class c L 18438 | testmine | 14117 | 10 | postgres | | 10.4.0.17 | | 48477 | 2018-04-06 17:44:15.139 331+08 | | 2018-04-06 17:44:15.14372+08 | 2018-04-06 17:44:15.143772+08 | f | idle | SHOW TRANSACTION ISOLATION LEVEL 18438 | testmine | 14119 | 10 | postgres | | 10.4.0.17 | | 48478 | 2018-04-06 17:44:15.145 721+08 | | 2018-04-06 17:44:15.149696+08 | 2018-04-06 17:44:15.149727+08 | f | idle | SHOW TRANSACTION ISOLATION LEVEL 18438 | testmine | 13619 | 10 | postgres | | 10.4.0.17 | | 48410 | 2018-04-06 17:35:45.137 484+08 | | 2018-04-06 17:35:45.143109+08 | 2018-04-06 17:35:45.143147+08 | f | idle | SHOW TRANSACTION ISOLATION LEVEL 18438 | testmine | 13942 | 10 | postgres | | 10.4.0.17 | | 48425 | 2018-04-06 17:43:45.141 417+08 | | 2018-04-06 17:43:45.145461+08 | 2018-04-06 17:43:45.145496+08 | f | idle | SHOW TRANSACTION ISOLATION LEVEL 18438 | testmine | 13944 | 10 | postgres | | 10.4.0.17 | | 48426 | 2018-04-06 17:43:45.146 769+08 | | 2018-04-06 17:43:45.150107+08 | 2018-04-06 17:43:45.150136+08 | f | idle | SHOW TRANSACTION ISOLATION LEVEL 18438 | testmine | 13946 | 10 | postgres | | 10.4.0.17 | | 48427 | 2018-04-06 17:43:45.152 21+08 | | 2018-04-06 17:43:45.155535+08 | 2018-04-06 17:43:45.155564+08 | f | idle | SHOW TRANSACTION ISOLATION LEVEL Best, Pengcheng On 2018-4-6 15:50, Julie Sullivan wrote: > You shouldn't need 500 connections. Human and flymine build fine with 20. > > 1. Is your code on github? Can I take a look? Specifically I am > interested in what your data sources are doing. > 2. Which version of postgres are you on? I recommend 9.3 but I am > using 9.5 without any issue. > 3. When this happens, when you see this error, what is the output of > `ps aux | grep postgres`. > 4. Do you see any hanging queries? `select * from pg_stat_activity` > will tell you which queries are running. > > No you shouldn't need to kill the postgres processes. I would have > thought that your earlier errors might have caused the database > connections to not close properly. This would account for the error > message you saw. However this would have been fixed by a postgres > restart - which you did. > > On 04/06/2018 08:48 AM, Pengcheng Yang wrote: >> Hi， >> >> I have resolved this problem by set "max_connections = 500" in >> /var/lib/pgsql/data/postgresql.conf file. >> >> Some useful links are: >> >> [1] >> https://stackoverflow.com/questions/2757549/org-postgresql-util-psqlexception-fatal-sorry-too-many-clients-already >> >> >> [2] >> https://stackoverflow.com/questions/30778015/how-to-increase-the-max-connections-in-postgres >> >> >> Best, >> >> Pengcheng >> >> >> On 2018-4-6 10:20, Pengcheng Yang wrote: >>> Hi miners, >>> >>> The similar error has been reported previously: >>> http://gmod.827538.n3.nabble.com/ratmine-data-loading-errors-td4029257.html >>> >>> >>> However, even I using project_build script. The error remain. >>> >>> 1) I have set the "db.production.datasource.maxConnections=150" in >>> the default.intermine.integrate.properties file. During the running >>> of the project_build, "ps -ef |grep postgres | grep idle |wc -l" >>> return as many as 167 processes. When I load the source one-by-one, >>> although one source was failed loaded during to "too many clients" >>> error, the post-process and the web deployment were successful. >>> >>> 2) I have also tested set the >>> "db.production.datasource.maxConnections=200". But the error message >>> suggest this number should not be as larger as 200. >>> >>> Further I found that many of the process of postgres were "idle". Is >>> it mean that these processes could be killed? But, I have tested to >>> kill the idle process, which resulted in the termination of the >>> building process. >>> >>> 3) I have tested to restart postgres server and then run >>> project_build again. Having tested many times, each time was stopped >>> at different steps with the same error "too many clients already" >>> >>> =================== error message============================ >>> >>> BUILD FAILED >>> /home/pengchy/soft/intermine/imbuild/integrate.xml:54: The following >>> error occurred while executing this line: >>> /home/pengchy/soft/intermine/imbuild/source.xml:201: Exception in >>> XmlDataLoaderTask >>> at >>> org.intermine.dataloader.XmlDataLoaderTask.execute(XmlDataLoaderTask.java:170) >>> >>> at >>> org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) >>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> >>> at java.lang.reflect.Method.invoke(Method.java:498) >>> at >>> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) >>> >>> at org.apache.tools.ant.Task.perform(Task.java:348) >>> at org.apache.tools.ant.Target.execute(Target.java:437) >>> at org.apache.tools.ant.Target.performTasks(Target.java:458) >>> at >>> org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) >>> at >>> org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36) >>> >>> at >>> org.apache.tools.ant.Project.executeTargets(Project.java:1260) >>> at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442) >>> at >>> org.intermine.task.Integrate.performAction(Integrate.java:219) >>> at >>> org.intermine.task.Integrate.performAction(Integrate.java:131) >>> at org.intermine.task.Integrate.execute(Integrate.java:123) >>> at >>> org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) >>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> >>> at java.lang.reflect.Method.invoke(Method.java:498) >>> at >>> org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) >>> >>> at org.apache.tools.ant.Task.perform(Task.java:348) >>> at org.apache.tools.ant.Target.execute(Target.java:437) >>> at org.apache.tools.ant.Target.performTasks(Target.java:458) >>> at >>> org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) >>> at >>> org.apache.tools.ant.Project.executeTarget(Project.java:1376) >>> at >>> org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) >>> >>> at >>> org.apache.tools.ant.Project.executeTargets(Project.java:1260) >>> at org.apache.tools.ant.Main.runBuild(Main.java:857) >>> at org.apache.tools.ant.Main.startAnt(Main.java:236) >>> at org.apache.tools.ant.launch.Launcher.run(Launcher.java:286) >>> at org.apache.tools.ant.launch.Launcher.main(Launcher.java:112) >>> Caused by: org.intermine.objectstore.ObjectStoreException: >>> org.intermine.objectstore.ObjectStoreException: Unable to get >>> database for InterMine ObjectStore >>> at >>> org.intermine.objectstore.ObjectStoreWriterFactory.getObjectStoreWriter(ObjectStoreWriterFactory.java:71) >>> >>> at >>> org.intermine.dataloader.IntegrationWriterDataTrackingImpl.getInstance(IntegrationWriterDataTrackingImpl.java:120) >>> >>> at >>> org.intermine.dataloader.IntegrationWriterDataTrackingImpl.getInstance(IntegrationWriterDataTrackingImpl.java:80) >>> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>> >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> >>> at java.lang.reflect.Method.invoke(Method.java:498) >>> at >>> org.intermine.dataloader.IntegrationWriterFactory.getIntegrationWriter(IntegrationWriterFactory.java:65) >>> >>> at >>> org.intermine.dataloader.XmlDataLoaderTask.execute(XmlDataLoaderTask.java:123) >>> >>> ... 31 more >>> Caused by: org.intermine.objectstore.ObjectStoreException: Unable to >>> get database for InterMine ObjectStore >>> at >>> org.intermine.objectstore.intermine.ObjectStoreInterMineImpl.getInstance(ObjectStoreInterMineImpl.java:358) >>> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>> >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> >>> at java.lang.reflect.Method.invoke(Method.java:498) >>> at >>> org.intermine.objectstore.ObjectStoreFactory.getObjectStore(ObjectStoreFactory.java:63) >>> >>> at >>> org.intermine.objectstore.ObjectStoreWriterFactory.getObjectStoreWriter(ObjectStoreWriterFactory.java:69) >>> >>> ... 39 more >>> Caused by: java.lang.RuntimeException: Failed to initialise >>> db.production >>> at >>> org.intermine.sql.DatabaseFactory.getDatabase(DatabaseFactory.java:58) >>> at >>> org.intermine.objectstore.intermine.ObjectStoreInterMineImpl.getInstance(ObjectStoreInterMineImpl.java:356) >>> >>> ... 45 more >>> Caused by: com.zaxxer.hikari.pool.PoolInitializationException: >>> Exception during pool initialization >>> at >>> com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:544) >>> >>> at >>> com.zaxxer.hikari.pool.BaseHikariPool.<init>(BaseHikariPool.java:171) >>> at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:60) >>> at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:48) >>> at >>> com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:80) >>> at org.intermine.sql.Database.<init>(Database.java:122) >>> at >>> org.intermine.sql.DatabaseFactory.getDatabase(DatabaseFactory.java:56) >>> ... 46 more >>> Caused by: org.postgresql.util.PSQLException: FATAL: sorry, too many >>> clients already >>> at >>> org.postgresql.Driver$ConnectThread.getResult(Driver.java:355) >>> at org.postgresql.Driver.connect(Driver.java:279) >>> at java.sql.DriverManager.getConnection(DriverManager.java:664) >>> at java.sql.DriverManager.getConnection(DriverManager.java:247) >>> at >>> org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:86) >>> >>> at >>> org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:71) >>> >>> at >>> com.zaxxer.hikari.pool.BaseHikariPool.addConnection(BaseHikariPool.java:438) >>> >>> at >>> com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:542) >>> >>> ... 52 more >>> >>> Total time: 22 seconds >>> Fri Apr 6 09:55:48 CST 2018 >>> >>> finished >>> >>> >>> ERROR: 256 >>> failed with exit code 0: ant -v -Dsource=entrez-organism >>> >>> >>> Best, >>> >>> Pengcheng >>> >>> >>> _______________________________________________ >>> dev mailing list >>> [hidden email] >>> https://lists.intermine.org/mailman/listinfo/dev >> >> _______________________________________________ >> dev mailing list >> [hidden email] >> https://lists.intermine.org/mailman/listinfo/dev > _______________________________________________ > dev mailing list > [hidden email] > https://lists.intermine.org/mailman/listinfo/dev _______________________________________________ dev mailing list [hidden email] https://lists.intermine.org/mailman/listinfo/dev |
Re: too many clients already
|
|
In reply to this post by Pengcheng Yang
For the record this was solved by not having a webapp running using the
same database that is being used by the build. On 04/06/2018 03:20 AM, Pengcheng Yang wrote: > Hi miners, > > The similar error has been reported previously: > http://gmod.827538.n3.nabble.com/ratmine-data-loading-errors-td4029257.html > > However, even I using project_build script. The error remain. > > 1) I have set the "db.production.datasource.maxConnections=150" in the > default.intermine.integrate.properties file. During the running of the > project_build, "ps -ef |grep postgres | grep idle |wc -l" return as many > as 167 processes. When I load the source one-by-one, although one source > was failed loaded during to "too many clients" error, the post-process > and the web deployment were successful. > > 2) I have also tested set the > "db.production.datasource.maxConnections=200". But the error message > suggest this number should not be as larger as 200. > > Further I found that many of the process of postgres were "idle". Is it > mean that these processes could be killed? But, I have tested to kill > the idle process, which resulted in the termination of the building > process. > > 3) I have tested to restart postgres server and then run project_build > again. Having tested many times, each time was stopped at different > steps with the same error "too many clients already" > > =================== error message============================ > > BUILD FAILED > /home/pengchy/soft/intermine/imbuild/integrate.xml:54: The following > error occurred while executing this line: > /home/pengchy/soft/intermine/imbuild/source.xml:201: Exception in > XmlDataLoaderTask > at > org.intermine.dataloader.XmlDataLoaderTask.execute(XmlDataLoaderTask.java:170) > > at > org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) > at org.apache.tools.ant.Task.perform(Task.java:348) > at org.apache.tools.ant.Target.execute(Target.java:437) > at org.apache.tools.ant.Target.performTasks(Target.java:458) > at > org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) > at > org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:36) > > at org.apache.tools.ant.Project.executeTargets(Project.java:1260) > at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442) > at org.intermine.task.Integrate.performAction(Integrate.java:219) > at org.intermine.task.Integrate.performAction(Integrate.java:131) > at org.intermine.task.Integrate.execute(Integrate.java:123) > at > org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293) > at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106) > at org.apache.tools.ant.Task.perform(Task.java:348) > at org.apache.tools.ant.Target.execute(Target.java:437) > at org.apache.tools.ant.Target.performTasks(Target.java:458) > at > org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405) > at org.apache.tools.ant.Project.executeTarget(Project.java:1376) > at > org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41) > > at org.apache.tools.ant.Project.executeTargets(Project.java:1260) > at org.apache.tools.ant.Main.runBuild(Main.java:857) > at org.apache.tools.ant.Main.startAnt(Main.java:236) > at org.apache.tools.ant.launch.Launcher.run(Launcher.java:286) > at org.apache.tools.ant.launch.Launcher.main(Launcher.java:112) > Caused by: org.intermine.objectstore.ObjectStoreException: > org.intermine.objectstore.ObjectStoreException: Unable to get database > for InterMine ObjectStore > at > org.intermine.objectstore.ObjectStoreWriterFactory.getObjectStoreWriter(ObjectStoreWriterFactory.java:71) > > at > org.intermine.dataloader.IntegrationWriterDataTrackingImpl.getInstance(IntegrationWriterDataTrackingImpl.java:120) > > at > org.intermine.dataloader.IntegrationWriterDataTrackingImpl.getInstance(IntegrationWriterDataTrackingImpl.java:80) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.intermine.dataloader.IntegrationWriterFactory.getIntegrationWriter(IntegrationWriterFactory.java:65) > > at > org.intermine.dataloader.XmlDataLoaderTask.execute(XmlDataLoaderTask.java:123) > > ... 31 more > Caused by: org.intermine.objectstore.ObjectStoreException: Unable to get > database for InterMine ObjectStore > at > org.intermine.objectstore.intermine.ObjectStoreInterMineImpl.getInstance(ObjectStoreInterMineImpl.java:358) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.intermine.objectstore.ObjectStoreFactory.getObjectStore(ObjectStoreFactory.java:63) > > at > org.intermine.objectstore.ObjectStoreWriterFactory.getObjectStoreWriter(ObjectStoreWriterFactory.java:69) > > ... 39 more > Caused by: java.lang.RuntimeException: Failed to initialise db.production > at > org.intermine.sql.DatabaseFactory.getDatabase(DatabaseFactory.java:58) > at > org.intermine.objectstore.intermine.ObjectStoreInterMineImpl.getInstance(ObjectStoreInterMineImpl.java:356) > > ... 45 more > Caused by: com.zaxxer.hikari.pool.PoolInitializationException: Exception > during pool initialization > at > com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:544) > > at > com.zaxxer.hikari.pool.BaseHikariPool.<init>(BaseHikariPool.java:171) > at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:60) > at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:48) > at > com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:80) > at org.intermine.sql.Database.<init>(Database.java:122) > at > org.intermine.sql.DatabaseFactory.getDatabase(DatabaseFactory.java:56) > ... 46 more > Caused by: org.postgresql.util.PSQLException: FATAL: sorry, too many > clients already > at org.postgresql.Driver$ConnectThread.getResult(Driver.java:355) > at org.postgresql.Driver.connect(Driver.java:279) > at java.sql.DriverManager.getConnection(DriverManager.java:664) > at java.sql.DriverManager.getConnection(DriverManager.java:247) > at > org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:86) > > at > org.postgresql.ds.common.BaseDataSource.getConnection(BaseDataSource.java:71) > > at > com.zaxxer.hikari.pool.BaseHikariPool.addConnection(BaseHikariPool.java:438) > > at > com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:542) > > ... 52 more > > Total time: 22 seconds > Fri Apr 6 09:55:48 CST 2018 > > finished > > > ERROR: 256 > failed with exit code 0: ant -v -Dsource=entrez-organism > > > Best, > > Pengcheng > > > _______________________________________________ > dev mailing list > [hidden email] > https://lists.intermine.org/mailman/listinfo/dev dev mailing list [hidden email] https://lists.intermine.org/mailman/listinfo/dev |
«
Return to InterMine
|
1 view|%1 views
| Free forum by Nabble | Edit this page |