sleeping mysql processes w/Bio::DB::SeqFeature::Store

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

sleeping mysql processes w/Bio::DB::SeqFeature::Store

Jim Hu
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Lincoln Stein
What problems did you see with the user login system? I am working on that subsystem now.

Odd that you would see any sleeping processes when running under vanilla CGI. The gbrowse process goes away after finishing the transaction.

Lincoln

On Monday, June 24, 2013, Jim Hu wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
<a href="javascript:;" onclick="_e(event, &#39;cvml&#39;, &#39;Gmod-gbrowse@lists.sourceforge.net&#39;)">Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Jim Hu
This may have been fixed already: the login system was spawning a huge number of mysql threads when fastcgi was enabled.... to the point where gbrowse was crashing.  For a while we were running a cron job that killed mysql processes that were where the time was > some value.  Not pretty.

We spent a little time trying to figure it out, but Nathan couldn't get the fastcgi to work on our VMs.

I agree that it's odd to see the sleeping processes.  I think some of them are being called by gbrowse_img. Could there be a difference there somewhere? 

Using 2.52 currently.

Jim

On Jun 24, 2013, at 10:07 AM, Lincoln Stein wrote:

What problems did you see with the user login system? I am working on that subsystem now.

Odd that you would see any sleeping processes when running under vanilla CGI. The gbrowse process goes away after finishing the transaction.

Lincoln

On Monday, June 24, 2013, Jim Hu wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
<a href="javascript:;" onclick="_e(event, 'cvml', 'Gmod-gbrowse@lists.sourceforge.net')">Gmod-gbrowse@...
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Lincoln Stein
Hi Jim,

I've been playing with this using the current GBrowse version 2.54 (from git) and Bio::Graphics 2.37. I can't document a process leak with either normal cgi-bin and FastCGI. When running under fastcgi there is one sleeping mysql process per database per FastCGI instance; this is expected, because the FastCGI instances cache database handles. Under normal CGI, I see the database handles being created as needed to display the tracks and then destroyed once the request is finished. There are no sleeping mysql processes after all tracks are rendered.

I can confirm that CGI is a lot slower than FastCGI. This is because the entire code and database has to be loaded for each displayed track. 

I tried to reproduce the accumulation of FastCGI processes that you reported being spawned by the login process, but couldn't do reproduce it. If you can help me reproduce the problems you experienced with FastCGI, I'd love to try to help you out.

Lincoln


On Mon, Jun 24, 2013 at 6:06 PM, Jim Hu <[hidden email]> wrote:
This may have been fixed already: the login system was spawning a huge number of mysql threads when fastcgi was enabled.... to the point where gbrowse was crashing.  For a while we were running a cron job that killed mysql processes that were where the time was > some value.  Not pretty.

We spent a little time trying to figure it out, but Nathan couldn't get the fastcgi to work on our VMs.

I agree that it's odd to see the sleeping processes.  I think some of them are being called by gbrowse_img. Could there be a difference there somewhere? 

Using 2.52 currently.

Jim

On Jun 24, 2013, at 10:07 AM, Lincoln Stein wrote:

What problems did you see with the user login system? I am working on that subsystem now.

Odd that you would see any sleeping processes when running under vanilla CGI. The gbrowse process goes away after finishing the transaction.

Lincoln

On Monday, June 24, 2013, Jim Hu wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054





--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Martin Mokrejs
In reply to this post by Jim Hu
Hi Jim
  to answer your original questions. The Info column tells you the mysql connections
are not waiting for anything to be done on the mysel server side, they are just idle,
cached connections or not closed explicitly by gbrowse. Maybe that has to do with the
way you run your webserver (Apache). It does not close all forked child processes unles
they are above certain count.

  Nevertheless, I recommend you to enable mysql logging, either the slow mysql.log file
or use the binary logging format. Then, you can lookup what those "Id"s did until they
went idle. The logfiles provide timing information so it should be easy to find out which
routine caused what commands. Please refer to http://dev.mysql.com/doc/refman/5.7/en/mysql-logging.html
for details how to enable that.

  I don't think you are facing performance issues because Info is NULL (clients got what
they asked for) so you should focus on finding why gbrowse does not explicitly close
some connections (maybe each thread opened more than one?) or whether you want to make
Apache webserver to close forked processes (hint: worker/prefork/... modes).

Martin


Jim Hu wrote:

> This may have been fixed already: the login system was spawning a huge number of mysql threads when fastcgi was enabled.... to the point where gbrowse was crashing.  For a while we were running a cron job that killed mysql processes that were where the time was > some value.  Not pretty.
>
> We spent a little time trying to figure it out, but Nathan couldn't get the fastcgi to work on our VMs.
>
> I agree that it's odd to see the sleeping processes.  I think some of them are being called by gbrowse_img. Could there be a difference there somewhere?
>
> Using 2.52 currently.
>
> Jim
>
> On Jun 24, 2013, at 10:07 AM, Lincoln Stein wrote:
>
>> What problems did you see with the user login system? I am working on that subsystem now.
>>
>> Odd that you would see any sleeping processes when running under vanilla CGI. The gbrowse process goes away after finishing the transaction.
>>
>> Lincoln
>>
>> On Monday, June 24, 2013, Jim Hu wrote:
>>
>>     I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:
>>
>>     mysql> show full processlist;
>>     +--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
>>     | Id     | User         | Host      | db              | Command | Time  | State | Info                  |
>>     +--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
>>     | 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
>>     | 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
>>     | 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
>>     | 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
>>     | 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
>>     | 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
>>     | 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
>>     | 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
>>     | 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
>>     | 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
>>     | 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
>>     | 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
>>     | 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
>>     | 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
>>     | 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
>>     | 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
>>     | 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
>>     | 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
>>     | 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
>>     | 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
>>     | 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
>>     | 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
>>     | 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
>>     | 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
>>     | 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
>>     | 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
>>     | 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
>>     | 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
>>     | 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
>>     +--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
>>     29 rows in set (0.00 sec)
>>
>>     This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:
>>
>>     - are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
>>     - do people use the default mysql wait_timeout?
>>     - can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.
>>
>>     Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.
>>
>>     Jim
>>
>>     =====================================
>>     Jim Hu
>>     Professor
>>     Dept. of Biochemistry and Biophysics
>>     2128 TAMU
>>     Texas A&M Univ.
>>     College Station, TX 77843-2128
>>     979-862-4054
>>
>>
>>
>>     ------------------------------------------------------------------------------
>>     This SF.net <http://SF.net> email is sponsored by Windows:
>>
>>     Build for Windows Store.
>>
>>     http://p.sf.net/sfu/windows-dev2dev
>>     _______________________________________________
>>     Gmod-gbrowse mailing list
>>     [hidden email] <javascript:;>
>>     https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>>
>>
>>
>> --
>> Lincoln D. Stein
>> Director, Informatics and Biocomputing Platform
>> Ontario Institute for Cancer Research
>> 101 College St., Suite 800
>> Toronto, ON, Canada M5G0A3
>> 416 673-8514
>> Assistant: Renata Musa <[hidden email] <mailto:[hidden email]>>
>
> =====================================
> Jim Hu
> Professor
> Dept. of Biochemistry and Biophysics
> 2128 TAMU
> Texas A&M Univ.
> College Station, TX 77843-2128
> 979-862-4054
>
>
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
>
>
>
> _______________________________________________
> Gmod-gbrowse mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Frédéric Sapet
In reply to this post by Jim Hu
Hello Jim

I'm facing the same problem (under FastCGI) : 176 process...

GGB 2.54
This is perl, v5.8.8 built for x86_64-linux-thread-multi
BioPerl is 1.006901
Bio::Graphics 2.32


gnpgenome_sessions is my users database.

xxxx and ArabidoV9 are my annotations databases.       

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+

| Id    | User      | Host                      | db                 | Command | Time  | State | Info                  |

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+

| 52849 | gnpgenome | xxx.xxx.xxx.xxx:56047     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 52857 | gnpgenome | xxx.xxx.xxx.xxx:56056     | Xxxxxxx            | Sleep   | 10027 |       | NULL                  |

| 52859 | gnpgenome | xxx.xxx.xxx.xxx:56059     | gnpgenome_sessions | Sleep   |  2263 |       | NULL                  |

| 53262 | gnpgenome | xxx.xxx.xxx.xxx:56339     | Xxxxxxx            | Sleep   |  2123 |       | NULL                  |

| 53264 | gnpgenome | xxx.xxx.xxx.xxx:56342     | gnpgenome_sessions | Sleep   |  2129 |       | NULL                  |

| 53360 | gnpgenome | xxx.xxx.xxx.xxx:56389     | Xxxxxxx            | Sleep   |  4952 |       | NULL                  |

| 53447 | gnpgenome | xxxxxxx.biogemma.fr:58547 | Xxxxxxx            | Sleep   |  4954 |       | NULL                  |

| 53473 | gnpgenome | xxx.xxx.xxx.xxx:56462     | Xxxxxxx            | Sleep   |  4954 |       | NULL                  |

| 53494 | gnpgenome | xxx.xxx.xxx.xxx:56464     | Xxxxxxxxxxx        | Sleep   |  4952 |       | NULL                  |

| 53565 | gnpgenome | xxx.xxx.xxx.xxx:56540     | Xxxxxxx            | Sleep   |  4951 |       | NULL                  |

| 53598 | gnpgenome | xxxxxxx.biogemma.fr:58592 | Xxxxxxx            | Sleep   |  4950 |       | NULL                  |

| 53661 | gnpgenome | xxxxxxx.biogemma.fr:58653 | Xxxxxxxxxxx        | Sleep   |  4949 |       | NULL                  |

| 53684 | gnpgenome | xxxxxxx.biogemma.fr:58676 | Xxxxxxxxxxx        | Sleep   |  4949 |       | NULL                  |

| 53744 | gnpgenome | xxx.xxx.xxx.xxx:56630     | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53805 | gnpgenome | xxxxxxx.biogemma.fr:58687 | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53815 | gnpgenome | xxxxxxx.biogemma.fr:58691 | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53916 | gnpgenome | xxx.xxx.xxx.xxx:56740     | Xxxxxxxxxxxxxx     | Sleep   |  4784 |       | NULL                  |

| 53953 | gnpgenome | xxxxxxx.biogemma.fr:58755 | Xxxxxxxxxxxxxx     | Sleep   |  4885 |       | NULL                  |

| 53966 | gnpgenome | xxxxxxx.biogemma.fr:58768 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53979 | gnpgenome | xxxxxxx.biogemma.fr:58781 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53987 | gnpgenome | xxxxxxx.biogemma.fr:58788 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53996 | gnpgenome | xxxxxxx.biogemma.fr:58797 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 54060 | gnpgenome | xxx.xxx.xxx.xxx:56820     | Xxxxxxxxxxxxxx     | Sleep   |  4815 |       | NULL                  |

| 54111 | gnpgenome | xxxxxxx.biogemma.fr:58832 | Xxxxxxxxxxxxxx     | Sleep   |  4870 |       | NULL                  |

| 54123 | gnpgenome | xxxxxxx.biogemma.fr:58844 | Xxxxxxxxxxxxxx     | Sleep   |  4870 |       | NULL                  |

| 54296 | gnpgenome | xxx.xxx.xxx.xxx:56938     | gnpgenome_sessions | Sleep   |  4812 |       | NULL                  |

| 54297 | gnpgenome | xxx.xxx.xxx.xxx:56939     | gnpgenome_sessions | Sleep   |  4494 |       | NULL                  |

| 54372 | gnpgenome | xxxxxxx.biogemma.fr:36159 | Xxxxxxxxxxxxxx     | Sleep   |  4804 |       | NULL                  |

| 54949 | gnpgenome | xxx.xxx.xxx.xxx:57274     | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 54951 | gnpgenome | xxx.xxx.xxx.xxx:57277     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 54987 | gnpgenome | xxx.xxx.xxx.xxx:57317     | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 54998 | gnpgenome | xxxxxxx.biogemma.fr:36502 | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 55033 | gnpgenome | xxxxxxx.biogemma.fr:36535 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55044 | gnpgenome | xxxxxxx.biogemma.fr:36546 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55045 | gnpgenome | xxxxxxx.biogemma.fr:36547 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55156 | gnpgenome | xxx.xxx.xxx.xxx:57348     | Xxxxxxx            | Sleep   |  2290 |       | NULL                  |

| 55784 | gnpgenome | xxx.xxx.xxx.xxx:57813     | Xxxxxxxxxxx        | Sleep   |  2594 |       | NULL                  |

| 55809 | gnpgenome | xxx.xxx.xxx.xxx:57838     | gnpgenome_sessions | Sleep   |  2556 |       | NULL                  |

| 55812 | gnpgenome | xxx.xxx.xxx.xxx:57841     | ArabidoV9          | Sleep   |  2522 |       | NULL                  |

| 55920 | gnpgenome | xxxxxxx.biogemma.fr:49728 | ArabidoV9          | Sleep   |  2463 |       | NULL                  |

| 55962 | gnpgenome | xxxxxxx.biogemma.fr:49769 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55973 | gnpgenome | xxxxxxx.biogemma.fr:49781 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55974 | gnpgenome | xxxxxxx.biogemma.fr:49782 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55975 | gnpgenome | xxxxxxx.biogemma.fr:49783 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 56041 | gnpgenome | xxx.xxx.xxx.xxx:57970     | gnpgenome_sessions | Sleep   |  2335 |       | NULL                  |

| 56050 | gnpgenome | xxx.xxx.xxx.xxx:57987     | ArabidoV9          | Sleep   |  2399 |       | NULL                  |

| 56135 | gnpgenome | xxx.xxx.xxx.xxx:58075     | ArabidoV9          | Sleep   |  2302 |       | NULL                  |

| 56136 | gnpgenome | xxx.xxx.xxx.xxx:58076     | Xxxxxxx            | Sleep   |  2302 |       | NULL                  |

| 56171 | gnpgenome | xxx.xxx.xxx.xxx:58112     | Xxxxxxxxx          | Sleep   |  2301 |       | NULL                  |

| 56174 | gnpgenome | xxx.xxx.xxx.xxx:58115     | Xxxxxxxxxxx        | Sleep   |  2301 |       | NULL                  |

| 56175 | gnpgenome | xxx.xxx.xxx.xxx:58116     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56176 | gnpgenome | xxx.xxx.xxx.xxx:58117     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56177 | gnpgenome | xxx.xxx.xxx.xxx:58118     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56226 | gnpgenome | xxx.xxx.xxx.xxx:58131     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56227 | gnpgenome | xxx.xxx.xxx.xxx:58132     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56228 | gnpgenome | xxx.xxx.xxx.xxx:58133     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56229 | gnpgenome | xxx.xxx.xxx.xxx:58134     | Xxxxxxxxxxxxxx     | Sleep   |  2300 |       | NULL                  |

| 56230 | gnpgenome | xxx.xxx.xxx.xxx:58135     | Xxxxxxxxxxx        | Sleep   |  2300 |       | NULL                  |

| 56231 | gnpgenome | xxx.xxx.xxx.xxx:58136     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56232 | gnpgenome | xxx.xxx.xxx.xxx:58137     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56233 | gnpgenome | xxx.xxx.xxx.xxx:58138     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56236 | gnpgenome | xxx.xxx.xxx.xxx:58141     | ArabidoV9          | Sleep   |  2300 |       | NULL                  |

| 56237 | gnpgenome | xxx.xxx.xxx.xxx:58142     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56238 | gnpgenome | xxx.xxx.xxx.xxx:58143     | Xxxxxxxxx          | Sleep   |  2300 |       | NULL                  |

| 56239 | gnpgenome | xxx.xxx.xxx.xxx:58144     | Xxxxxxxxxxx        | Sleep   |  2300 |       | NULL                  |

| 56240 | gnpgenome | xxx.xxx.xxx.xxx:58145     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56241 | gnpgenome | xxx.xxx.xxx.xxx:58146     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56242 | gnpgenome | xxx.xxx.xxx.xxx:58147     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56273 | gnpgenome | xxx.xxx.xxx.xxx:58178     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56274 | gnpgenome | xxx.xxx.xxx.xxx:58179     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56275 | gnpgenome | xxx.xxx.xxx.xxx:58180     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56276 | gnpgenome | xxx.xxx.xxx.xxx:58181     | Xxxxxxxxxxxxxx     | Sleep   |  2299 |       | NULL                  |

| 56277 | gnpgenome | xxx.xxx.xxx.xxx:58182     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56278 | gnpgenome | xxx.xxx.xxx.xxx:58183     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56279 | gnpgenome | xxx.xxx.xxx.xxx:58184     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56280 | gnpgenome | xxx.xxx.xxx.xxx:58185     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56317 | gnpgenome | xxx.xxx.xxx.xxx:58210     | ArabidoV9          | Sleep   |  2299 |       | NULL                  |

| 56318 | gnpgenome | xxx.xxx.xxx.xxx:58211     | ArabidoV9          | Sleep   |  2299 |       | NULL                  |

| 56322 | gnpgenome | xxx.xxx.xxx.xxx:58212     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56323 | gnpgenome | xxx.xxx.xxx.xxx:58213     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56325 | gnpgenome | xxx.xxx.xxx.xxx:58214     | Xxxxxxxxx          | Sleep   |  2299 |       | NULL                  |

| 56326 | gnpgenome | xxx.xxx.xxx.xxx:58215     | Xxxxxxxxx          | Sleep   |  2299 |       | NULL                  |

| 56327 | gnpgenome | xxx.xxx.xxx.xxx:58216     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56328 | gnpgenome | xxx.xxx.xxx.xxx:58217     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56329 | gnpgenome | xxx.xxx.xxx.xxx:58218     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56330 | gnpgenome | xxx.xxx.xxx.xxx:58219     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56331 | gnpgenome | xxx.xxx.xxx.xxx:58220     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56332 | gnpgenome | xxx.xxx.xxx.xxx:58221     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56333 | gnpgenome | xxx.xxx.xxx.xxx:58222     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56334 | gnpgenome | xxx.xxx.xxx.xxx:58223     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56340 | gnpgenome | xxx.xxx.xxx.xxx:58229     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56341 | gnpgenome | xxx.xxx.xxx.xxx:58230     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56342 | gnpgenome | xxx.xxx.xxx.xxx:58231     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56343 | gnpgenome | xxx.xxx.xxx.xxx:58232     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56345 | gnpgenome | xxx.xxx.xxx.xxx:58234     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56346 | gnpgenome | xxx.xxx.xxx.xxx:58235     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56347 | gnpgenome | xxx.xxx.xxx.xxx:58236     | Xxxxxxxxxxxxxx     | Sleep   |  2298 |       | NULL                  |

| 56348 | gnpgenome | xxx.xxx.xxx.xxx:58237     | Xxxxxxxxxxxxxx     | Sleep   |  2298 |       | NULL                  |

| 56349 | gnpgenome | xxx.xxx.xxx.xxx:58238     | Xxxxxxxxxxx        | Sleep   |  2298 |       | NULL                  |

| 56350 | gnpgenome | xxx.xxx.xxx.xxx:58239     | Xxxxxxxxxxx        | Sleep   |  2298 |       | NULL                  |

| 56351 | gnpgenome | xxx.xxx.xxx.xxx:58240     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56352 | gnpgenome | xxx.xxx.xxx.xxx:58241     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56353 | gnpgenome | xxx.xxx.xxx.xxx:58242     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56354 | gnpgenome | xxx.xxx.xxx.xxx:58243     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56355 | gnpgenome | xxx.xxx.xxx.xxx:58244     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56357 | gnpgenome | xxx.xxx.xxx.xxx:58246     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56411 | gnpgenome | xxx.xxx.xxx.xxx:58276     | Xxxxxxx            | Sleep   |  2278 |       | NULL                  |

| 56413 | gnpgenome | xxx.xxx.xxx.xxx:58279     | gnpgenome_sessions | Sleep   |  2020 |       | NULL                  |

| 56423 | gnpgenome | xxx.xxx.xxx.xxx:58289     | Xxxxxxx            | Sleep   |  2218 |       | NULL                  |

| 56440 | gnpgenome | xxx.xxx.xxx.xxx:58305     | ArabidoV9          | Sleep   |  2292 |       | NULL                  |

| 56442 | gnpgenome | xxx.xxx.xxx.xxx:58307     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56443 | gnpgenome | xxx.xxx.xxx.xxx:58308     | Xxxxxxxxx          | Sleep   |  2292 |       | NULL                  |

| 56448 | gnpgenome | xxx.xxx.xxx.xxx:58309     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56449 | gnpgenome | xxx.xxx.xxx.xxx:58310     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56450 | gnpgenome | xxx.xxx.xxx.xxx:58311     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56452 | gnpgenome | xxx.xxx.xxx.xxx:58313     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56477 | gnpgenome | xxx.xxx.xxx.xxx:58319     | ArabidoV9          | Sleep   |  2292 |       | NULL                  |

| 56478 | gnpgenome | xxx.xxx.xxx.xxx:58320     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56479 | gnpgenome | xxx.xxx.xxx.xxx:58321     | Xxxxxxxxx          | Sleep   |  2292 |       | NULL                  |

| 56480 | gnpgenome | xxx.xxx.xxx.xxx:58322     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56481 | gnpgenome | xxx.xxx.xxx.xxx:58323     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56482 | gnpgenome | xxx.xxx.xxx.xxx:58324     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56483 | gnpgenome | xxx.xxx.xxx.xxx:58325     | Xxxxxxxxxxxxxx     | Sleep   |  2292 |       | NULL                  |

| 56485 | gnpgenome | xxx.xxx.xxx.xxx:58327     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56486 | gnpgenome | xxx.xxx.xxx.xxx:58328     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56487 | gnpgenome | xxx.xxx.xxx.xxx:58329     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56488 | gnpgenome | xxx.xxx.xxx.xxx:58330     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56489 | gnpgenome | xxx.xxx.xxx.xxx:58331     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56490 | gnpgenome | xxx.xxx.xxx.xxx:58332     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56491 | gnpgenome | xxx.xxx.xxx.xxx:58333     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56492 | gnpgenome | xxx.xxx.xxx.xxx:58334     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56521 | gnpgenome | xxx.xxx.xxx.xxx:58363     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56531 | gnpgenome | xxx.xxx.xxx.xxx:58374     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56532 | gnpgenome | xxx.xxx.xxx.xxx:58375     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56534 | gnpgenome | xxx.xxx.xxx.xxx:58377     | Xxxxxxxxxxxxxx     | Sleep   |  2291 |       | NULL                  |

| 56536 | gnpgenome | xxx.xxx.xxx.xxx:58379     | Xxxxxxxxxxx        | Sleep   |  2291 |       | NULL                  |

| 56537 | gnpgenome | xxx.xxx.xxx.xxx:58380     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56539 | gnpgenome | xxx.xxx.xxx.xxx:58381     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56542 | gnpgenome | xxx.xxx.xxx.xxx:58382     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56600 | gnpgenome | xxx.xxx.xxx.xxx:58421     | ArabidoV9          | Sleep   |  2290 |       | NULL                  |

| 56631 | gnpgenome | xxx.xxx.xxx.xxx:58438     | Xxxxxxx            | Sleep   |  2289 |       | NULL                  |

| 56658 | gnpgenome | xxx.xxx.xxx.xxx:58447     | Xxxxxxxxx          | Sleep   |  2285 |       | NULL                  |

| 56671 | gnpgenome | xxx.xxx.xxx.xxx:58451     | Xxxxxxxxxxx        | Sleep   |  2283 |       | NULL                  |

| 56683 | gnpgenome | xxx.xxx.xxx.xxx:58453     | gnpgenome_sessions | Sleep   |  2282 |       | NULL                  |

| 56685 | gnpgenome | xxx.xxx.xxx.xxx:58455     | gnpgenome_sessions | Sleep   |  2281 |       | NULL                  |

| 56690 | gnpgenome | xxx.xxx.xxx.xxx:58460     | gnpgenome_sessions | Sleep   |  2279 |       | NULL                  |

| 56733 | gnpgenome | xxx.xxx.xxx.xxx:58494     | gnpgenome_sessions | Sleep   |  2262 |       | NULL                  |

| 56738 | gnpgenome | xxx.xxx.xxx.xxx:58499     | gnpgenome_sessions | Sleep   |  2262 |       | NULL                  |

| 56750 | gnpgenome | xxx.xxx.xxx.xxx:58511     | gnpgenome_sessions | Sleep   |  2263 |       | NULL                  |

| 56753 | gnpgenome | xxx.xxx.xxx.xxx:58516     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 56761 | gnpgenome | xxx.xxx.xxx.xxx:58518     | gnpgenome_sessions | Sleep   |  2254 |       | NULL                  |

| 56769 | gnpgenome | xxx.xxx.xxx.xxx:58520     | Xxxxxxxxx          | Sleep   |  2262 |       | NULL                  |

| 56770 | gnpgenome | xxx.xxx.xxx.xxx:58521     | Xxxxxxxxxxxxxx     | Sleep   |  2262 |       | NULL                  |

| 56771 | gnpgenome | xxx.xxx.xxx.xxx:58522     | Xxxxxxxxxxx        | Sleep   |  2262 |       | NULL                  |

| 56778 | gnpgenome | xxx.xxx.xxx.xxx:58523     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 56779 | gnpgenome | xxx.xxx.xxx.xxx:58524     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56780 | gnpgenome | xxx.xxx.xxx.xxx:58525     | ArabidoV9          | Sleep   |  2261 |       | NULL                  |

| 56781 | gnpgenome | xxx.xxx.xxx.xxx:58526     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56782 | gnpgenome | xxx.xxx.xxx.xxx:58527     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56783 | gnpgenome | xxx.xxx.xxx.xxx:58528     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56790 | gnpgenome | xxx.xxx.xxx.xxx:58529     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56791 | gnpgenome | xxx.xxx.xxx.xxx:58530     | Xxxxxxxxx          | Sleep   |  2261 |       | NULL                  |

| 56792 | gnpgenome | xxx.xxx.xxx.xxx:58531     | Xxxxxxxxxxxxxx     | Sleep   |  2261 |       | NULL                  |

| 56793 | gnpgenome | xxx.xxx.xxx.xxx:58532     | Xxxxxxxxxxx        | Sleep   |  2261 |       | NULL                  |

| 56795 | gnpgenome | xxx.xxx.xxx.xxx:58534     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56796 | gnpgenome | xxx.xxx.xxx.xxx:58535     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56797 | gnpgenome | xxx.xxx.xxx.xxx:58536     | ArabidoV9          | Sleep   |  2261 |       | NULL                  |

| 56798 | gnpgenome | xxx.xxx.xxx.xxx:58537     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56799 | gnpgenome | xxx.xxx.xxx.xxx:58538     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56800 | gnpgenome | xxx.xxx.xxx.xxx:58539     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56827 | gnpgenome | xxx.xxx.xxx.xxx:58560     | Xxxxxxx            | Sleep   |  2241 |       | NULL                  |

| 56829 | gnpgenome | xxx.xxx.xxx.xxx:58563     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57019 | gnpgenome | xxx.xxx.xxx.xxx:58642     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57021 | gnpgenome | xxx.xxx.xxx.xxx:58644     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57043 | gnpgenome | xxx.xxx.xxx.xxx:58672     | gnpgenome_sessions | Sleep   |  2020 |       | NULL                  |

| 57121 | gnpgenome | xxx.xxx.xxx.xxx:58676     | Xxxxxxx            | Sleep   |   542 |       | NULL                  |

| 57144 | gnpgenome | xxxxxxx.biogemma.fr:43501 | NULL               | Query   |     0 | NULL  | show full processlist |

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+
177 rows in set (0.00 sec)


Fred

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Lincoln Stein
How many apache processes are running simultaneously? Are you using the prefork or the threaded apache?

Lincoln


On Fri, Jul 5, 2013 at 9:08 AM, Frédéric Sapet <[hidden email]> wrote:
Hello Jim

I'm facing the same problem (under FastCGI) : 176 process...

GGB 2.54
This is perl, v5.8.8 built for x86_64-linux-thread-multi
BioPerl is 1.006901
Bio::Graphics 2.32


gnpgenome_sessions is my users database.

xxxx and ArabidoV9 are my annotations databases.       

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+


| Id    | User      | Host                      | db                 | Command | Time  | State | Info                  |

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+

| 52849 | gnpgenome | xxx.xxx.xxx.xxx:56047     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 52857 | gnpgenome | xxx.xxx.xxx.xxx:56056     | Xxxxxxx            | Sleep   | 10027 |       | NULL                  |

| 52859 | gnpgenome | xxx.xxx.xxx.xxx:56059     | gnpgenome_sessions | Sleep   |  2263 |       | NULL                  |

| 53262 | gnpgenome | xxx.xxx.xxx.xxx:56339     | Xxxxxxx            | Sleep   |  2123 |       | NULL                  |

| 53264 | gnpgenome | xxx.xxx.xxx.xxx:56342     | gnpgenome_sessions | Sleep   |  2129 |       | NULL                  |

| 53360 | gnpgenome | xxx.xxx.xxx.xxx:56389     | Xxxxxxx            | Sleep   |  4952 |       | NULL                  |

| 53447 | gnpgenome | xxxxxxx.biogemma.fr:58547 | Xxxxxxx            | Sleep   |  4954 |       | NULL                  |

| 53473 | gnpgenome | xxx.xxx.xxx.xxx:56462     | Xxxxxxx            | Sleep   |  4954 |       | NULL                  |

| 53494 | gnpgenome | xxx.xxx.xxx.xxx:56464     | Xxxxxxxxxxx        | Sleep   |  4952 |       | NULL                  |

| 53565 | gnpgenome | xxx.xxx.xxx.xxx:56540     | Xxxxxxx            | Sleep   |  4951 |       | NULL                  |

| 53598 | gnpgenome | xxxxxxx.biogemma.fr:58592 | Xxxxxxx            | Sleep   |  4950 |       | NULL                  |

| 53661 | gnpgenome | xxxxxxx.biogemma.fr:58653 | Xxxxxxxxxxx        | Sleep   |  4949 |       | NULL                  |

| 53684 | gnpgenome | xxxxxxx.biogemma.fr:58676 | Xxxxxxxxxxx        | Sleep   |  4949 |       | NULL                  |

| 53744 | gnpgenome | xxx.xxx.xxx.xxx:56630     | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53805 | gnpgenome | xxxxxxx.biogemma.fr:58687 | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53815 | gnpgenome | xxxxxxx.biogemma.fr:58691 | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53916 | gnpgenome | xxx.xxx.xxx.xxx:56740     | Xxxxxxxxxxxxxx     | Sleep   |  4784 |       | NULL                  |

| 53953 | gnpgenome | xxxxxxx.biogemma.fr:58755 | Xxxxxxxxxxxxxx     | Sleep   |  4885 |       | NULL                  |

| 53966 | gnpgenome | xxxxxxx.biogemma.fr:58768 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53979 | gnpgenome | xxxxxxx.biogemma.fr:58781 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53987 | gnpgenome | xxxxxxx.biogemma.fr:58788 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53996 | gnpgenome | xxxxxxx.biogemma.fr:58797 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 54060 | gnpgenome | xxx.xxx.xxx.xxx:56820     | Xxxxxxxxxxxxxx     | Sleep   |  4815 |       | NULL                  |

| 54111 | gnpgenome | xxxxxxx.biogemma.fr:58832 | Xxxxxxxxxxxxxx     | Sleep   |  4870 |       | NULL                  |

| 54123 | gnpgenome | xxxxxxx.biogemma.fr:58844 | Xxxxxxxxxxxxxx     | Sleep   |  4870 |       | NULL                  |

| 54296 | gnpgenome | xxx.xxx.xxx.xxx:56938     | gnpgenome_sessions | Sleep   |  4812 |       | NULL                  |

| 54297 | gnpgenome | xxx.xxx.xxx.xxx:56939     | gnpgenome_sessions | Sleep   |  4494 |       | NULL                  |

| 54372 | gnpgenome | xxxxxxx.biogemma.fr:36159 | Xxxxxxxxxxxxxx     | Sleep   |  4804 |       | NULL                  |

| 54949 | gnpgenome | xxx.xxx.xxx.xxx:57274     | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 54951 | gnpgenome | xxx.xxx.xxx.xxx:57277     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 54987 | gnpgenome | xxx.xxx.xxx.xxx:57317     | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 54998 | gnpgenome | xxxxxxx.biogemma.fr:36502 | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 55033 | gnpgenome | xxxxxxx.biogemma.fr:36535 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55044 | gnpgenome | xxxxxxx.biogemma.fr:36546 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55045 | gnpgenome | xxxxxxx.biogemma.fr:36547 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55156 | gnpgenome | xxx.xxx.xxx.xxx:57348     | Xxxxxxx            | Sleep   |  2290 |       | NULL                  |

| 55784 | gnpgenome | xxx.xxx.xxx.xxx:57813     | Xxxxxxxxxxx        | Sleep   |  2594 |       | NULL                  |

| 55809 | gnpgenome | xxx.xxx.xxx.xxx:57838     | gnpgenome_sessions | Sleep   |  2556 |       | NULL                  |

| 55812 | gnpgenome | xxx.xxx.xxx.xxx:57841     | ArabidoV9          | Sleep   |  2522 |       | NULL                  |

| 55920 | gnpgenome | xxxxxxx.biogemma.fr:49728 | ArabidoV9          | Sleep   |  2463 |       | NULL                  |

| 55962 | gnpgenome | xxxxxxx.biogemma.fr:49769 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55973 | gnpgenome | xxxxxxx.biogemma.fr:49781 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55974 | gnpgenome | xxxxxxx.biogemma.fr:49782 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55975 | gnpgenome | xxxxxxx.biogemma.fr:49783 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 56041 | gnpgenome | xxx.xxx.xxx.xxx:57970     | gnpgenome_sessions | Sleep   |  2335 |       | NULL                  |

| 56050 | gnpgenome | xxx.xxx.xxx.xxx:57987     | ArabidoV9          | Sleep   |  2399 |       | NULL                  |

| 56135 | gnpgenome | xxx.xxx.xxx.xxx:58075     | ArabidoV9          | Sleep   |  2302 |       | NULL                  |

| 56136 | gnpgenome | xxx.xxx.xxx.xxx:58076     | Xxxxxxx            | Sleep   |  2302 |       | NULL                  |

| 56171 | gnpgenome | xxx.xxx.xxx.xxx:58112     | Xxxxxxxxx          | Sleep   |  2301 |       | NULL                  |

| 56174 | gnpgenome | xxx.xxx.xxx.xxx:58115     | Xxxxxxxxxxx        | Sleep   |  2301 |       | NULL                  |

| 56175 | gnpgenome | xxx.xxx.xxx.xxx:58116     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56176 | gnpgenome | xxx.xxx.xxx.xxx:58117     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56177 | gnpgenome | xxx.xxx.xxx.xxx:58118     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56226 | gnpgenome | xxx.xxx.xxx.xxx:58131     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56227 | gnpgenome | xxx.xxx.xxx.xxx:58132     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56228 | gnpgenome | xxx.xxx.xxx.xxx:58133     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56229 | gnpgenome | xxx.xxx.xxx.xxx:58134     | Xxxxxxxxxxxxxx     | Sleep   |  2300 |       | NULL                  |

| 56230 | gnpgenome | xxx.xxx.xxx.xxx:58135     | Xxxxxxxxxxx        | Sleep   |  2300 |       | NULL                  |

| 56231 | gnpgenome | xxx.xxx.xxx.xxx:58136     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56232 | gnpgenome | xxx.xxx.xxx.xxx:58137     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56233 | gnpgenome | xxx.xxx.xxx.xxx:58138     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56236 | gnpgenome | xxx.xxx.xxx.xxx:58141     | ArabidoV9          | Sleep   |  2300 |       | NULL                  |

| 56237 | gnpgenome | xxx.xxx.xxx.xxx:58142     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56238 | gnpgenome | xxx.xxx.xxx.xxx:58143     | Xxxxxxxxx          | Sleep   |  2300 |       | NULL                  |

| 56239 | gnpgenome | xxx.xxx.xxx.xxx:58144     | Xxxxxxxxxxx        | Sleep   |  2300 |       | NULL                  |

| 56240 | gnpgenome | xxx.xxx.xxx.xxx:58145     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56241 | gnpgenome | xxx.xxx.xxx.xxx:58146     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56242 | gnpgenome | xxx.xxx.xxx.xxx:58147     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56273 | gnpgenome | xxx.xxx.xxx.xxx:58178     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56274 | gnpgenome | xxx.xxx.xxx.xxx:58179     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56275 | gnpgenome | xxx.xxx.xxx.xxx:58180     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56276 | gnpgenome | xxx.xxx.xxx.xxx:58181     | Xxxxxxxxxxxxxx     | Sleep   |  2299 |       | NULL                  |

| 56277 | gnpgenome | xxx.xxx.xxx.xxx:58182     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56278 | gnpgenome | xxx.xxx.xxx.xxx:58183     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56279 | gnpgenome | xxx.xxx.xxx.xxx:58184     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56280 | gnpgenome | xxx.xxx.xxx.xxx:58185     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56317 | gnpgenome | xxx.xxx.xxx.xxx:58210     | ArabidoV9          | Sleep   |  2299 |       | NULL                  |

| 56318 | gnpgenome | xxx.xxx.xxx.xxx:58211     | ArabidoV9          | Sleep   |  2299 |       | NULL                  |

| 56322 | gnpgenome | xxx.xxx.xxx.xxx:58212     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56323 | gnpgenome | xxx.xxx.xxx.xxx:58213     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56325 | gnpgenome | xxx.xxx.xxx.xxx:58214     | Xxxxxxxxx          | Sleep   |  2299 |       | NULL                  |

| 56326 | gnpgenome | xxx.xxx.xxx.xxx:58215     | Xxxxxxxxx          | Sleep   |  2299 |       | NULL                  |

| 56327 | gnpgenome | xxx.xxx.xxx.xxx:58216     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56328 | gnpgenome | xxx.xxx.xxx.xxx:58217     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56329 | gnpgenome | xxx.xxx.xxx.xxx:58218     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56330 | gnpgenome | xxx.xxx.xxx.xxx:58219     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56331 | gnpgenome | xxx.xxx.xxx.xxx:58220     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56332 | gnpgenome | xxx.xxx.xxx.xxx:58221     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56333 | gnpgenome | xxx.xxx.xxx.xxx:58222     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56334 | gnpgenome | xxx.xxx.xxx.xxx:58223     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56340 | gnpgenome | xxx.xxx.xxx.xxx:58229     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56341 | gnpgenome | xxx.xxx.xxx.xxx:58230     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56342 | gnpgenome | xxx.xxx.xxx.xxx:58231     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56343 | gnpgenome | xxx.xxx.xxx.xxx:58232     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56345 | gnpgenome | xxx.xxx.xxx.xxx:58234     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56346 | gnpgenome | xxx.xxx.xxx.xxx:58235     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56347 | gnpgenome | xxx.xxx.xxx.xxx:58236     | Xxxxxxxxxxxxxx     | Sleep   |  2298 |       | NULL                  |

| 56348 | gnpgenome | xxx.xxx.xxx.xxx:58237     | Xxxxxxxxxxxxxx     | Sleep   |  2298 |       | NULL                  |

| 56349 | gnpgenome | xxx.xxx.xxx.xxx:58238     | Xxxxxxxxxxx        | Sleep   |  2298 |       | NULL                  |

| 56350 | gnpgenome | xxx.xxx.xxx.xxx:58239     | Xxxxxxxxxxx        | Sleep   |  2298 |       | NULL                  |

| 56351 | gnpgenome | xxx.xxx.xxx.xxx:58240     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56352 | gnpgenome | xxx.xxx.xxx.xxx:58241     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56353 | gnpgenome | xxx.xxx.xxx.xxx:58242     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56354 | gnpgenome | xxx.xxx.xxx.xxx:58243     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56355 | gnpgenome | xxx.xxx.xxx.xxx:58244     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56357 | gnpgenome | xxx.xxx.xxx.xxx:58246     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56411 | gnpgenome | xxx.xxx.xxx.xxx:58276     | Xxxxxxx            | Sleep   |  2278 |       | NULL                  |

| 56413 | gnpgenome | xxx.xxx.xxx.xxx:58279     | gnpgenome_sessions | Sleep   |  2020 |       | NULL                  |

| 56423 | gnpgenome | xxx.xxx.xxx.xxx:58289     | Xxxxxxx            | Sleep   |  2218 |       | NULL                  |

| 56440 | gnpgenome | xxx.xxx.xxx.xxx:58305     | ArabidoV9          | Sleep   |  2292 |       | NULL                  |

| 56442 | gnpgenome | xxx.xxx.xxx.xxx:58307     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56443 | gnpgenome | xxx.xxx.xxx.xxx:58308     | Xxxxxxxxx          | Sleep   |  2292 |       | NULL                  |

| 56448 | gnpgenome | xxx.xxx.xxx.xxx:58309     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56449 | gnpgenome | xxx.xxx.xxx.xxx:58310     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56450 | gnpgenome | xxx.xxx.xxx.xxx:58311     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56452 | gnpgenome | xxx.xxx.xxx.xxx:58313     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56477 | gnpgenome | xxx.xxx.xxx.xxx:58319     | ArabidoV9          | Sleep   |  2292 |       | NULL                  |

| 56478 | gnpgenome | xxx.xxx.xxx.xxx:58320     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56479 | gnpgenome | xxx.xxx.xxx.xxx:58321     | Xxxxxxxxx          | Sleep   |  2292 |       | NULL                  |

| 56480 | gnpgenome | xxx.xxx.xxx.xxx:58322     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56481 | gnpgenome | xxx.xxx.xxx.xxx:58323     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56482 | gnpgenome | xxx.xxx.xxx.xxx:58324     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56483 | gnpgenome | xxx.xxx.xxx.xxx:58325     | Xxxxxxxxxxxxxx     | Sleep   |  2292 |       | NULL                  |

| 56485 | gnpgenome | xxx.xxx.xxx.xxx:58327     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56486 | gnpgenome | xxx.xxx.xxx.xxx:58328     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56487 | gnpgenome | xxx.xxx.xxx.xxx:58329     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56488 | gnpgenome | xxx.xxx.xxx.xxx:58330     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56489 | gnpgenome | xxx.xxx.xxx.xxx:58331     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56490 | gnpgenome | xxx.xxx.xxx.xxx:58332     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56491 | gnpgenome | xxx.xxx.xxx.xxx:58333     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56492 | gnpgenome | xxx.xxx.xxx.xxx:58334     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56521 | gnpgenome | xxx.xxx.xxx.xxx:58363     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56531 | gnpgenome | xxx.xxx.xxx.xxx:58374     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56532 | gnpgenome | xxx.xxx.xxx.xxx:58375     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56534 | gnpgenome | xxx.xxx.xxx.xxx:58377     | Xxxxxxxxxxxxxx     | Sleep   |  2291 |       | NULL                  |

| 56536 | gnpgenome | xxx.xxx.xxx.xxx:58379     | Xxxxxxxxxxx        | Sleep   |  2291 |       | NULL                  |

| 56537 | gnpgenome | xxx.xxx.xxx.xxx:58380     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56539 | gnpgenome | xxx.xxx.xxx.xxx:58381     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56542 | gnpgenome | xxx.xxx.xxx.xxx:58382     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56600 | gnpgenome | xxx.xxx.xxx.xxx:58421     | ArabidoV9          | Sleep   |  2290 |       | NULL                  |

| 56631 | gnpgenome | xxx.xxx.xxx.xxx:58438     | Xxxxxxx            | Sleep   |  2289 |       | NULL                  |

| 56658 | gnpgenome | xxx.xxx.xxx.xxx:58447     | Xxxxxxxxx          | Sleep   |  2285 |       | NULL                  |

| 56671 | gnpgenome | xxx.xxx.xxx.xxx:58451     | Xxxxxxxxxxx        | Sleep   |  2283 |       | NULL                  |

| 56683 | gnpgenome | xxx.xxx.xxx.xxx:58453     | gnpgenome_sessions | Sleep   |  2282 |       | NULL                  |

| 56685 | gnpgenome | xxx.xxx.xxx.xxx:58455     | gnpgenome_sessions | Sleep   |  2281 |       | NULL                  |

| 56690 | gnpgenome | xxx.xxx.xxx.xxx:58460     | gnpgenome_sessions | Sleep   |  2279 |       | NULL                  |

| 56733 | gnpgenome | xxx.xxx.xxx.xxx:58494     | gnpgenome_sessions | Sleep   |  2262 |       | NULL                  |

| 56738 | gnpgenome | xxx.xxx.xxx.xxx:58499     | gnpgenome_sessions | Sleep   |  2262 |       | NULL                  |

| 56750 | gnpgenome | xxx.xxx.xxx.xxx:58511     | gnpgenome_sessions | Sleep   |  2263 |       | NULL                  |

| 56753 | gnpgenome | xxx.xxx.xxx.xxx:58516     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 56761 | gnpgenome | xxx.xxx.xxx.xxx:58518     | gnpgenome_sessions | Sleep   |  2254 |       | NULL                  |

| 56769 | gnpgenome | xxx.xxx.xxx.xxx:58520     | Xxxxxxxxx          | Sleep   |  2262 |       | NULL                  |

| 56770 | gnpgenome | xxx.xxx.xxx.xxx:58521     | Xxxxxxxxxxxxxx     | Sleep   |  2262 |       | NULL                  |

| 56771 | gnpgenome | xxx.xxx.xxx.xxx:58522     | Xxxxxxxxxxx        | Sleep   |  2262 |       | NULL                  |

| 56778 | gnpgenome | xxx.xxx.xxx.xxx:58523     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 56779 | gnpgenome | xxx.xxx.xxx.xxx:58524     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56780 | gnpgenome | xxx.xxx.xxx.xxx:58525     | ArabidoV9          | Sleep   |  2261 |       | NULL                  |

| 56781 | gnpgenome | xxx.xxx.xxx.xxx:58526     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56782 | gnpgenome | xxx.xxx.xxx.xxx:58527     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56783 | gnpgenome | xxx.xxx.xxx.xxx:58528     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56790 | gnpgenome | xxx.xxx.xxx.xxx:58529     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56791 | gnpgenome | xxx.xxx.xxx.xxx:58530     | Xxxxxxxxx          | Sleep   |  2261 |       | NULL                  |

| 56792 | gnpgenome | xxx.xxx.xxx.xxx:58531     | Xxxxxxxxxxxxxx     | Sleep   |  2261 |       | NULL                  |

| 56793 | gnpgenome | xxx.xxx.xxx.xxx:58532     | Xxxxxxxxxxx        | Sleep   |  2261 |       | NULL                  |

| 56795 | gnpgenome | xxx.xxx.xxx.xxx:58534     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56796 | gnpgenome | xxx.xxx.xxx.xxx:58535     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56797 | gnpgenome | xxx.xxx.xxx.xxx:58536     | ArabidoV9          | Sleep   |  2261 |       | NULL                  |

| 56798 | gnpgenome | xxx.xxx.xxx.xxx:58537     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56799 | gnpgenome | xxx.xxx.xxx.xxx:58538     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56800 | gnpgenome | xxx.xxx.xxx.xxx:58539     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56827 | gnpgenome | xxx.xxx.xxx.xxx:58560     | Xxxxxxx            | Sleep   |  2241 |       | NULL                  |

| 56829 | gnpgenome | xxx.xxx.xxx.xxx:58563     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57019 | gnpgenome | xxx.xxx.xxx.xxx:58642     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57021 | gnpgenome | xxx.xxx.xxx.xxx:58644     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57043 | gnpgenome | xxx.xxx.xxx.xxx:58672     | gnpgenome_sessions | Sleep   |  2020 |       | NULL                  |

| 57121 | gnpgenome | xxx.xxx.xxx.xxx:58676     | Xxxxxxx            | Sleep   |   542 |       | NULL                  |

| 57144 | gnpgenome | xxxxxxx.biogemma.fr:43501 | NULL               | Query   |     0 | NULL  | show full processlist |

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+
177 rows in set (0.00 sec)


Fred

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Jim Hu
In reply to this post by Lincoln Stein
I'll see if I can reproduce the problems, but I'm going to upgrade first.

Jim
On Jul 3, 2013, at 2:58 PM, Lincoln Stein wrote:

Hi Jim,

I've been playing with this using the current GBrowse version 2.54 (from git) and Bio::Graphics 2.37. I can't document a process leak with either normal cgi-bin and FastCGI. When running under fastcgi there is one sleeping mysql process per database per FastCGI instance; this is expected, because the FastCGI instances cache database handles. Under normal CGI, I see the database handles being created as needed to display the tracks and then destroyed once the request is finished. There are no sleeping mysql processes after all tracks are rendered.

I can confirm that CGI is a lot slower than FastCGI. This is because the entire code and database has to be loaded for each displayed track. 

I tried to reproduce the accumulation of FastCGI processes that you reported being spawned by the login process, but couldn't do reproduce it. If you can help me reproduce the problems you experienced with FastCGI, I'd love to try to help you out.

Lincoln


On Mon, Jun 24, 2013 at 6:06 PM, Jim Hu <[hidden email]> wrote:
This may have been fixed already: the login system was spawning a huge number of mysql threads when fastcgi was enabled.... to the point where gbrowse was crashing.  For a while we were running a cron job that killed mysql processes that were where the time was > some value.  Not pretty.

We spent a little time trying to figure it out, but Nathan couldn't get the fastcgi to work on our VMs.

I agree that it's odd to see the sleeping processes.  I think some of them are being called by gbrowse_img. Could there be a difference there somewhere? 

Using 2.52 currently.

Jim

On Jun 24, 2013, at 10:07 AM, Lincoln Stein wrote:

What problems did you see with the user login system? I am working on that subsystem now.

Odd that you would see any sleeping processes when running under vanilla CGI. The gbrowse process goes away after finishing the transaction.

Lincoln

On Monday, June 24, 2013, Jim Hu wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054





--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Jim Hu
In reply to this post by Lincoln Stein
Hi Lincoln,

Here's what I've been doing today:

updated gbrowse and Bio::Graphics via CPAN. About shows v 2.54. Kept all my old conf files and plugins. 

perl -MBio::Graphics -e 'print "$Bio::Graphics::VERSION\n"'
2.37

Updated lib_fastcgid and turned it on in /etc/apache2/conf.d/gbrowse.conf with

<IfModule mod_fcgid.c>
  Alias /fgb2 "/var/www/cgi-bin/gb2"
  <Location /fgb2>
    SetHandler   fcgid-script
  </Location>
  FcgidInitialEnv GBROWSE_CONF /etc/gbrowse2
  # these directives prevent idle/busy timeouts and may need to be                                                                                                                                      
  # adjusted up or down                                                                                                                                                                                 
  FcgidMinProcessesPerClass 6
  FcgidIOTimeout   600
  FcgidBusyTimeout 600
</IfModule>

I do not have this set to redirect via mod rewrite, so I can access gbrowse either from 


or 


ps ax|grep -c apache
gives between 14-18 threads pretty consistently

mysql show processlist is giving

mysql> show processlist;
+------+--------------+-----------+---------------+---------+------+-------+------------------+
| Id   | User         | Host      | db            | Command | Time | State | Info             |
+------+--------------+-----------+---------------+---------+------+-------+------------------+
|  276 | jimhu        | localhost | NULL          | Sleep   | 7507 |       | NULL             |
|  298 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   17 |       | NULL             |
|  299 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   17 |       | NULL             |
|  300 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   17 |       | NULL             |
|  302 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  103 |       | NULL             |
|  306 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  103 |       | NULL             |
|  307 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   19 |       | NULL             |
|  308 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  103 |       | NULL             |
|  309 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   19 |       | NULL             |
|  311 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   19 |       | NULL             |
| 5899 | jimhu        | localhost | NULL          | Query   |    0 | NULL  | show processlist |
| 6890 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  465 |       | NULL             |
| 7276 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  640 |       | NULL             |
| 7288 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  767 |       | NULL             |
| 7322 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  472 |       | NULL             |
| 7356 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  767 |       | NULL             |
| 7415 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  640 |       | NULL             |
| 7544 | gbrowse_user | localhost | gbrowse_login | Sleep   |  679 |       | NULL             |
| 7798 | gbrowse_user | localhost | gbrowse_login | Sleep   |  640 |       | NULL             |
| 7799 | gbrowse_user | localhost | gbrowse_login | Sleep   |  640 |       | NULL             |
| 7800 | gbrowse_user | localhost | gbrowse_login | Sleep   |  640 |       | NULL             |
| 7974 | gbrowse_user | localhost | gbrowse_login | Sleep   |  602 |       | NULL             |
| 7980 | gbrowse_user | localhost | gbrowse_login | Sleep   |  602 |       | NULL             |
| 8014 | gbrowse_user | localhost | gbrowse_login | Sleep   |  602 |       | NULL             |
| 8148 | gbrowse_user | localhost | gbrowse_login | Sleep   |  559 |       | NULL             |
| 8149 | gbrowse_user | localhost | gbrowse_login | Sleep   |  559 |       | NULL             |
| 8150 | gbrowse_user | localhost | gbrowse_login | Sleep   |  559 |       | NULL             |
| 8284 | gbrowse_user | localhost | gbrowse_login | Sleep   |  466 |       | NULL             |
| 8285 | gbrowse_user | localhost | gbrowse_login | Sleep   |  466 |       | NULL             |
| 8286 | gbrowse_user | localhost | gbrowse_login | Sleep   |  466 |       | NULL             |
| 8302 | gbrowse_user | localhost | gbrowse_login | Sleep   |  465 |       | NULL             |
| 8303 | gbrowse_user | localhost | gbrowse_login | Sleep   |  465 |       | NULL             |
| 8304 | gbrowse_user | localhost | gbrowse_login | Sleep   |  465 |       | NULL             |
+------+--------------+-----------+---------------+---------+------+-------+------------------+
33 rows in set (0.00 sec)

after I logged in on the fgb2 version. I restarted mysql and got 

+-----+--------------+-----------+-----------+---------+------+-------+------------------+
| Id  | User         | Host      | db        | Command | Time | State | Info             |
+-----+--------------+-----------+-----------+---------+------+-------+------------------+
| 283 | gbrowse_user | localhost | gb_MG1655 | Sleep   |    9 |       | NULL             |
| 284 | gbrowse_user | localhost | gb_MG1655 | Sleep   |    9 |       | NULL             |
| 285 | gbrowse_user | localhost | gb_MG1655 | Sleep   |    9 |       | NULL             |
| 298 | jimhu        | localhost | NULL      | Query   |    0 | NULL  | show processlist |
+-----+--------------+-----------+-----------+---------+------+-------+------------------+
4 rows in set (0.00 sec)

I did a rightward move on the gb2 version (not logged in) and got

mysql> show processlist;
+-----+--------------+-----------+---------------+---------+------+-------+------------------+
| Id  | User         | Host      | db            | Command | Time | State | Info             |
+-----+--------------+-----------+---------------+---------+------+-------+------------------+
| 283 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   15 |       | NULL             |
| 284 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   15 |       | NULL             |
| 285 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   15 |       | NULL             |
| 298 | jimhu        | localhost | NULL          | Query   |    0 | NULL  | show processlist |
| 306 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   73 |       | NULL             |
| 307 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   73 |       | NULL             |
| 308 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   73 |       | NULL             |
| 321 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   59 |       | NULL             |
| 322 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   59 |       | NULL             |
| 323 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   59 |       | NULL             |
| 371 | gbrowse_user | localhost | gbrowse_login | Sleep   |    8 |       | NULL             |
| 372 | gbrowse_user | localhost | gbrowse_login | Sleep   |    8 |       | NULL             |
| 373 | gbrowse_user | localhost | gbrowse_login | Sleep   |    8 |       | NULL             |
| 374 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    7 |       | NULL             |
+-----+--------------+-----------+---------------+---------+------+-------+------------------+
14 rows in set (0.00 sec)

but a little later it was back to

+-----+--------------+-----------+-----------+---------+------+-------+------------------+
| Id  | User         | Host      | db        | Command | Time | State | Info             |
+-----+--------------+-----------+-----------+---------+------+-------+------------------+
| 283 | gbrowse_user | localhost | gb_MG1655 | Sleep   |   83 |       | NULL             |
| 284 | gbrowse_user | localhost | gb_MG1655 | Sleep   |   83 |       | NULL             |
| 285 | gbrowse_user | localhost | gb_MG1655 | Sleep   |   83 |       | NULL             |
| 298 | jimhu        | localhost | NULL      | Query   |    0 | NULL  | show processlist |
| 306 | gbrowse_user | localhost | gb_MG1655 | Sleep   |   59 |       | NULL             |
| 307 | gbrowse_user | localhost | gb_MG1655 | Sleep   |   59 |       | NULL             |
| 308 | gbrowse_user | localhost | gb_MG1655 | Sleep   |   59 |       | NULL             |
| 321 | gbrowse_user | localhost | gb_MG1655 | Sleep   |   55 |       | NULL             |
| 322 | gbrowse_user | localhost | gb_MG1655 | Sleep   |   55 |       | NULL             |
| 323 | gbrowse_user | localhost | gb_MG1655 | Sleep   |   55 |       | NULL             |
+-----+--------------+-----------+-----------+---------+------+-------+------------------+
10 rows in set (0.00 sec)

My browser with fgb2/logged in has about a dozen tracks turned on, but only a few are using mysql. The others are bam or bigwig.  I resized the view from 20kb to 5kb and the mysql processes blow up 

mysql> show processlist;
+-----+--------------+-----------+---------------+---------+------+-------+------------------+
| Id  | User         | Host      | db            | Command | Time | State | Info             |
+-----+--------------+-----------+---------------+---------+------+-------+------------------+
| 283 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  167 |       | NULL             |
| 284 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  167 |       | NULL             |
| 285 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  167 |       | NULL             |
| 298 | jimhu        | localhost | NULL          | Query   |    0 | NULL  | show processlist |
| 306 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  143 |       | NULL             |
| 307 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  143 |       | NULL             |
| 308 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  143 |       | NULL             |
| 321 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  139 |       | NULL             |
| 322 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  139 |       | NULL             |
| 323 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  139 |       | NULL             |
| 529 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    7 |       | NULL             |
| 534 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    5 |       | NULL             |
| 539 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 540 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 541 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 542 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 543 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 544 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 545 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 546 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 547 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 548 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 549 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 550 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 551 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 552 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 553 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 554 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 555 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 556 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 557 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 558 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 559 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 560 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 561 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 562 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 563 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 564 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 565 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 566 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 567 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 568 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 569 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 570 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 571 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 572 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 573 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 574 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 575 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 576 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 577 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 578 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 579 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 580 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 581 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 582 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 583 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 584 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 585 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    6 |       | NULL             |
| 586 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 587 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 588 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 589 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 590 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 592 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 594 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 595 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 596 | gbrowse_user | localhost | gbrowse_login | Sleep   |    6 |       | NULL             |
| 601 | gbrowse_user | localhost | gbrowse_login | Sleep   |    5 |       | NULL             |
| 602 | gbrowse_user | localhost | gbrowse_login | Sleep   |    5 |       | NULL             |
| 603 | gbrowse_user | localhost | gbrowse_login | Sleep   |    5 |       | NULL             |
+-----+--------------+-----------+---------------+---------+------+-------+------------------+
71 rows in set (0.00 sec)

and then they subside

mysql> show processlist;
+-----+--------------+-----------+---------------+---------+------+-------+------------------+
| Id  | User         | Host      | db            | Command | Time | State | Info             |
+-----+--------------+-----------+---------------+---------+------+-------+------------------+
| 283 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    7 |       | NULL             |
| 284 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    7 |       | NULL             |
| 285 | gbrowse_user | localhost | gb_MG1655     | Sleep   |    7 |       | NULL             |
| 298 | jimhu        | localhost | NULL          | Query   |    0 | NULL  | show processlist |
| 306 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  229 |       | NULL             |
| 307 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  229 |       | NULL             |
| 308 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  229 |       | NULL             |
| 321 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  225 |       | NULL             |
| 322 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  225 |       | NULL             |
| 323 | gbrowse_user | localhost | gb_MG1655     | Sleep   |  225 |       | NULL             |
| 529 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   93 |       | NULL             |
| 534 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   91 |       | NULL             |
| 634 | gbrowse_user | localhost | gbrowse_login | Sleep   |   68 |       | NULL             |
| 635 | gbrowse_user | localhost | gbrowse_login | Sleep   |   68 |       | NULL             |
| 636 | gbrowse_user | localhost | gbrowse_login | Sleep   |   69 |       | NULL             |
| 637 | gbrowse_user | localhost | gbrowse_login | Sleep   |   63 |       | NULL             |
| 638 | gbrowse_user | localhost | gbrowse_login | Sleep   |   64 |       | NULL             |
| 639 | gbrowse_user | localhost | gbrowse_login | Sleep   |   64 |       | NULL             |
| 640 | gbrowse_user | localhost | gb_MG1655     | Sleep   |   63 |       | NULL             |
+-----+--------------+-----------+---------------+---------+------+-------+------------------+
19 rows in set (0.00 sec)

This expansion/contraction of mysql processes happens without any increase that I can catch in the number of apache2 threads.

In the apache error log, I'm seeing lots of this:

[Fri Jul  5 13:44:47 2013] gbrowse: NC_000913 coverage at /usr/local/share/perl/5.14.2/Bio/Graphics/Glyph/wiggle_data.pm line 158.
[Fri Jul  5 13:44:47 2013] gbrowse: NC_000913 coverage at /usr/local/share/perl/5.14.2/Bio/Graphics/Glyph/wiggle_data.pm line 158.
[Fri Jul  5 13:44:47 2013] gbrowse: NC_000913 coverage at /usr/local/share/perl/5.14.2/Bio/Graphics/Glyph/wiggle_data.pm line 158.
[Fri Jul  5 13:44:47 2013] gbrowse: NC_000913 coverage at /usr/local/share/perl/5.14.2/Bio/Graphics/Glyph/wiggle_data.pm line 158.
[Fri Jul  5 13:44:47 2013] gbrowse: Warning: something's wrong at /usr/local/share/perl/5.14.2/Bio/Graphics/Glyph/wiggle_data.pm line 158.
...

This doesn't happen when I log in and move around using gb2 instead of fgb2.

I also turned on slow query logging for mysql. The log is working, but the only things I see there are debian maintenance messages.

I think there are a couple of things going on. 

Some of the persistent connections to gb_MG1655 may be coming from gbrowse_img instead of gbrowse, since the apache access logs show many more calls to gbrowse_img than gbrowse due to the way we generate thumbnails for porteco.org and ecoliwiki.  Yesterday I modified our mediawiki extensions to do more image caching, but porteco.org is still calling gbrowse_img on our server directly.  It's interesting that since I did that, I'm not seeing sleeping connections to the other mysql Bio::DB::SeqFeature::Store databases (compare to the snippet from the June 24 email).  I think our setup may be unusual in terms of the number of genomes (43 in mysql) we have running under the same gbrowse installation, but most of them aren't getting much traffic.

The gbrowse_login connections causing the mysql processes to hang around seems to depend on fcgid.  I haven't tried the other fastcgi module yet. Should I?

Not sure how helpful this is, but I hope it gives some clues.  

Jim

On Jul 3, 2013, at 2:58 PM, Lincoln Stein wrote:

Hi Jim,

I've been playing with this using the current GBrowse version 2.54 (from git) and Bio::Graphics 2.37. I can't document a process leak with either normal cgi-bin and FastCGI. When running under fastcgi there is one sleeping mysql process per database per FastCGI instance; this is expected, because the FastCGI instances cache database handles. Under normal CGI, I see the database handles being created as needed to display the tracks and then destroyed once the request is finished. There are no sleeping mysql processes after all tracks are rendered.

I can confirm that CGI is a lot slower than FastCGI. This is because the entire code and database has to be loaded for each displayed track. 

I tried to reproduce the accumulation of FastCGI processes that you reported being spawned by the login process, but couldn't do reproduce it. If you can help me reproduce the problems you experienced with FastCGI, I'd love to try to help you out.

Lincoln


On Mon, Jun 24, 2013 at 6:06 PM, Jim Hu <[hidden email]> wrote:
This may have been fixed already: the login system was spawning a huge number of mysql threads when fastcgi was enabled.... to the point where gbrowse was crashing.  For a while we were running a cron job that killed mysql processes that were where the time was > some value.  Not pretty.

We spent a little time trying to figure it out, but Nathan couldn't get the fastcgi to work on our VMs.

I agree that it's odd to see the sleeping processes.  I think some of them are being called by gbrowse_img. Could there be a difference there somewhere? 

Using 2.52 currently.

Jim

On Jun 24, 2013, at 10:07 AM, Lincoln Stein wrote:

What problems did you see with the user login system? I am working on that subsystem now.

Odd that you would see any sleeping processes when running under vanilla CGI. The gbrowse process goes away after finishing the transaction.

Lincoln

On Monday, June 24, 2013, Jim Hu wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054





--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Jim Hu
In reply to this post by Jim Hu
Tried mod_fastcgi instead of mod_fcgid after reading this:

http://www.apachelounge.com/viewtopic.php?t=4385

installed and activated via apt-get and a2enmod.  Edited conf.d/gbrowse.conf to uncomment. Commented out the fcgid stanza. 

<IfModule mod_fastcgi.c>
  Alias /fgb2 "/var/www/cgi-bin/gb2"
  <Location /fgb2>
    SetHandler   fastcgi-script
  </Location>
  # Note: you may need to increase -idle-timeout if file uploads are timing out and returning server                                                                                                    
  # errors.                                                                                                                                                                                             
  FastCgiConfig -idle-timeout 600 -maxClassProcesses 20 -initial-env GBROWSE_CONF=/etc/gbrowse2
</IfModule>

Realized that apache was also running mod_perl and fcgid too, so I inactivated those.

Overall, it was a disaster. Sample from the error log: 
[Fri Jul  5 15:18:38 2013] [notice] GBrowse FastCGI process caught sigTERM. Exiting... (pid 17225)
[Fri Jul  5 15:18:38 2013] [notice] GBrowse FastCGI process caught sigTERM. Exiting... (pid 17225)
[Fri Jul 05 15:25:00 2013] [error] [client 157.56.92.158] File does not exist: /var/www/htdocs/textpresso/robots.txt
[Fri Jul 05 15:27:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:27:02 2013] gbrowse: Warning: something's wrong at /usr/local/share/perl/5.14.2/Bio/Graphics/Glyph/wiggle_data.pm line 158., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:27:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:27:02 2013] gbrowse: Use of uninitialized value $_ in lc at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Render.pm line 3736., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:28:02 2013] gbrowse: alarm clock at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/RenderPanels.pm line 1518., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:28:02 2013] gbrowse: RenderPanels error: timeout at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/RenderPanels.pm line 1518., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:14 2013] [error] [client 75.102.127.32] [18082] session lock timed out on request after 10.000s, referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:14 2013] [error] [client 75.102.127.32] POST: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/ action=update_sections;section_names=page_title;section_names=span;section_names=galaxy_form;section_names=search_form_objects at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Session.pm line 153., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:17 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:28:17 2013] gbrowse: [18049] rendering error track: Timeout; Try turning off tracks or looking at a smaller region. at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Render.pm line 3678., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:38:26 2013] [alert] [client 165.91.51.61] (2)No such file or directory: FastCGI: failed to connect to (dynamic) server "/var/www/cgi-bin/gb2/gbrowse": something is seriously wrong, any chance the socket/named_pipe directory was removed?, see the FastCgiIpcDir directive, referer: http://heptamer.tamu.edu/fgb2/gbrowse/MG1655/
[Fri Jul 05 15:38:26 2013] [error] [client 165.91.51.61] FastCGI: incomplete headers (0 bytes) received from server "/var/www/cgi-bin/gb2/gbrowse", referer: http://heptamer.tamu.edu/fgb2/gbrowse/MG1655/
[Fri Jul 05 15:38:42 2013] [error] [client 165.91.51.61] UNAUTHORIZED ATTEMPT at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2.pm line 514., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/
[Fri Jul 05 15:38:42 2013] [error] [client 165.91.51.61] \tBio::Graphics::Browser2::authorized_session('Bio::Graphics::Browser2=HASH(0x2252130)', 'e78b8e09f2ea73e4aaa43f7e8fc3d98d', 'ea154c2dac7ce9bc37a469fe4a0fe4ce', undef) called at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Render.pm line 70, referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/
[Fri Jul 05 15:38:42 2013] [error] [client 165.91.51.61] \tBio::Graphics::Browser2::Render::new('Bio::Graphics::Browser2::Render::HTML', 'Bio::Graphics::Browser2=HASH(0x2252130)') called at /var/www/cgi-bin/gb2/gbrowse line 50, referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/
[Fri Jul 05 15:39:48 2013] [alert] [client 165.91.51.61] (2)No such file or directory: FastCGI: failed to connect to (dynamic) server "/var/www/cgi-bin/gb2/gbrowse": something is seriously wrong, any chance the socket/named_pipe directory was removed?, see the FastCgiIpcDir directive

Also, when it didn't crash it wasn't improving performance. But the good news is it's didn't create extra sleeping mysql processes! 

Jim


On Jul 5, 2013, at 11:07 AM, Jim Hu wrote:

I'll see if I can reproduce the problems, but I'm going to upgrade first.

Jim
On Jul 3, 2013, at 2:58 PM, Lincoln Stein wrote:

Hi Jim,

I've been playing with this using the current GBrowse version 2.54 (from git) and Bio::Graphics 2.37. I can't document a process leak with either normal cgi-bin and FastCGI. When running under fastcgi there is one sleeping mysql process per database per FastCGI instance; this is expected, because the FastCGI instances cache database handles. Under normal CGI, I see the database handles being created as needed to display the tracks and then destroyed once the request is finished. There are no sleeping mysql processes after all tracks are rendered.

I can confirm that CGI is a lot slower than FastCGI. This is because the entire code and database has to be loaded for each displayed track. 

I tried to reproduce the accumulation of FastCGI processes that you reported being spawned by the login process, but couldn't do reproduce it. If you can help me reproduce the problems you experienced with FastCGI, I'd love to try to help you out.

Lincoln


On Mon, Jun 24, 2013 at 6:06 PM, Jim Hu <[hidden email]> wrote:
This may have been fixed already: the login system was spawning a huge number of mysql threads when fastcgi was enabled.... to the point where gbrowse was crashing.  For a while we were running a cron job that killed mysql processes that were where the time was > some value.  Not pretty.

We spent a little time trying to figure it out, but Nathan couldn't get the fastcgi to work on our VMs.

I agree that it's odd to see the sleeping processes.  I think some of them are being called by gbrowse_img. Could there be a difference there somewhere? 

Using 2.52 currently.

Jim

On Jun 24, 2013, at 10:07 AM, Lincoln Stein wrote:

What problems did you see with the user login system? I am working on that subsystem now.

Odd that you would see any sleeping processes when running under vanilla CGI. The gbrowse process goes away after finishing the transaction.

Lincoln

On Monday, June 24, 2013, Jim Hu wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054





--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
979-862-4054


------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Lincoln Stein
Agreed. I just tested mod_fastcgi and gbrowse is very broken against it. It spawns a new process for each request. I will see what can be fixed. However mod_fcgid is working very well in my hands.

Are you using the "worker" style Apache2 or the prefork version? The latter is faster for gbrowse.

Lincoln


On Fri, Jul 5, 2013 at 4:52 PM, Jim Hu <[hidden email]> wrote:
Tried mod_fastcgi instead of mod_fcgid after reading this:

http://www.apachelounge.com/viewtopic.php?t=4385

installed and activated via apt-get and a2enmod.  Edited conf.d/gbrowse.conf to uncomment. Commented out the fcgid stanza. 

<IfModule mod_fastcgi.c>
  Alias /fgb2 "/var/www/cgi-bin/gb2"
  <Location /fgb2>
    SetHandler   fastcgi-script
  </Location>
  # Note: you may need to increase -idle-timeout if file uploads are timing out and returning server                                                                                                    
  # errors.                                                                                                                                                                                             
  FastCgiConfig -idle-timeout 600 -maxClassProcesses 20 -initial-env GBROWSE_CONF=/etc/gbrowse2
</IfModule>

Realized that apache was also running mod_perl and fcgid too, so I inactivated those.

Overall, it was a disaster. Sample from the error log: 
[Fri Jul  5 15:18:38 2013] [notice] GBrowse FastCGI process caught sigTERM. Exiting... (pid 17225)
[Fri Jul  5 15:18:38 2013] [notice] GBrowse FastCGI process caught sigTERM. Exiting... (pid 17225)
[Fri Jul 05 15:25:00 2013] [error] [client 157.56.92.158] File does not exist: /var/www/htdocs/textpresso/robots.txt
[Fri Jul 05 15:27:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:27:02 2013] gbrowse: Warning: something's wrong at /usr/local/share/perl/5.14.2/Bio/Graphics/Glyph/wiggle_data.pm line 158., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:27:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:27:02 2013] gbrowse: Use of uninitialized value $_ in lc at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Render.pm line 3736., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:28:02 2013] gbrowse: alarm clock at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/RenderPanels.pm line 1518., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:28:02 2013] gbrowse: RenderPanels error: timeout at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/RenderPanels.pm line 1518., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:14 2013] [error] [client 75.102.127.32] [18082] session lock timed out on request after 10.000s, referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:14 2013] [error] [client 75.102.127.32] POST: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/ action=update_sections;section_names=page_title;section_names=span;section_names=galaxy_form;section_names=search_form_objects at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Session.pm line 153., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:17 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:28:17 2013] gbrowse: [18049] rendering error track: Timeout; Try turning off tracks or looking at a smaller region. at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Render.pm line 3678., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:38:26 2013] [alert] [client 165.91.51.61] (2)No such file or directory: FastCGI: failed to connect to (dynamic) server "/var/www/cgi-bin/gb2/gbrowse": something is seriously wrong, any chance the socket/named_pipe directory was removed?, see the FastCgiIpcDir directive, referer: http://heptamer.tamu.edu/fgb2/gbrowse/MG1655/
[Fri Jul 05 15:38:26 2013] [error] [client 165.91.51.61] FastCGI: incomplete headers (0 bytes) received from server "/var/www/cgi-bin/gb2/gbrowse", referer: http://heptamer.tamu.edu/fgb2/gbrowse/MG1655/
[Fri Jul 05 15:38:42 2013] [error] [client 165.91.51.61] UNAUTHORIZED ATTEMPT at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2.pm line 514., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/
[Fri Jul 05 15:38:42 2013] [error] [client 165.91.51.61] \tBio::Graphics::Browser2::authorized_session('Bio::Graphics::Browser2=HASH(0x2252130)', 'e78b8e09f2ea73e4aaa43f7e8fc3d98d', 'ea154c2dac7ce9bc37a469fe4a0fe4ce', undef) called at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Render.pm line 70, referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/
[Fri Jul 05 15:38:42 2013] [error] [client 165.91.51.61] \tBio::Graphics::Browser2::Render::new('Bio::Graphics::Browser2::Render::HTML', 'Bio::Graphics::Browser2=HASH(0x2252130)') called at /var/www/cgi-bin/gb2/gbrowse line 50, referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/
[Fri Jul 05 15:39:48 2013] [alert] [client 165.91.51.61] (2)No such file or directory: FastCGI: failed to connect to (dynamic) server "/var/www/cgi-bin/gb2/gbrowse": something is seriously wrong, any chance the socket/named_pipe directory was removed?, see the FastCgiIpcDir directive

Also, when it didn't crash it wasn't improving performance. But the good news is it's didn't create extra sleeping mysql processes! 

Jim


On Jul 5, 2013, at 11:07 AM, Jim Hu wrote:

I'll see if I can reproduce the problems, but I'm going to upgrade first.

Jim
On Jul 3, 2013, at 2:58 PM, Lincoln Stein wrote:

Hi Jim,

I've been playing with this using the current GBrowse version 2.54 (from git) and Bio::Graphics 2.37. I can't document a process leak with either normal cgi-bin and FastCGI. When running under fastcgi there is one sleeping mysql process per database per FastCGI instance; this is expected, because the FastCGI instances cache database handles. Under normal CGI, I see the database handles being created as needed to display the tracks and then destroyed once the request is finished. There are no sleeping mysql processes after all tracks are rendered.

I can confirm that CGI is a lot slower than FastCGI. This is because the entire code and database has to be loaded for each displayed track. 

I tried to reproduce the accumulation of FastCGI processes that you reported being spawned by the login process, but couldn't do reproduce it. If you can help me reproduce the problems you experienced with FastCGI, I'd love to try to help you out.

Lincoln


On Mon, Jun 24, 2013 at 6:06 PM, Jim Hu <[hidden email]> wrote:
This may have been fixed already: the login system was spawning a huge number of mysql threads when fastcgi was enabled.... to the point where gbrowse was crashing.  For a while we were running a cron job that killed mysql processes that were where the time was > some value.  Not pretty.

We spent a little time trying to figure it out, but Nathan couldn't get the fastcgi to work on our VMs.

I agree that it's odd to see the sleeping processes.  I think some of them are being called by gbrowse_img. Could there be a difference there somewhere? 

Using 2.52 currently.

Jim

On Jun 24, 2013, at 10:07 AM, Lincoln Stein wrote:

What problems did you see with the user login system? I am working on that subsystem now.

Odd that you would see any sleeping processes when running under vanilla CGI. The gbrowse process goes away after finishing the transaction.

Lincoln

On Monday, June 24, 2013, Jim Hu wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054





--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054


------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054





--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Jim Hu
acachectl -V says prefork.

the fcgid version of fgb2 is definitely faster. I wish I understood what's going on with the multiple sleeping processes.  I just added these lines to my.cnf

# * Sleep timeout (added by JH 7/5/2013)
wait_timeout             =30
interactive_timeout      =30

Is there any reason to not comment out or delete the warn on line 158 of Bio::Graphics::Glyph::wiggle_data?

Jim

On Jul 5, 2013, at 4:35 PM, Lincoln Stein wrote:

Agreed. I just tested mod_fastcgi and gbrowse is very broken against it. It spawns a new process for each request. I will see what can be fixed. However mod_fcgid is working very well in my hands.

Are you using the "worker" style Apache2 or the prefork version? The latter is faster for gbrowse.

Lincoln


On Fri, Jul 5, 2013 at 4:52 PM, Jim Hu <[hidden email]> wrote:
Tried mod_fastcgi instead of mod_fcgid after reading this:

http://www.apachelounge.com/viewtopic.php?t=4385

installed and activated via apt-get and a2enmod.  Edited conf.d/gbrowse.conf to uncomment. Commented out the fcgid stanza. 

<IfModule mod_fastcgi.c>
  Alias /fgb2 "/var/www/cgi-bin/gb2"
  <Location /fgb2>
    SetHandler   fastcgi-script
  </Location>
  # Note: you may need to increase -idle-timeout if file uploads are timing out and returning server                                                                                                    
  # errors.                                                                                                                                                                                             
  FastCgiConfig -idle-timeout 600 -maxClassProcesses 20 -initial-env GBROWSE_CONF=/etc/gbrowse2
</IfModule>

Realized that apache was also running mod_perl and fcgid too, so I inactivated those.

Overall, it was a disaster. Sample from the error log: 
[Fri Jul  5 15:18:38 2013] [notice] GBrowse FastCGI process caught sigTERM. Exiting... (pid 17225)
[Fri Jul  5 15:18:38 2013] [notice] GBrowse FastCGI process caught sigTERM. Exiting... (pid 17225)
[Fri Jul 05 15:25:00 2013] [error] [client 157.56.92.158] File does not exist: /var/www/htdocs/textpresso/robots.txt
[Fri Jul 05 15:27:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:27:02 2013] gbrowse: Warning: something's wrong at /usr/local/share/perl/5.14.2/Bio/Graphics/Glyph/wiggle_data.pm line 158., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:27:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:27:02 2013] gbrowse: Use of uninitialized value $_ in lc at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Render.pm line 3736., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:28:02 2013] gbrowse: alarm clock at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/RenderPanels.pm line 1518., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:02 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:28:02 2013] gbrowse: RenderPanels error: timeout at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/RenderPanels.pm line 1518., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:14 2013] [error] [client 75.102.127.32] [18082] session lock timed out on request after 10.000s, referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:14 2013] [error] [client 75.102.127.32] POST: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/ action=update_sections;section_names=page_title;section_names=span;section_names=galaxy_form;section_names=search_form_objects at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Session.pm line 153., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:28:17 2013] [error] [client 75.102.127.32] [Fri Jul  5 15:28:17 2013] gbrowse: [18049] rendering error track: Timeout; Try turning off tracks or looking at a smaller region. at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Render.pm line 3678., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/?name=id:7347;dbid=mg1655:database
[Fri Jul 05 15:38:26 2013] [alert] [client 165.91.51.61] (2)No such file or directory: FastCGI: failed to connect to (dynamic) server "/var/www/cgi-bin/gb2/gbrowse": something is seriously wrong, any chance the socket/named_pipe directory was removed?, see the FastCgiIpcDir directive, referer: http://heptamer.tamu.edu/fgb2/gbrowse/MG1655/
[Fri Jul 05 15:38:26 2013] [error] [client 165.91.51.61] FastCGI: incomplete headers (0 bytes) received from server "/var/www/cgi-bin/gb2/gbrowse", referer: http://heptamer.tamu.edu/fgb2/gbrowse/MG1655/
[Fri Jul 05 15:38:42 2013] [error] [client 165.91.51.61] UNAUTHORIZED ATTEMPT at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2.pm line 514., referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/
[Fri Jul 05 15:38:42 2013] [error] [client 165.91.51.61] \tBio::Graphics::Browser2::authorized_session('Bio::Graphics::Browser2=HASH(0x2252130)', 'e78b8e09f2ea73e4aaa43f7e8fc3d98d', 'ea154c2dac7ce9bc37a469fe4a0fe4ce', undef) called at /usr/local/lib/perl/5.14.2/Bio/Graphics/Browser2/Render.pm line 70, referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/
[Fri Jul 05 15:38:42 2013] [error] [client 165.91.51.61] \tBio::Graphics::Browser2::Render::new('Bio::Graphics::Browser2::Render::HTML', 'Bio::Graphics::Browser2=HASH(0x2252130)') called at /var/www/cgi-bin/gb2/gbrowse line 50, referer: http://heptamer.tamu.edu/cgi-bin/gb2/gbrowse/MG1655/
[Fri Jul 05 15:39:48 2013] [alert] [client 165.91.51.61] (2)No such file or directory: FastCGI: failed to connect to (dynamic) server "/var/www/cgi-bin/gb2/gbrowse": something is seriously wrong, any chance the socket/named_pipe directory was removed?, see the FastCgiIpcDir directive

Also, when it didn't crash it wasn't improving performance. But the good news is it's didn't create extra sleeping mysql processes! 

Jim


On Jul 5, 2013, at 11:07 AM, Jim Hu wrote:

I'll see if I can reproduce the problems, but I'm going to upgrade first.

Jim
On Jul 3, 2013, at 2:58 PM, Lincoln Stein wrote:

Hi Jim,

I've been playing with this using the current GBrowse version 2.54 (from git) and Bio::Graphics 2.37. I can't document a process leak with either normal cgi-bin and FastCGI. When running under fastcgi there is one sleeping mysql process per database per FastCGI instance; this is expected, because the FastCGI instances cache database handles. Under normal CGI, I see the database handles being created as needed to display the tracks and then destroyed once the request is finished. There are no sleeping mysql processes after all tracks are rendered.

I can confirm that CGI is a lot slower than FastCGI. This is because the entire code and database has to be loaded for each displayed track. 

I tried to reproduce the accumulation of FastCGI processes that you reported being spawned by the login process, but couldn't do reproduce it. If you can help me reproduce the problems you experienced with FastCGI, I'd love to try to help you out.

Lincoln


On Mon, Jun 24, 2013 at 6:06 PM, Jim Hu <[hidden email]> wrote:
This may have been fixed already: the login system was spawning a huge number of mysql threads when fastcgi was enabled.... to the point where gbrowse was crashing.  For a while we were running a cron job that killed mysql processes that were where the time was > some value.  Not pretty.

We spent a little time trying to figure it out, but Nathan couldn't get the fastcgi to work on our VMs.

I agree that it's odd to see the sleeping processes.  I think some of them are being called by gbrowse_img. Could there be a difference there somewhere? 

Using 2.52 currently.

Jim

On Jun 24, 2013, at 10:07 AM, Lincoln Stein wrote:

What problems did you see with the user login system? I am working on that subsystem now.

Odd that you would see any sleeping processes when running under vanilla CGI. The gbrowse process goes away after finishing the transaction.

Lincoln

On Monday, June 24, 2013, Jim Hu wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054





--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054


------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054





--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>
------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Frédéric Sapet
In reply to this post by Lincoln Stein
Hello

GGB is running under the prefork Apache.

This morning I have done some tests.

CGI and renderfarm = 0 : everything went fine. I can see some processes in MySQL, related to session database and the working database (ie : Rice Genome Browser). When the rendering is done, all the processes disapeared in MySQL

FastCGI and renderfarm = 0 : some processes seem to be sticky. Some of them are related to the working database and some to the session database. And when I work with another database, at least one process is created and became sticky too.

So FastCGI seems to be responsible for all the processes that remain in MySQL.

CGI and renderfarm = 1 : looks strange. Up to 600 processes in MySQL. A lot of them are related to the session and working database. But a lot of processes are created for all databases (other genomes). It seems to me that all databases I have worked with (another day, stored in my session data) are involved in processes creation. After the rendering is done, everything calm down and all the processes disapeared.

FastCGI and renderfarm = 1 : the same as above, but a lot of processes become sticky.

So the renderfarm has a strange behaviour. .
it launchs a lot of query, even queries that are not directly related to the current working database. And in use with fastCGI, all the processes are still alive more than one hour later...

Fred


2013/7/5 Lincoln Stein <[hidden email]>
How many apache processes are running simultaneously? Are you using the prefork or the threaded apache?

Lincoln


On Fri, Jul 5, 2013 at 9:08 AM, Frédéric Sapet <[hidden email]> wrote:
Hello Jim

I'm facing the same problem (under FastCGI) : 176 process...

GGB 2.54
This is perl, v5.8.8 built for x86_64-linux-thread-multi
BioPerl is 1.006901
Bio::Graphics 2.32


gnpgenome_sessions is my users database.

xxxx and ArabidoV9 are my annotations databases.       

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+


| Id    | User      | Host                      | db                 | Command | Time  | State | Info                  |

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+

| 52849 | gnpgenome | xxx.xxx.xxx.xxx:56047     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 52857 | gnpgenome | xxx.xxx.xxx.xxx:56056     | Xxxxxxx            | Sleep   | 10027 |       | NULL                  |

| 52859 | gnpgenome | xxx.xxx.xxx.xxx:56059     | gnpgenome_sessions | Sleep   |  2263 |       | NULL                  |

| 53262 | gnpgenome | xxx.xxx.xxx.xxx:56339     | Xxxxxxx            | Sleep   |  2123 |       | NULL                  |

| 53264 | gnpgenome | xxx.xxx.xxx.xxx:56342     | gnpgenome_sessions | Sleep   |  2129 |       | NULL                  |

| 53360 | gnpgenome | xxx.xxx.xxx.xxx:56389     | Xxxxxxx            | Sleep   |  4952 |       | NULL                  |

| 53447 | gnpgenome | xxxxxxx.biogemma.fr:58547 | Xxxxxxx            | Sleep   |  4954 |       | NULL                  |

| 53473 | gnpgenome | xxx.xxx.xxx.xxx:56462     | Xxxxxxx            | Sleep   |  4954 |       | NULL                  |

| 53494 | gnpgenome | xxx.xxx.xxx.xxx:56464     | Xxxxxxxxxxx        | Sleep   |  4952 |       | NULL                  |

| 53565 | gnpgenome | xxx.xxx.xxx.xxx:56540     | Xxxxxxx            | Sleep   |  4951 |       | NULL                  |

| 53598 | gnpgenome | xxxxxxx.biogemma.fr:58592 | Xxxxxxx            | Sleep   |  4950 |       | NULL                  |

| 53661 | gnpgenome | xxxxxxx.biogemma.fr:58653 | Xxxxxxxxxxx        | Sleep   |  4949 |       | NULL                  |

| 53684 | gnpgenome | xxxxxxx.biogemma.fr:58676 | Xxxxxxxxxxx        | Sleep   |  4949 |       | NULL                  |

| 53744 | gnpgenome | xxx.xxx.xxx.xxx:56630     | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53805 | gnpgenome | xxxxxxx.biogemma.fr:58687 | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53815 | gnpgenome | xxxxxxx.biogemma.fr:58691 | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53916 | gnpgenome | xxx.xxx.xxx.xxx:56740     | Xxxxxxxxxxxxxx     | Sleep   |  4784 |       | NULL                  |

| 53953 | gnpgenome | xxxxxxx.biogemma.fr:58755 | Xxxxxxxxxxxxxx     | Sleep   |  4885 |       | NULL                  |

| 53966 | gnpgenome | xxxxxxx.biogemma.fr:58768 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53979 | gnpgenome | xxxxxxx.biogemma.fr:58781 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53987 | gnpgenome | xxxxxxx.biogemma.fr:58788 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53996 | gnpgenome | xxxxxxx.biogemma.fr:58797 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 54060 | gnpgenome | xxx.xxx.xxx.xxx:56820     | Xxxxxxxxxxxxxx     | Sleep   |  4815 |       | NULL                  |

| 54111 | gnpgenome | xxxxxxx.biogemma.fr:58832 | Xxxxxxxxxxxxxx     | Sleep   |  4870 |       | NULL                  |

| 54123 | gnpgenome | xxxxxxx.biogemma.fr:58844 | Xxxxxxxxxxxxxx     | Sleep   |  4870 |       | NULL                  |

| 54296 | gnpgenome | xxx.xxx.xxx.xxx:56938     | gnpgenome_sessions | Sleep   |  4812 |       | NULL                  |

| 54297 | gnpgenome | xxx.xxx.xxx.xxx:56939     | gnpgenome_sessions | Sleep   |  4494 |       | NULL                  |

| 54372 | gnpgenome | xxxxxxx.biogemma.fr:36159 | Xxxxxxxxxxxxxx     | Sleep   |  4804 |       | NULL                  |

| 54949 | gnpgenome | xxx.xxx.xxx.xxx:57274     | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 54951 | gnpgenome | xxx.xxx.xxx.xxx:57277     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 54987 | gnpgenome | xxx.xxx.xxx.xxx:57317     | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 54998 | gnpgenome | xxxxxxx.biogemma.fr:36502 | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 55033 | gnpgenome | xxxxxxx.biogemma.fr:36535 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55044 | gnpgenome | xxxxxxx.biogemma.fr:36546 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55045 | gnpgenome | xxxxxxx.biogemma.fr:36547 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55156 | gnpgenome | xxx.xxx.xxx.xxx:57348     | Xxxxxxx            | Sleep   |  2290 |       | NULL                  |

| 55784 | gnpgenome | xxx.xxx.xxx.xxx:57813     | Xxxxxxxxxxx        | Sleep   |  2594 |       | NULL                  |

| 55809 | gnpgenome | xxx.xxx.xxx.xxx:57838     | gnpgenome_sessions | Sleep   |  2556 |       | NULL                  |

| 55812 | gnpgenome | xxx.xxx.xxx.xxx:57841     | ArabidoV9          | Sleep   |  2522 |       | NULL                  |

| 55920 | gnpgenome | xxxxxxx.biogemma.fr:49728 | ArabidoV9          | Sleep   |  2463 |       | NULL                  |

| 55962 | gnpgenome | xxxxxxx.biogemma.fr:49769 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55973 | gnpgenome | xxxxxxx.biogemma.fr:49781 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55974 | gnpgenome | xxxxxxx.biogemma.fr:49782 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55975 | gnpgenome | xxxxxxx.biogemma.fr:49783 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 56041 | gnpgenome | xxx.xxx.xxx.xxx:57970     | gnpgenome_sessions | Sleep   |  2335 |       | NULL                  |

| 56050 | gnpgenome | xxx.xxx.xxx.xxx:57987     | ArabidoV9          | Sleep   |  2399 |       | NULL                  |

| 56135 | gnpgenome | xxx.xxx.xxx.xxx:58075     | ArabidoV9          | Sleep   |  2302 |       | NULL                  |

| 56136 | gnpgenome | xxx.xxx.xxx.xxx:58076     | Xxxxxxx            | Sleep   |  2302 |       | NULL                  |

| 56171 | gnpgenome | xxx.xxx.xxx.xxx:58112     | Xxxxxxxxx          | Sleep   |  2301 |       | NULL                  |

| 56174 | gnpgenome | xxx.xxx.xxx.xxx:58115     | Xxxxxxxxxxx        | Sleep   |  2301 |       | NULL                  |

| 56175 | gnpgenome | xxx.xxx.xxx.xxx:58116     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56176 | gnpgenome | xxx.xxx.xxx.xxx:58117     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56177 | gnpgenome | xxx.xxx.xxx.xxx:58118     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56226 | gnpgenome | xxx.xxx.xxx.xxx:58131     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56227 | gnpgenome | xxx.xxx.xxx.xxx:58132     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56228 | gnpgenome | xxx.xxx.xxx.xxx:58133     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56229 | gnpgenome | xxx.xxx.xxx.xxx:58134     | Xxxxxxxxxxxxxx     | Sleep   |  2300 |       | NULL                  |

| 56230 | gnpgenome | xxx.xxx.xxx.xxx:58135     | Xxxxxxxxxxx        | Sleep   |  2300 |       | NULL                  |

| 56231 | gnpgenome | xxx.xxx.xxx.xxx:58136     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56232 | gnpgenome | xxx.xxx.xxx.xxx:58137     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56233 | gnpgenome | xxx.xxx.xxx.xxx:58138     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56236 | gnpgenome | xxx.xxx.xxx.xxx:58141     | ArabidoV9          | Sleep   |  2300 |       | NULL                  |

| 56237 | gnpgenome | xxx.xxx.xxx.xxx:58142     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56238 | gnpgenome | xxx.xxx.xxx.xxx:58143     | Xxxxxxxxx          | Sleep   |  2300 |       | NULL                  |

| 56239 | gnpgenome | xxx.xxx.xxx.xxx:58144     | Xxxxxxxxxxx        | Sleep   |  2300 |       | NULL                  |

| 56240 | gnpgenome | xxx.xxx.xxx.xxx:58145     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56241 | gnpgenome | xxx.xxx.xxx.xxx:58146     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56242 | gnpgenome | xxx.xxx.xxx.xxx:58147     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56273 | gnpgenome | xxx.xxx.xxx.xxx:58178     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56274 | gnpgenome | xxx.xxx.xxx.xxx:58179     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56275 | gnpgenome | xxx.xxx.xxx.xxx:58180     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56276 | gnpgenome | xxx.xxx.xxx.xxx:58181     | Xxxxxxxxxxxxxx     | Sleep   |  2299 |       | NULL                  |

| 56277 | gnpgenome | xxx.xxx.xxx.xxx:58182     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56278 | gnpgenome | xxx.xxx.xxx.xxx:58183     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56279 | gnpgenome | xxx.xxx.xxx.xxx:58184     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56280 | gnpgenome | xxx.xxx.xxx.xxx:58185     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56317 | gnpgenome | xxx.xxx.xxx.xxx:58210     | ArabidoV9          | Sleep   |  2299 |       | NULL                  |

| 56318 | gnpgenome | xxx.xxx.xxx.xxx:58211     | ArabidoV9          | Sleep   |  2299 |       | NULL                  |

| 56322 | gnpgenome | xxx.xxx.xxx.xxx:58212     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56323 | gnpgenome | xxx.xxx.xxx.xxx:58213     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56325 | gnpgenome | xxx.xxx.xxx.xxx:58214     | Xxxxxxxxx          | Sleep   |  2299 |       | NULL                  |

| 56326 | gnpgenome | xxx.xxx.xxx.xxx:58215     | Xxxxxxxxx          | Sleep   |  2299 |       | NULL                  |

| 56327 | gnpgenome | xxx.xxx.xxx.xxx:58216     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56328 | gnpgenome | xxx.xxx.xxx.xxx:58217     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56329 | gnpgenome | xxx.xxx.xxx.xxx:58218     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56330 | gnpgenome | xxx.xxx.xxx.xxx:58219     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56331 | gnpgenome | xxx.xxx.xxx.xxx:58220     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56332 | gnpgenome | xxx.xxx.xxx.xxx:58221     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56333 | gnpgenome | xxx.xxx.xxx.xxx:58222     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56334 | gnpgenome | xxx.xxx.xxx.xxx:58223     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56340 | gnpgenome | xxx.xxx.xxx.xxx:58229     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56341 | gnpgenome | xxx.xxx.xxx.xxx:58230     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56342 | gnpgenome | xxx.xxx.xxx.xxx:58231     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56343 | gnpgenome | xxx.xxx.xxx.xxx:58232     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56345 | gnpgenome | xxx.xxx.xxx.xxx:58234     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56346 | gnpgenome | xxx.xxx.xxx.xxx:58235     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56347 | gnpgenome | xxx.xxx.xxx.xxx:58236     | Xxxxxxxxxxxxxx     | Sleep   |  2298 |       | NULL                  |

| 56348 | gnpgenome | xxx.xxx.xxx.xxx:58237     | Xxxxxxxxxxxxxx     | Sleep   |  2298 |       | NULL                  |

| 56349 | gnpgenome | xxx.xxx.xxx.xxx:58238     | Xxxxxxxxxxx        | Sleep   |  2298 |       | NULL                  |

| 56350 | gnpgenome | xxx.xxx.xxx.xxx:58239     | Xxxxxxxxxxx        | Sleep   |  2298 |       | NULL                  |

| 56351 | gnpgenome | xxx.xxx.xxx.xxx:58240     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56352 | gnpgenome | xxx.xxx.xxx.xxx:58241     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56353 | gnpgenome | xxx.xxx.xxx.xxx:58242     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56354 | gnpgenome | xxx.xxx.xxx.xxx:58243     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56355 | gnpgenome | xxx.xxx.xxx.xxx:58244     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56357 | gnpgenome | xxx.xxx.xxx.xxx:58246     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56411 | gnpgenome | xxx.xxx.xxx.xxx:58276     | Xxxxxxx            | Sleep   |  2278 |       | NULL                  |

| 56413 | gnpgenome | xxx.xxx.xxx.xxx:58279     | gnpgenome_sessions | Sleep   |  2020 |       | NULL                  |

| 56423 | gnpgenome | xxx.xxx.xxx.xxx:58289     | Xxxxxxx            | Sleep   |  2218 |       | NULL                  |

| 56440 | gnpgenome | xxx.xxx.xxx.xxx:58305     | ArabidoV9          | Sleep   |  2292 |       | NULL                  |

| 56442 | gnpgenome | xxx.xxx.xxx.xxx:58307     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56443 | gnpgenome | xxx.xxx.xxx.xxx:58308     | Xxxxxxxxx          | Sleep   |  2292 |       | NULL                  |

| 56448 | gnpgenome | xxx.xxx.xxx.xxx:58309     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56449 | gnpgenome | xxx.xxx.xxx.xxx:58310     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56450 | gnpgenome | xxx.xxx.xxx.xxx:58311     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56452 | gnpgenome | xxx.xxx.xxx.xxx:58313     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56477 | gnpgenome | xxx.xxx.xxx.xxx:58319     | ArabidoV9          | Sleep   |  2292 |       | NULL                  |

| 56478 | gnpgenome | xxx.xxx.xxx.xxx:58320     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56479 | gnpgenome | xxx.xxx.xxx.xxx:58321     | Xxxxxxxxx          | Sleep   |  2292 |       | NULL                  |

| 56480 | gnpgenome | xxx.xxx.xxx.xxx:58322     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56481 | gnpgenome | xxx.xxx.xxx.xxx:58323     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56482 | gnpgenome | xxx.xxx.xxx.xxx:58324     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56483 | gnpgenome | xxx.xxx.xxx.xxx:58325     | Xxxxxxxxxxxxxx     | Sleep   |  2292 |       | NULL                  |

| 56485 | gnpgenome | xxx.xxx.xxx.xxx:58327     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56486 | gnpgenome | xxx.xxx.xxx.xxx:58328     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56487 | gnpgenome | xxx.xxx.xxx.xxx:58329     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56488 | gnpgenome | xxx.xxx.xxx.xxx:58330     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56489 | gnpgenome | xxx.xxx.xxx.xxx:58331     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56490 | gnpgenome | xxx.xxx.xxx.xxx:58332     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56491 | gnpgenome | xxx.xxx.xxx.xxx:58333     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56492 | gnpgenome | xxx.xxx.xxx.xxx:58334     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56521 | gnpgenome | xxx.xxx.xxx.xxx:58363     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56531 | gnpgenome | xxx.xxx.xxx.xxx:58374     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56532 | gnpgenome | xxx.xxx.xxx.xxx:58375     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56534 | gnpgenome | xxx.xxx.xxx.xxx:58377     | Xxxxxxxxxxxxxx     | Sleep   |  2291 |       | NULL                  |

| 56536 | gnpgenome | xxx.xxx.xxx.xxx:58379     | Xxxxxxxxxxx        | Sleep   |  2291 |       | NULL                  |

| 56537 | gnpgenome | xxx.xxx.xxx.xxx:58380     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56539 | gnpgenome | xxx.xxx.xxx.xxx:58381     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56542 | gnpgenome | xxx.xxx.xxx.xxx:58382     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56600 | gnpgenome | xxx.xxx.xxx.xxx:58421     | ArabidoV9          | Sleep   |  2290 |       | NULL                  |

| 56631 | gnpgenome | xxx.xxx.xxx.xxx:58438     | Xxxxxxx            | Sleep   |  2289 |       | NULL                  |

| 56658 | gnpgenome | xxx.xxx.xxx.xxx:58447     | Xxxxxxxxx          | Sleep   |  2285 |       | NULL                  |

| 56671 | gnpgenome | xxx.xxx.xxx.xxx:58451     | Xxxxxxxxxxx        | Sleep   |  2283 |       | NULL                  |

| 56683 | gnpgenome | xxx.xxx.xxx.xxx:58453     | gnpgenome_sessions | Sleep   |  2282 |       | NULL                  |

| 56685 | gnpgenome | xxx.xxx.xxx.xxx:58455     | gnpgenome_sessions | Sleep   |  2281 |       | NULL                  |

| 56690 | gnpgenome | xxx.xxx.xxx.xxx:58460     | gnpgenome_sessions | Sleep   |  2279 |       | NULL                  |

| 56733 | gnpgenome | xxx.xxx.xxx.xxx:58494     | gnpgenome_sessions | Sleep   |  2262 |       | NULL                  |

| 56738 | gnpgenome | xxx.xxx.xxx.xxx:58499     | gnpgenome_sessions | Sleep   |  2262 |       | NULL                  |

| 56750 | gnpgenome | xxx.xxx.xxx.xxx:58511     | gnpgenome_sessions | Sleep   |  2263 |       | NULL                  |

| 56753 | gnpgenome | xxx.xxx.xxx.xxx:58516     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 56761 | gnpgenome | xxx.xxx.xxx.xxx:58518     | gnpgenome_sessions | Sleep   |  2254 |       | NULL                  |

| 56769 | gnpgenome | xxx.xxx.xxx.xxx:58520     | Xxxxxxxxx          | Sleep   |  2262 |       | NULL                  |

| 56770 | gnpgenome | xxx.xxx.xxx.xxx:58521     | Xxxxxxxxxxxxxx     | Sleep   |  2262 |       | NULL                  |

| 56771 | gnpgenome | xxx.xxx.xxx.xxx:58522     | Xxxxxxxxxxx        | Sleep   |  2262 |       | NULL                  |

| 56778 | gnpgenome | xxx.xxx.xxx.xxx:58523     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 56779 | gnpgenome | xxx.xxx.xxx.xxx:58524     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56780 | gnpgenome | xxx.xxx.xxx.xxx:58525     | ArabidoV9          | Sleep   |  2261 |       | NULL                  |

| 56781 | gnpgenome | xxx.xxx.xxx.xxx:58526     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56782 | gnpgenome | xxx.xxx.xxx.xxx:58527     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56783 | gnpgenome | xxx.xxx.xxx.xxx:58528     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56790 | gnpgenome | xxx.xxx.xxx.xxx:58529     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56791 | gnpgenome | xxx.xxx.xxx.xxx:58530     | Xxxxxxxxx          | Sleep   |  2261 |       | NULL                  |

| 56792 | gnpgenome | xxx.xxx.xxx.xxx:58531     | Xxxxxxxxxxxxxx     | Sleep   |  2261 |       | NULL                  |

| 56793 | gnpgenome | xxx.xxx.xxx.xxx:58532     | Xxxxxxxxxxx        | Sleep   |  2261 |       | NULL                  |

| 56795 | gnpgenome | xxx.xxx.xxx.xxx:58534     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56796 | gnpgenome | xxx.xxx.xxx.xxx:58535     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56797 | gnpgenome | xxx.xxx.xxx.xxx:58536     | ArabidoV9          | Sleep   |  2261 |       | NULL                  |

| 56798 | gnpgenome | xxx.xxx.xxx.xxx:58537     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56799 | gnpgenome | xxx.xxx.xxx.xxx:58538     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56800 | gnpgenome | xxx.xxx.xxx.xxx:58539     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56827 | gnpgenome | xxx.xxx.xxx.xxx:58560     | Xxxxxxx            | Sleep   |  2241 |       | NULL                  |

| 56829 | gnpgenome | xxx.xxx.xxx.xxx:58563     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57019 | gnpgenome | xxx.xxx.xxx.xxx:58642     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57021 | gnpgenome | xxx.xxx.xxx.xxx:58644     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57043 | gnpgenome | xxx.xxx.xxx.xxx:58672     | gnpgenome_sessions | Sleep   |  2020 |       | NULL                  |

| 57121 | gnpgenome | xxx.xxx.xxx.xxx:58676     | Xxxxxxx            | Sleep   |   542 |       | NULL                  |

| 57144 | gnpgenome | xxxxxxx.biogemma.fr:43501 | NULL               | Query   |     0 | NULL  | show full processlist |

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+
177 rows in set (0.00 sec)


Fred

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>


------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Lincoln Stein
Hi All,

Something is wrong with the way that the rendering processes are exiting. I promise to get this sorted out soon -- it has risen to the top of my priority list.

Lincoln


On Mon, Jul 8, 2013 at 10:03 AM, Frédéric Sapet <[hidden email]> wrote:
Hello

GGB is running under the prefork Apache.

This morning I have done some tests.

CGI and renderfarm = 0 : everything went fine. I can see some processes in MySQL, related to session database and the working database (ie : Rice Genome Browser). When the rendering is done, all the processes disapeared in MySQL

FastCGI and renderfarm = 0 : some processes seem to be sticky. Some of them are related to the working database and some to the session database. And when I work with another database, at least one process is created and became sticky too.

So FastCGI seems to be responsible for all the processes that remain in MySQL.

CGI and renderfarm = 1 : looks strange. Up to 600 processes in MySQL. A lot of them are related to the session and working database. But a lot of processes are created for all databases (other genomes). It seems to me that all databases I have worked with (another day, stored in my session data) are involved in processes creation. After the rendering is done, everything calm down and all the processes disapeared.

FastCGI and renderfarm = 1 : the same as above, but a lot of processes become sticky.

So the renderfarm has a strange behaviour. .
it launchs a lot of query, even queries that are not directly related to the current working database. And in use with fastCGI, all the processes are still alive more than one hour later...

Fred


2013/7/5 Lincoln Stein <[hidden email]>
How many apache processes are running simultaneously? Are you using the prefork or the threaded apache?

Lincoln


On Fri, Jul 5, 2013 at 9:08 AM, Frédéric Sapet <[hidden email]> wrote:
Hello Jim

I'm facing the same problem (under FastCGI) : 176 process...

GGB 2.54
This is perl, v5.8.8 built for x86_64-linux-thread-multi
BioPerl is 1.006901
Bio::Graphics 2.32


gnpgenome_sessions is my users database.

xxxx and ArabidoV9 are my annotations databases.       

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+


| Id    | User      | Host                      | db                 | Command | Time  | State | Info                  |

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+

| 52849 | gnpgenome | xxx.xxx.xxx.xxx:56047     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 52857 | gnpgenome | xxx.xxx.xxx.xxx:56056     | Xxxxxxx            | Sleep   | 10027 |       | NULL                  |

| 52859 | gnpgenome | xxx.xxx.xxx.xxx:56059     | gnpgenome_sessions | Sleep   |  2263 |       | NULL                  |

| 53262 | gnpgenome | xxx.xxx.xxx.xxx:56339     | Xxxxxxx            | Sleep   |  2123 |       | NULL                  |

| 53264 | gnpgenome | xxx.xxx.xxx.xxx:56342     | gnpgenome_sessions | Sleep   |  2129 |       | NULL                  |

| 53360 | gnpgenome | xxx.xxx.xxx.xxx:56389     | Xxxxxxx            | Sleep   |  4952 |       | NULL                  |

| 53447 | gnpgenome | xxxxxxx.biogemma.fr:58547 | Xxxxxxx            | Sleep   |  4954 |       | NULL                  |

| 53473 | gnpgenome | xxx.xxx.xxx.xxx:56462     | Xxxxxxx            | Sleep   |  4954 |       | NULL                  |

| 53494 | gnpgenome | xxx.xxx.xxx.xxx:56464     | Xxxxxxxxxxx        | Sleep   |  4952 |       | NULL                  |

| 53565 | gnpgenome | xxx.xxx.xxx.xxx:56540     | Xxxxxxx            | Sleep   |  4951 |       | NULL                  |

| 53598 | gnpgenome | xxxxxxx.biogemma.fr:58592 | Xxxxxxx            | Sleep   |  4950 |       | NULL                  |

| 53661 | gnpgenome | xxxxxxx.biogemma.fr:58653 | Xxxxxxxxxxx        | Sleep   |  4949 |       | NULL                  |

| 53684 | gnpgenome | xxxxxxx.biogemma.fr:58676 | Xxxxxxxxxxx        | Sleep   |  4949 |       | NULL                  |

| 53744 | gnpgenome | xxx.xxx.xxx.xxx:56630     | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53805 | gnpgenome | xxxxxxx.biogemma.fr:58687 | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53815 | gnpgenome | xxxxxxx.biogemma.fr:58691 | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53916 | gnpgenome | xxx.xxx.xxx.xxx:56740     | Xxxxxxxxxxxxxx     | Sleep   |  4784 |       | NULL                  |

| 53953 | gnpgenome | xxxxxxx.biogemma.fr:58755 | Xxxxxxxxxxxxxx     | Sleep   |  4885 |       | NULL                  |

| 53966 | gnpgenome | xxxxxxx.biogemma.fr:58768 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53979 | gnpgenome | xxxxxxx.biogemma.fr:58781 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53987 | gnpgenome | xxxxxxx.biogemma.fr:58788 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53996 | gnpgenome | xxxxxxx.biogemma.fr:58797 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 54060 | gnpgenome | xxx.xxx.xxx.xxx:56820     | Xxxxxxxxxxxxxx     | Sleep   |  4815 |       | NULL                  |

| 54111 | gnpgenome | xxxxxxx.biogemma.fr:58832 | Xxxxxxxxxxxxxx     | Sleep   |  4870 |       | NULL                  |

| 54123 | gnpgenome | xxxxxxx.biogemma.fr:58844 | Xxxxxxxxxxxxxx     | Sleep   |  4870 |       | NULL                  |

| 54296 | gnpgenome | xxx.xxx.xxx.xxx:56938     | gnpgenome_sessions | Sleep   |  4812 |       | NULL                  |

| 54297 | gnpgenome | xxx.xxx.xxx.xxx:56939     | gnpgenome_sessions | Sleep   |  4494 |       | NULL                  |

| 54372 | gnpgenome | xxxxxxx.biogemma.fr:36159 | Xxxxxxxxxxxxxx     | Sleep   |  4804 |       | NULL                  |

| 54949 | gnpgenome | xxx.xxx.xxx.xxx:57274     | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 54951 | gnpgenome | xxx.xxx.xxx.xxx:57277     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 54987 | gnpgenome | xxx.xxx.xxx.xxx:57317     | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 54998 | gnpgenome | xxxxxxx.biogemma.fr:36502 | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 55033 | gnpgenome | xxxxxxx.biogemma.fr:36535 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55044 | gnpgenome | xxxxxxx.biogemma.fr:36546 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55045 | gnpgenome | xxxxxxx.biogemma.fr:36547 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55156 | gnpgenome | xxx.xxx.xxx.xxx:57348     | Xxxxxxx            | Sleep   |  2290 |       | NULL                  |

| 55784 | gnpgenome | xxx.xxx.xxx.xxx:57813     | Xxxxxxxxxxx        | Sleep   |  2594 |       | NULL                  |

| 55809 | gnpgenome | xxx.xxx.xxx.xxx:57838     | gnpgenome_sessions | Sleep   |  2556 |       | NULL                  |

| 55812 | gnpgenome | xxx.xxx.xxx.xxx:57841     | ArabidoV9          | Sleep   |  2522 |       | NULL                  |

| 55920 | gnpgenome | xxxxxxx.biogemma.fr:49728 | ArabidoV9          | Sleep   |  2463 |       | NULL                  |

| 55962 | gnpgenome | xxxxxxx.biogemma.fr:49769 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55973 | gnpgenome | xxxxxxx.biogemma.fr:49781 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55974 | gnpgenome | xxxxxxx.biogemma.fr:49782 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55975 | gnpgenome | xxxxxxx.biogemma.fr:49783 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 56041 | gnpgenome | xxx.xxx.xxx.xxx:57970     | gnpgenome_sessions | Sleep   |  2335 |       | NULL                  |

| 56050 | gnpgenome | xxx.xxx.xxx.xxx:57987     | ArabidoV9          | Sleep   |  2399 |       | NULL                  |

| 56135 | gnpgenome | xxx.xxx.xxx.xxx:58075     | ArabidoV9          | Sleep   |  2302 |       | NULL                  |

| 56136 | gnpgenome | xxx.xxx.xxx.xxx:58076     | Xxxxxxx            | Sleep   |  2302 |       | NULL                  |

| 56171 | gnpgenome | xxx.xxx.xxx.xxx:58112     | Xxxxxxxxx          | Sleep   |  2301 |       | NULL                  |

| 56174 | gnpgenome | xxx.xxx.xxx.xxx:58115     | Xxxxxxxxxxx        | Sleep   |  2301 |       | NULL                  |

| 56175 | gnpgenome | xxx.xxx.xxx.xxx:58116     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56176 | gnpgenome | xxx.xxx.xxx.xxx:58117     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56177 | gnpgenome | xxx.xxx.xxx.xxx:58118     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56226 | gnpgenome | xxx.xxx.xxx.xxx:58131     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56227 | gnpgenome | xxx.xxx.xxx.xxx:58132     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56228 | gnpgenome | xxx.xxx.xxx.xxx:58133     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56229 | gnpgenome | xxx.xxx.xxx.xxx:58134     | Xxxxxxxxxxxxxx     | Sleep   |  2300 |       | NULL                  |

| 56230 | gnpgenome | xxx.xxx.xxx.xxx:58135     | Xxxxxxxxxxx        | Sleep   |  2300 |       | NULL                  |

| 56231 | gnpgenome | xxx.xxx.xxx.xxx:58136     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56232 | gnpgenome | xxx.xxx.xxx.xxx:58137     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56233 | gnpgenome | xxx.xxx.xxx.xxx:58138     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56236 | gnpgenome | xxx.xxx.xxx.xxx:58141     | ArabidoV9          | Sleep   |  2300 |       | NULL                  |

| 56237 | gnpgenome | xxx.xxx.xxx.xxx:58142     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56238 | gnpgenome | xxx.xxx.xxx.xxx:58143     | Xxxxxxxxx          | Sleep   |  2300 |       | NULL                  |

| 56239 | gnpgenome | xxx.xxx.xxx.xxx:58144     | Xxxxxxxxxxx        | Sleep   |  2300 |       | NULL                  |

| 56240 | gnpgenome | xxx.xxx.xxx.xxx:58145     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56241 | gnpgenome | xxx.xxx.xxx.xxx:58146     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56242 | gnpgenome | xxx.xxx.xxx.xxx:58147     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56273 | gnpgenome | xxx.xxx.xxx.xxx:58178     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56274 | gnpgenome | xxx.xxx.xxx.xxx:58179     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56275 | gnpgenome | xxx.xxx.xxx.xxx:58180     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56276 | gnpgenome | xxx.xxx.xxx.xxx:58181     | Xxxxxxxxxxxxxx     | Sleep   |  2299 |       | NULL                  |

| 56277 | gnpgenome | xxx.xxx.xxx.xxx:58182     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56278 | gnpgenome | xxx.xxx.xxx.xxx:58183     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56279 | gnpgenome | xxx.xxx.xxx.xxx:58184     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56280 | gnpgenome | xxx.xxx.xxx.xxx:58185     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56317 | gnpgenome | xxx.xxx.xxx.xxx:58210     | ArabidoV9          | Sleep   |  2299 |       | NULL                  |

| 56318 | gnpgenome | xxx.xxx.xxx.xxx:58211     | ArabidoV9          | Sleep   |  2299 |       | NULL                  |

| 56322 | gnpgenome | xxx.xxx.xxx.xxx:58212     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56323 | gnpgenome | xxx.xxx.xxx.xxx:58213     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56325 | gnpgenome | xxx.xxx.xxx.xxx:58214     | Xxxxxxxxx          | Sleep   |  2299 |       | NULL                  |

| 56326 | gnpgenome | xxx.xxx.xxx.xxx:58215     | Xxxxxxxxx          | Sleep   |  2299 |       | NULL                  |

| 56327 | gnpgenome | xxx.xxx.xxx.xxx:58216     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56328 | gnpgenome | xxx.xxx.xxx.xxx:58217     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56329 | gnpgenome | xxx.xxx.xxx.xxx:58218     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56330 | gnpgenome | xxx.xxx.xxx.xxx:58219     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56331 | gnpgenome | xxx.xxx.xxx.xxx:58220     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56332 | gnpgenome | xxx.xxx.xxx.xxx:58221     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56333 | gnpgenome | xxx.xxx.xxx.xxx:58222     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56334 | gnpgenome | xxx.xxx.xxx.xxx:58223     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56340 | gnpgenome | xxx.xxx.xxx.xxx:58229     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56341 | gnpgenome | xxx.xxx.xxx.xxx:58230     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56342 | gnpgenome | xxx.xxx.xxx.xxx:58231     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56343 | gnpgenome | xxx.xxx.xxx.xxx:58232     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56345 | gnpgenome | xxx.xxx.xxx.xxx:58234     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56346 | gnpgenome | xxx.xxx.xxx.xxx:58235     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56347 | gnpgenome | xxx.xxx.xxx.xxx:58236     | Xxxxxxxxxxxxxx     | Sleep   |  2298 |       | NULL                  |

| 56348 | gnpgenome | xxx.xxx.xxx.xxx:58237     | Xxxxxxxxxxxxxx     | Sleep   |  2298 |       | NULL                  |

| 56349 | gnpgenome | xxx.xxx.xxx.xxx:58238     | Xxxxxxxxxxx        | Sleep   |  2298 |       | NULL                  |

| 56350 | gnpgenome | xxx.xxx.xxx.xxx:58239     | Xxxxxxxxxxx        | Sleep   |  2298 |       | NULL                  |

| 56351 | gnpgenome | xxx.xxx.xxx.xxx:58240     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56352 | gnpgenome | xxx.xxx.xxx.xxx:58241     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56353 | gnpgenome | xxx.xxx.xxx.xxx:58242     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56354 | gnpgenome | xxx.xxx.xxx.xxx:58243     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56355 | gnpgenome | xxx.xxx.xxx.xxx:58244     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56357 | gnpgenome | xxx.xxx.xxx.xxx:58246     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56411 | gnpgenome | xxx.xxx.xxx.xxx:58276     | Xxxxxxx            | Sleep   |  2278 |       | NULL                  |

| 56413 | gnpgenome | xxx.xxx.xxx.xxx:58279     | gnpgenome_sessions | Sleep   |  2020 |       | NULL                  |

| 56423 | gnpgenome | xxx.xxx.xxx.xxx:58289     | Xxxxxxx            | Sleep   |  2218 |       | NULL                  |

| 56440 | gnpgenome | xxx.xxx.xxx.xxx:58305     | ArabidoV9          | Sleep   |  2292 |       | NULL                  |

| 56442 | gnpgenome | xxx.xxx.xxx.xxx:58307     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56443 | gnpgenome | xxx.xxx.xxx.xxx:58308     | Xxxxxxxxx          | Sleep   |  2292 |       | NULL                  |

| 56448 | gnpgenome | xxx.xxx.xxx.xxx:58309     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56449 | gnpgenome | xxx.xxx.xxx.xxx:58310     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56450 | gnpgenome | xxx.xxx.xxx.xxx:58311     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56452 | gnpgenome | xxx.xxx.xxx.xxx:58313     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56477 | gnpgenome | xxx.xxx.xxx.xxx:58319     | ArabidoV9          | Sleep   |  2292 |       | NULL                  |

| 56478 | gnpgenome | xxx.xxx.xxx.xxx:58320     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56479 | gnpgenome | xxx.xxx.xxx.xxx:58321     | Xxxxxxxxx          | Sleep   |  2292 |       | NULL                  |

| 56480 | gnpgenome | xxx.xxx.xxx.xxx:58322     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56481 | gnpgenome | xxx.xxx.xxx.xxx:58323     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56482 | gnpgenome | xxx.xxx.xxx.xxx:58324     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56483 | gnpgenome | xxx.xxx.xxx.xxx:58325     | Xxxxxxxxxxxxxx     | Sleep   |  2292 |       | NULL                  |

| 56485 | gnpgenome | xxx.xxx.xxx.xxx:58327     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56486 | gnpgenome | xxx.xxx.xxx.xxx:58328     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56487 | gnpgenome | xxx.xxx.xxx.xxx:58329     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56488 | gnpgenome | xxx.xxx.xxx.xxx:58330     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56489 | gnpgenome | xxx.xxx.xxx.xxx:58331     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56490 | gnpgenome | xxx.xxx.xxx.xxx:58332     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56491 | gnpgenome | xxx.xxx.xxx.xxx:58333     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56492 | gnpgenome | xxx.xxx.xxx.xxx:58334     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56521 | gnpgenome | xxx.xxx.xxx.xxx:58363     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56531 | gnpgenome | xxx.xxx.xxx.xxx:58374     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56532 | gnpgenome | xxx.xxx.xxx.xxx:58375     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56534 | gnpgenome | xxx.xxx.xxx.xxx:58377     | Xxxxxxxxxxxxxx     | Sleep   |  2291 |       | NULL                  |

| 56536 | gnpgenome | xxx.xxx.xxx.xxx:58379     | Xxxxxxxxxxx        | Sleep   |  2291 |       | NULL                  |

| 56537 | gnpgenome | xxx.xxx.xxx.xxx:58380     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56539 | gnpgenome | xxx.xxx.xxx.xxx:58381     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56542 | gnpgenome | xxx.xxx.xxx.xxx:58382     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56600 | gnpgenome | xxx.xxx.xxx.xxx:58421     | ArabidoV9          | Sleep   |  2290 |       | NULL                  |

| 56631 | gnpgenome | xxx.xxx.xxx.xxx:58438     | Xxxxxxx            | Sleep   |  2289 |       | NULL                  |

| 56658 | gnpgenome | xxx.xxx.xxx.xxx:58447     | Xxxxxxxxx          | Sleep   |  2285 |       | NULL                  |

| 56671 | gnpgenome | xxx.xxx.xxx.xxx:58451     | Xxxxxxxxxxx        | Sleep   |  2283 |       | NULL                  |

| 56683 | gnpgenome | xxx.xxx.xxx.xxx:58453     | gnpgenome_sessions | Sleep   |  2282 |       | NULL                  |

| 56685 | gnpgenome | xxx.xxx.xxx.xxx:58455     | gnpgenome_sessions | Sleep   |  2281 |       | NULL                  |

| 56690 | gnpgenome | xxx.xxx.xxx.xxx:58460     | gnpgenome_sessions | Sleep   |  2279 |       | NULL                  |

| 56733 | gnpgenome | xxx.xxx.xxx.xxx:58494     | gnpgenome_sessions | Sleep   |  2262 |       | NULL                  |

| 56738 | gnpgenome | xxx.xxx.xxx.xxx:58499     | gnpgenome_sessions | Sleep   |  2262 |       | NULL                  |

| 56750 | gnpgenome | xxx.xxx.xxx.xxx:58511     | gnpgenome_sessions | Sleep   |  2263 |       | NULL                  |

| 56753 | gnpgenome | xxx.xxx.xxx.xxx:58516     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 56761 | gnpgenome | xxx.xxx.xxx.xxx:58518     | gnpgenome_sessions | Sleep   |  2254 |       | NULL                  |

| 56769 | gnpgenome | xxx.xxx.xxx.xxx:58520     | Xxxxxxxxx          | Sleep   |  2262 |       | NULL                  |

| 56770 | gnpgenome | xxx.xxx.xxx.xxx:58521     | Xxxxxxxxxxxxxx     | Sleep   |  2262 |       | NULL                  |

| 56771 | gnpgenome | xxx.xxx.xxx.xxx:58522     | Xxxxxxxxxxx        | Sleep   |  2262 |       | NULL                  |

| 56778 | gnpgenome | xxx.xxx.xxx.xxx:58523     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 56779 | gnpgenome | xxx.xxx.xxx.xxx:58524     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56780 | gnpgenome | xxx.xxx.xxx.xxx:58525     | ArabidoV9          | Sleep   |  2261 |       | NULL                  |

| 56781 | gnpgenome | xxx.xxx.xxx.xxx:58526     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56782 | gnpgenome | xxx.xxx.xxx.xxx:58527     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56783 | gnpgenome | xxx.xxx.xxx.xxx:58528     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56790 | gnpgenome | xxx.xxx.xxx.xxx:58529     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56791 | gnpgenome | xxx.xxx.xxx.xxx:58530     | Xxxxxxxxx          | Sleep   |  2261 |       | NULL                  |

| 56792 | gnpgenome | xxx.xxx.xxx.xxx:58531     | Xxxxxxxxxxxxxx     | Sleep   |  2261 |       | NULL                  |

| 56793 | gnpgenome | xxx.xxx.xxx.xxx:58532     | Xxxxxxxxxxx        | Sleep   |  2261 |       | NULL                  |

| 56795 | gnpgenome | xxx.xxx.xxx.xxx:58534     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56796 | gnpgenome | xxx.xxx.xxx.xxx:58535     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56797 | gnpgenome | xxx.xxx.xxx.xxx:58536     | ArabidoV9          | Sleep   |  2261 |       | NULL                  |

| 56798 | gnpgenome | xxx.xxx.xxx.xxx:58537     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56799 | gnpgenome | xxx.xxx.xxx.xxx:58538     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56800 | gnpgenome | xxx.xxx.xxx.xxx:58539     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56827 | gnpgenome | xxx.xxx.xxx.xxx:58560     | Xxxxxxx            | Sleep   |  2241 |       | NULL                  |

| 56829 | gnpgenome | xxx.xxx.xxx.xxx:58563     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57019 | gnpgenome | xxx.xxx.xxx.xxx:58642     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57021 | gnpgenome | xxx.xxx.xxx.xxx:58644     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57043 | gnpgenome | xxx.xxx.xxx.xxx:58672     | gnpgenome_sessions | Sleep   |  2020 |       | NULL                  |

| 57121 | gnpgenome | xxx.xxx.xxx.xxx:58676     | Xxxxxxx            | Sleep   |   542 |       | NULL                  |

| 57144 | gnpgenome | xxxxxxx.biogemma.fr:43501 | NULL               | Query   |     0 | NULL  | show full processlist |

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+
177 rows in set (0.00 sec)


Fred

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Frédéric Sapet
Hi Lincoln

Thank you.

Fred


2013/7/8 Lincoln Stein <[hidden email]>
Hi All,

Something is wrong with the way that the rendering processes are exiting. I promise to get this sorted out soon -- it has risen to the top of my priority list.

Lincoln


On Mon, Jul 8, 2013 at 10:03 AM, Frédéric Sapet <[hidden email]> wrote:
Hello

GGB is running under the prefork Apache.

This morning I have done some tests.

CGI and renderfarm = 0 : everything went fine. I can see some processes in MySQL, related to session database and the working database (ie : Rice Genome Browser). When the rendering is done, all the processes disapeared in MySQL

FastCGI and renderfarm = 0 : some processes seem to be sticky. Some of them are related to the working database and some to the session database. And when I work with another database, at least one process is created and became sticky too.

So FastCGI seems to be responsible for all the processes that remain in MySQL.

CGI and renderfarm = 1 : looks strange. Up to 600 processes in MySQL. A lot of them are related to the session and working database. But a lot of processes are created for all databases (other genomes). It seems to me that all databases I have worked with (another day, stored in my session data) are involved in processes creation. After the rendering is done, everything calm down and all the processes disapeared.

FastCGI and renderfarm = 1 : the same as above, but a lot of processes become sticky.

So the renderfarm has a strange behaviour. .
it launchs a lot of query, even queries that are not directly related to the current working database. And in use with fastCGI, all the processes are still alive more than one hour later...

Fred


2013/7/5 Lincoln Stein <[hidden email]>
How many apache processes are running simultaneously? Are you using the prefork or the threaded apache?

Lincoln


On Fri, Jul 5, 2013 at 9:08 AM, Frédéric Sapet <[hidden email]> wrote:
Hello Jim

I'm facing the same problem (under FastCGI) : 176 process...

GGB 2.54
This is perl, v5.8.8 built for x86_64-linux-thread-multi
BioPerl is 1.006901
Bio::Graphics 2.32


gnpgenome_sessions is my users database.

xxxx and ArabidoV9 are my annotations databases.       

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+


| Id    | User      | Host                      | db                 | Command | Time  | State | Info                  |

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+

| 52849 | gnpgenome | xxx.xxx.xxx.xxx:56047     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 52857 | gnpgenome | xxx.xxx.xxx.xxx:56056     | Xxxxxxx            | Sleep   | 10027 |       | NULL                  |

| 52859 | gnpgenome | xxx.xxx.xxx.xxx:56059     | gnpgenome_sessions | Sleep   |  2263 |       | NULL                  |

| 53262 | gnpgenome | xxx.xxx.xxx.xxx:56339     | Xxxxxxx            | Sleep   |  2123 |       | NULL                  |

| 53264 | gnpgenome | xxx.xxx.xxx.xxx:56342     | gnpgenome_sessions | Sleep   |  2129 |       | NULL                  |

| 53360 | gnpgenome | xxx.xxx.xxx.xxx:56389     | Xxxxxxx            | Sleep   |  4952 |       | NULL                  |

| 53447 | gnpgenome | xxxxxxx.biogemma.fr:58547 | Xxxxxxx            | Sleep   |  4954 |       | NULL                  |

| 53473 | gnpgenome | xxx.xxx.xxx.xxx:56462     | Xxxxxxx            | Sleep   |  4954 |       | NULL                  |

| 53494 | gnpgenome | xxx.xxx.xxx.xxx:56464     | Xxxxxxxxxxx        | Sleep   |  4952 |       | NULL                  |

| 53565 | gnpgenome | xxx.xxx.xxx.xxx:56540     | Xxxxxxx            | Sleep   |  4951 |       | NULL                  |

| 53598 | gnpgenome | xxxxxxx.biogemma.fr:58592 | Xxxxxxx            | Sleep   |  4950 |       | NULL                  |

| 53661 | gnpgenome | xxxxxxx.biogemma.fr:58653 | Xxxxxxxxxxx        | Sleep   |  4949 |       | NULL                  |

| 53684 | gnpgenome | xxxxxxx.biogemma.fr:58676 | Xxxxxxxxxxx        | Sleep   |  4949 |       | NULL                  |

| 53744 | gnpgenome | xxx.xxx.xxx.xxx:56630     | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53805 | gnpgenome | xxxxxxx.biogemma.fr:58687 | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53815 | gnpgenome | xxxxxxx.biogemma.fr:58691 | Xxxxxxxxxxx        | Sleep   |  4907 |       | NULL                  |

| 53916 | gnpgenome | xxx.xxx.xxx.xxx:56740     | Xxxxxxxxxxxxxx     | Sleep   |  4784 |       | NULL                  |

| 53953 | gnpgenome | xxxxxxx.biogemma.fr:58755 | Xxxxxxxxxxxxxx     | Sleep   |  4885 |       | NULL                  |

| 53966 | gnpgenome | xxxxxxx.biogemma.fr:58768 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53979 | gnpgenome | xxxxxxx.biogemma.fr:58781 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53987 | gnpgenome | xxxxxxx.biogemma.fr:58788 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 53996 | gnpgenome | xxxxxxx.biogemma.fr:58797 | Xxxxxxxxxxxxxx     | Sleep   |  4884 |       | NULL                  |

| 54060 | gnpgenome | xxx.xxx.xxx.xxx:56820     | Xxxxxxxxxxxxxx     | Sleep   |  4815 |       | NULL                  |

| 54111 | gnpgenome | xxxxxxx.biogemma.fr:58832 | Xxxxxxxxxxxxxx     | Sleep   |  4870 |       | NULL                  |

| 54123 | gnpgenome | xxxxxxx.biogemma.fr:58844 | Xxxxxxxxxxxxxx     | Sleep   |  4870 |       | NULL                  |

| 54296 | gnpgenome | xxx.xxx.xxx.xxx:56938     | gnpgenome_sessions | Sleep   |  4812 |       | NULL                  |

| 54297 | gnpgenome | xxx.xxx.xxx.xxx:56939     | gnpgenome_sessions | Sleep   |  4494 |       | NULL                  |

| 54372 | gnpgenome | xxxxxxx.biogemma.fr:36159 | Xxxxxxxxxxxxxx     | Sleep   |  4804 |       | NULL                  |

| 54949 | gnpgenome | xxx.xxx.xxx.xxx:57274     | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 54951 | gnpgenome | xxx.xxx.xxx.xxx:57277     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 54987 | gnpgenome | xxx.xxx.xxx.xxx:57317     | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 54998 | gnpgenome | xxxxxxx.biogemma.fr:36502 | Xxxxxxxxx          | Sleep   |  4729 |       | NULL                  |

| 55033 | gnpgenome | xxxxxxx.biogemma.fr:36535 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55044 | gnpgenome | xxxxxxx.biogemma.fr:36546 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55045 | gnpgenome | xxxxxxx.biogemma.fr:36547 | Xxxxxxxxx          | Sleep   |  4728 |       | NULL                  |

| 55156 | gnpgenome | xxx.xxx.xxx.xxx:57348     | Xxxxxxx            | Sleep   |  2290 |       | NULL                  |

| 55784 | gnpgenome | xxx.xxx.xxx.xxx:57813     | Xxxxxxxxxxx        | Sleep   |  2594 |       | NULL                  |

| 55809 | gnpgenome | xxx.xxx.xxx.xxx:57838     | gnpgenome_sessions | Sleep   |  2556 |       | NULL                  |

| 55812 | gnpgenome | xxx.xxx.xxx.xxx:57841     | ArabidoV9          | Sleep   |  2522 |       | NULL                  |

| 55920 | gnpgenome | xxxxxxx.biogemma.fr:49728 | ArabidoV9          | Sleep   |  2463 |       | NULL                  |

| 55962 | gnpgenome | xxxxxxx.biogemma.fr:49769 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55973 | gnpgenome | xxxxxxx.biogemma.fr:49781 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55974 | gnpgenome | xxxxxxx.biogemma.fr:49782 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 55975 | gnpgenome | xxxxxxx.biogemma.fr:49783 | ArabidoV9          | Sleep   |  2462 |       | NULL                  |

| 56041 | gnpgenome | xxx.xxx.xxx.xxx:57970     | gnpgenome_sessions | Sleep   |  2335 |       | NULL                  |

| 56050 | gnpgenome | xxx.xxx.xxx.xxx:57987     | ArabidoV9          | Sleep   |  2399 |       | NULL                  |

| 56135 | gnpgenome | xxx.xxx.xxx.xxx:58075     | ArabidoV9          | Sleep   |  2302 |       | NULL                  |

| 56136 | gnpgenome | xxx.xxx.xxx.xxx:58076     | Xxxxxxx            | Sleep   |  2302 |       | NULL                  |

| 56171 | gnpgenome | xxx.xxx.xxx.xxx:58112     | Xxxxxxxxx          | Sleep   |  2301 |       | NULL                  |

| 56174 | gnpgenome | xxx.xxx.xxx.xxx:58115     | Xxxxxxxxxxx        | Sleep   |  2301 |       | NULL                  |

| 56175 | gnpgenome | xxx.xxx.xxx.xxx:58116     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56176 | gnpgenome | xxx.xxx.xxx.xxx:58117     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56177 | gnpgenome | xxx.xxx.xxx.xxx:58118     | gnpgenome_sessions | Sleep   |  2301 |       | NULL                  |

| 56226 | gnpgenome | xxx.xxx.xxx.xxx:58131     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56227 | gnpgenome | xxx.xxx.xxx.xxx:58132     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56228 | gnpgenome | xxx.xxx.xxx.xxx:58133     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56229 | gnpgenome | xxx.xxx.xxx.xxx:58134     | Xxxxxxxxxxxxxx     | Sleep   |  2300 |       | NULL                  |

| 56230 | gnpgenome | xxx.xxx.xxx.xxx:58135     | Xxxxxxxxxxx        | Sleep   |  2300 |       | NULL                  |

| 56231 | gnpgenome | xxx.xxx.xxx.xxx:58136     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56232 | gnpgenome | xxx.xxx.xxx.xxx:58137     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56233 | gnpgenome | xxx.xxx.xxx.xxx:58138     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56236 | gnpgenome | xxx.xxx.xxx.xxx:58141     | ArabidoV9          | Sleep   |  2300 |       | NULL                  |

| 56237 | gnpgenome | xxx.xxx.xxx.xxx:58142     | Xxxxxxx            | Sleep   |  2300 |       | NULL                  |

| 56238 | gnpgenome | xxx.xxx.xxx.xxx:58143     | Xxxxxxxxx          | Sleep   |  2300 |       | NULL                  |

| 56239 | gnpgenome | xxx.xxx.xxx.xxx:58144     | Xxxxxxxxxxx        | Sleep   |  2300 |       | NULL                  |

| 56240 | gnpgenome | xxx.xxx.xxx.xxx:58145     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56241 | gnpgenome | xxx.xxx.xxx.xxx:58146     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56242 | gnpgenome | xxx.xxx.xxx.xxx:58147     | gnpgenome_sessions | Sleep   |  2300 |       | NULL                  |

| 56273 | gnpgenome | xxx.xxx.xxx.xxx:58178     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56274 | gnpgenome | xxx.xxx.xxx.xxx:58179     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56275 | gnpgenome | xxx.xxx.xxx.xxx:58180     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56276 | gnpgenome | xxx.xxx.xxx.xxx:58181     | Xxxxxxxxxxxxxx     | Sleep   |  2299 |       | NULL                  |

| 56277 | gnpgenome | xxx.xxx.xxx.xxx:58182     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56278 | gnpgenome | xxx.xxx.xxx.xxx:58183     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56279 | gnpgenome | xxx.xxx.xxx.xxx:58184     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56280 | gnpgenome | xxx.xxx.xxx.xxx:58185     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56317 | gnpgenome | xxx.xxx.xxx.xxx:58210     | ArabidoV9          | Sleep   |  2299 |       | NULL                  |

| 56318 | gnpgenome | xxx.xxx.xxx.xxx:58211     | ArabidoV9          | Sleep   |  2299 |       | NULL                  |

| 56322 | gnpgenome | xxx.xxx.xxx.xxx:58212     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56323 | gnpgenome | xxx.xxx.xxx.xxx:58213     | Xxxxxxx            | Sleep   |  2299 |       | NULL                  |

| 56325 | gnpgenome | xxx.xxx.xxx.xxx:58214     | Xxxxxxxxx          | Sleep   |  2299 |       | NULL                  |

| 56326 | gnpgenome | xxx.xxx.xxx.xxx:58215     | Xxxxxxxxx          | Sleep   |  2299 |       | NULL                  |

| 56327 | gnpgenome | xxx.xxx.xxx.xxx:58216     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56328 | gnpgenome | xxx.xxx.xxx.xxx:58217     | Xxxxxxxxxxx        | Sleep   |  2299 |       | NULL                  |

| 56329 | gnpgenome | xxx.xxx.xxx.xxx:58218     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56330 | gnpgenome | xxx.xxx.xxx.xxx:58219     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56331 | gnpgenome | xxx.xxx.xxx.xxx:58220     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56332 | gnpgenome | xxx.xxx.xxx.xxx:58221     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56333 | gnpgenome | xxx.xxx.xxx.xxx:58222     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56334 | gnpgenome | xxx.xxx.xxx.xxx:58223     | gnpgenome_sessions | Sleep   |  2299 |       | NULL                  |

| 56340 | gnpgenome | xxx.xxx.xxx.xxx:58229     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56341 | gnpgenome | xxx.xxx.xxx.xxx:58230     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56342 | gnpgenome | xxx.xxx.xxx.xxx:58231     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56343 | gnpgenome | xxx.xxx.xxx.xxx:58232     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56345 | gnpgenome | xxx.xxx.xxx.xxx:58234     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56346 | gnpgenome | xxx.xxx.xxx.xxx:58235     | Xxxxxxx            | Sleep   |  2298 |       | NULL                  |

| 56347 | gnpgenome | xxx.xxx.xxx.xxx:58236     | Xxxxxxxxxxxxxx     | Sleep   |  2298 |       | NULL                  |

| 56348 | gnpgenome | xxx.xxx.xxx.xxx:58237     | Xxxxxxxxxxxxxx     | Sleep   |  2298 |       | NULL                  |

| 56349 | gnpgenome | xxx.xxx.xxx.xxx:58238     | Xxxxxxxxxxx        | Sleep   |  2298 |       | NULL                  |

| 56350 | gnpgenome | xxx.xxx.xxx.xxx:58239     | Xxxxxxxxxxx        | Sleep   |  2298 |       | NULL                  |

| 56351 | gnpgenome | xxx.xxx.xxx.xxx:58240     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56352 | gnpgenome | xxx.xxx.xxx.xxx:58241     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56353 | gnpgenome | xxx.xxx.xxx.xxx:58242     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56354 | gnpgenome | xxx.xxx.xxx.xxx:58243     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56355 | gnpgenome | xxx.xxx.xxx.xxx:58244     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56357 | gnpgenome | xxx.xxx.xxx.xxx:58246     | gnpgenome_sessions | Sleep   |  2298 |       | NULL                  |

| 56411 | gnpgenome | xxx.xxx.xxx.xxx:58276     | Xxxxxxx            | Sleep   |  2278 |       | NULL                  |

| 56413 | gnpgenome | xxx.xxx.xxx.xxx:58279     | gnpgenome_sessions | Sleep   |  2020 |       | NULL                  |

| 56423 | gnpgenome | xxx.xxx.xxx.xxx:58289     | Xxxxxxx            | Sleep   |  2218 |       | NULL                  |

| 56440 | gnpgenome | xxx.xxx.xxx.xxx:58305     | ArabidoV9          | Sleep   |  2292 |       | NULL                  |

| 56442 | gnpgenome | xxx.xxx.xxx.xxx:58307     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56443 | gnpgenome | xxx.xxx.xxx.xxx:58308     | Xxxxxxxxx          | Sleep   |  2292 |       | NULL                  |

| 56448 | gnpgenome | xxx.xxx.xxx.xxx:58309     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56449 | gnpgenome | xxx.xxx.xxx.xxx:58310     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56450 | gnpgenome | xxx.xxx.xxx.xxx:58311     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56452 | gnpgenome | xxx.xxx.xxx.xxx:58313     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56477 | gnpgenome | xxx.xxx.xxx.xxx:58319     | ArabidoV9          | Sleep   |  2292 |       | NULL                  |

| 56478 | gnpgenome | xxx.xxx.xxx.xxx:58320     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56479 | gnpgenome | xxx.xxx.xxx.xxx:58321     | Xxxxxxxxx          | Sleep   |  2292 |       | NULL                  |

| 56480 | gnpgenome | xxx.xxx.xxx.xxx:58322     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56481 | gnpgenome | xxx.xxx.xxx.xxx:58323     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56482 | gnpgenome | xxx.xxx.xxx.xxx:58324     | Xxxxxxx            | Sleep   |  2292 |       | NULL                  |

| 56483 | gnpgenome | xxx.xxx.xxx.xxx:58325     | Xxxxxxxxxxxxxx     | Sleep   |  2292 |       | NULL                  |

| 56485 | gnpgenome | xxx.xxx.xxx.xxx:58327     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56486 | gnpgenome | xxx.xxx.xxx.xxx:58328     | Xxxxxxxxxxx        | Sleep   |  2292 |       | NULL                  |

| 56487 | gnpgenome | xxx.xxx.xxx.xxx:58329     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56488 | gnpgenome | xxx.xxx.xxx.xxx:58330     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56489 | gnpgenome | xxx.xxx.xxx.xxx:58331     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56490 | gnpgenome | xxx.xxx.xxx.xxx:58332     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56491 | gnpgenome | xxx.xxx.xxx.xxx:58333     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56492 | gnpgenome | xxx.xxx.xxx.xxx:58334     | gnpgenome_sessions | Sleep   |  2292 |       | NULL                  |

| 56521 | gnpgenome | xxx.xxx.xxx.xxx:58363     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56531 | gnpgenome | xxx.xxx.xxx.xxx:58374     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56532 | gnpgenome | xxx.xxx.xxx.xxx:58375     | Xxxxxxx            | Sleep   |  2291 |       | NULL                  |

| 56534 | gnpgenome | xxx.xxx.xxx.xxx:58377     | Xxxxxxxxxxxxxx     | Sleep   |  2291 |       | NULL                  |

| 56536 | gnpgenome | xxx.xxx.xxx.xxx:58379     | Xxxxxxxxxxx        | Sleep   |  2291 |       | NULL                  |

| 56537 | gnpgenome | xxx.xxx.xxx.xxx:58380     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56539 | gnpgenome | xxx.xxx.xxx.xxx:58381     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56542 | gnpgenome | xxx.xxx.xxx.xxx:58382     | gnpgenome_sessions | Sleep   |  2291 |       | NULL                  |

| 56600 | gnpgenome | xxx.xxx.xxx.xxx:58421     | ArabidoV9          | Sleep   |  2290 |       | NULL                  |

| 56631 | gnpgenome | xxx.xxx.xxx.xxx:58438     | Xxxxxxx            | Sleep   |  2289 |       | NULL                  |

| 56658 | gnpgenome | xxx.xxx.xxx.xxx:58447     | Xxxxxxxxx          | Sleep   |  2285 |       | NULL                  |

| 56671 | gnpgenome | xxx.xxx.xxx.xxx:58451     | Xxxxxxxxxxx        | Sleep   |  2283 |       | NULL                  |

| 56683 | gnpgenome | xxx.xxx.xxx.xxx:58453     | gnpgenome_sessions | Sleep   |  2282 |       | NULL                  |

| 56685 | gnpgenome | xxx.xxx.xxx.xxx:58455     | gnpgenome_sessions | Sleep   |  2281 |       | NULL                  |

| 56690 | gnpgenome | xxx.xxx.xxx.xxx:58460     | gnpgenome_sessions | Sleep   |  2279 |       | NULL                  |

| 56733 | gnpgenome | xxx.xxx.xxx.xxx:58494     | gnpgenome_sessions | Sleep   |  2262 |       | NULL                  |

| 56738 | gnpgenome | xxx.xxx.xxx.xxx:58499     | gnpgenome_sessions | Sleep   |  2262 |       | NULL                  |

| 56750 | gnpgenome | xxx.xxx.xxx.xxx:58511     | gnpgenome_sessions | Sleep   |  2263 |       | NULL                  |

| 56753 | gnpgenome | xxx.xxx.xxx.xxx:58516     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 56761 | gnpgenome | xxx.xxx.xxx.xxx:58518     | gnpgenome_sessions | Sleep   |  2254 |       | NULL                  |

| 56769 | gnpgenome | xxx.xxx.xxx.xxx:58520     | Xxxxxxxxx          | Sleep   |  2262 |       | NULL                  |

| 56770 | gnpgenome | xxx.xxx.xxx.xxx:58521     | Xxxxxxxxxxxxxx     | Sleep   |  2262 |       | NULL                  |

| 56771 | gnpgenome | xxx.xxx.xxx.xxx:58522     | Xxxxxxxxxxx        | Sleep   |  2262 |       | NULL                  |

| 56778 | gnpgenome | xxx.xxx.xxx.xxx:58523     | Xxxxxxx            | Sleep   |  2262 |       | NULL                  |

| 56779 | gnpgenome | xxx.xxx.xxx.xxx:58524     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56780 | gnpgenome | xxx.xxx.xxx.xxx:58525     | ArabidoV9          | Sleep   |  2261 |       | NULL                  |

| 56781 | gnpgenome | xxx.xxx.xxx.xxx:58526     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56782 | gnpgenome | xxx.xxx.xxx.xxx:58527     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56783 | gnpgenome | xxx.xxx.xxx.xxx:58528     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56790 | gnpgenome | xxx.xxx.xxx.xxx:58529     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56791 | gnpgenome | xxx.xxx.xxx.xxx:58530     | Xxxxxxxxx          | Sleep   |  2261 |       | NULL                  |

| 56792 | gnpgenome | xxx.xxx.xxx.xxx:58531     | Xxxxxxxxxxxxxx     | Sleep   |  2261 |       | NULL                  |

| 56793 | gnpgenome | xxx.xxx.xxx.xxx:58532     | Xxxxxxxxxxx        | Sleep   |  2261 |       | NULL                  |

| 56795 | gnpgenome | xxx.xxx.xxx.xxx:58534     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56796 | gnpgenome | xxx.xxx.xxx.xxx:58535     | Xxxxxxx            | Sleep   |  2261 |       | NULL                  |

| 56797 | gnpgenome | xxx.xxx.xxx.xxx:58536     | ArabidoV9          | Sleep   |  2261 |       | NULL                  |

| 56798 | gnpgenome | xxx.xxx.xxx.xxx:58537     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56799 | gnpgenome | xxx.xxx.xxx.xxx:58538     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56800 | gnpgenome | xxx.xxx.xxx.xxx:58539     | gnpgenome_sessions | Sleep   |  2261 |       | NULL                  |

| 56827 | gnpgenome | xxx.xxx.xxx.xxx:58560     | Xxxxxxx            | Sleep   |  2241 |       | NULL                  |

| 56829 | gnpgenome | xxx.xxx.xxx.xxx:58563     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57019 | gnpgenome | xxx.xxx.xxx.xxx:58642     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57021 | gnpgenome | xxx.xxx.xxx.xxx:58644     | gnpgenome_sessions | Sleep   |  2171 |       | NULL                  |

| 57043 | gnpgenome | xxx.xxx.xxx.xxx:58672     | gnpgenome_sessions | Sleep   |  2020 |       | NULL                  |

| 57121 | gnpgenome | xxx.xxx.xxx.xxx:58676     | Xxxxxxx            | Sleep   |   542 |       | NULL                  |

| 57144 | gnpgenome | xxxxxxx.biogemma.fr:43501 | NULL               | Query   |     0 | NULL  | show full processlist |

+-------+-----------+---------------------------+--------------------+---------+-------+-------+-----------------------+
177 rows in set (0.00 sec)


Fred

------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>


------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Lincoln Stein
In reply to this post by Jim Hu
Hi Folks,

With respect to the FastCGI errors, I have tracked the problem down to the following sequence of events:
  1. The FastCGI executive module launches a fresh gbrowse instance.
  2. GBrowse tries to load its default database into memory.
  3. If the default database takes more than 3s to load, then FastCGI times it out.
  4. FastCGI launches a new instance of GBrowse.
  5. GBrowse tries to load its default database into memory.
  6. FastCGI times the new instance out.
  7. Repeat 4-6 indefinitely.
Note that this only happens for databases that are slow to load, typically in-memory databases. For example, the full tutorial database takes ~8s to load on my machine (unreasonably slow for reasons that are unclear to me: I am going to start debugging bioperl next). Once the database is loaded, however, all subsequent accesses are fast.

Here's what to do to fix the problem:
  • for mod_fcgid, add the following directive to /etc/apache2/conf.d/gbrowse2 (in the same section as FcgidIOTimeout)
    •   FcgidConnectTimeout 30
  • for mod_fastcgi, change FastCGIConfig to 
    • FastCGIConfig -startDelay 30 -appConnTimeout 30 -idle-timeout 600 -maxClassProcesses 20  -initial-env GBROWSE_CONF=/etc/gbrowse2
You may also wish to experiment with the current GitHub "master" branch, where I refactored the code so that all default databases are loaded at the time of GBrowse initialization rather than when first needed. This slows down startup under FastCGI, but gives more predictable (and subjectively faster) performance once the processes are loaded and running.

Lincoln


On Mon, Jun 24, 2013 at 11:01 AM, Jim Hu <[hidden email]> wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse



--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Lincoln Stein
Another trick I just discovered: You can speed up the initial loading by about 8 fold by using a tmpfs mounted on /tmp. 

Lincoln


On Tue, Jul 9, 2013 at 5:08 PM, Lincoln Stein <[hidden email]> wrote:
Hi Folks,

With respect to the FastCGI errors, I have tracked the problem down to the following sequence of events:
  1. The FastCGI executive module launches a fresh gbrowse instance.
  2. GBrowse tries to load its default database into memory.
  3. If the default database takes more than 3s to load, then FastCGI times it out.
  4. FastCGI launches a new instance of GBrowse.
  5. GBrowse tries to load its default database into memory.
  6. FastCGI times the new instance out.
  7. Repeat 4-6 indefinitely.
Note that this only happens for databases that are slow to load, typically in-memory databases. For example, the full tutorial database takes ~8s to load on my machine (unreasonably slow for reasons that are unclear to me: I am going to start debugging bioperl next). Once the database is loaded, however, all subsequent accesses are fast.

Here's what to do to fix the problem:
  • for mod_fcgid, add the following directive to /etc/apache2/conf.d/gbrowse2 (in the same section as FcgidIOTimeout)
    •   FcgidConnectTimeout 30
  • for mod_fastcgi, change FastCGIConfig to 
    • FastCGIConfig -startDelay 30 -appConnTimeout 30 -idle-timeout 600 -maxClassProcesses 20  -initial-env GBROWSE_CONF=/etc/gbrowse2
You may also wish to experiment with the current GitHub "master" branch, where I refactored the code so that all default databases are loaded at the time of GBrowse initialization rather than when first needed. This slows down startup under FastCGI, but gives more predictable (and subjectively faster) performance once the processes are loaded and running.

Lincoln


On Mon, Jun 24, 2013 at 11:01 AM, Jim Hu <[hidden email]> wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse



--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>



--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Martin Mokrejs
In reply to this post by Lincoln Stein
Hi,

Lincoln Stein wrote:

> Hi Folks,
>
> With respect to the FastCGI errors, I have tracked the problem down to the following sequence of events:
>
>  1. The FastCGI executive module launches a fresh gbrowse instance.
>  2. GBrowse tries to load its default database into memory.
>  3. If the default database takes more than 3s to load, then FastCGI times it out.
>  4. FastCGI launches a new instance of GBrowse.
>  5. GBrowse tries to load its default database into memory.
>  6. FastCGI times the new instance out.
>  7. Repeat 4-6 indefinitely.
>
> Note that this only happens for databases that are slow to load, typically in-memory databases. For example, the full tutorial database takes ~8s to load on my machine (unreasonably slow for reasons that are unclear to me: I am going to start debugging bioperl next). Once the database is loaded, however, all subsequent accesses are fast.
>
> Here's what to do to fix the problem:
>
>   * for mod_fcgid, add the following directive to /etc/apache2/conf.d/gbrowse2 (in the same section as *FcgidIOTimeout*)
>       o   *FcgidConnectTimeout 30*
>   * for mod_fastcgi, change *FastCGIConfig* to
>       o *FastCGIConfig *-startDelay 30 -appConnTimeout 30 -idle-timeout 600 -maxClassProcesses 20  -initial-env GBROWSE_CONF=/etc/gbrowse2

Hmm, I would call this a workaround instead. Why isn't there something like a lock
in Gbrowse so that multiple connections to mysql wouldn't be established? This
will definitely trick again some user or sysadmin. At least if gbrowse would test
for a timeout value and complain and exit if it would be too short.

In my eyes gbrowse is full of such tricks and reading this lists only ensures me that
the code should be more careful about what user has configured, what is and what is not
available, and ... provide helpful error messages. I don't have the time to contribute
some code in this regard but anything leading to perl unitialized values should be
fixed. At least, I could fish some emails from the archives and emphasize the worst
examples. Or the other way around, jsut take a working setup and start to screw file/dir
permission, place bad filenames/dbnames/dirnames into config, set screw variable names.
It will all generate all kinds of funny messages to apache logs and that should fairly
doable to put a couple of roadblocks along the way.

This would only be helpful to everybody asking here on this list for help with some weird
messages in apache logs.

Just my 2cents, ;-)
Martin

------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Prem Anand
In reply to this post by Lincoln Stein
Hi Lincoln

I did NOT notice the sleeping mysql process with mod_fcgid when we tried the gbrowse vm (2.52) (http://gmod.org/wiki/GBrowse2_VMs).  However, I did notice it when we installed gbrowse (2.54) in our vbox. Comparing both the machines, one noticeable difference is with the apache compiled in modules.
Gbrowse vm has worker.c and in our vbox we had the prefork.c (where the mysql sleep processes were  noticed)  

Wondering if the MPM (prefork) is causing the issue or the different gbrowse versions.

Note: Updating FcgidConnectTimeout to 30 seconds didn't make any difference to me(in Ubuntu 12.04). The mod_fcgid doc (http://httpd.apache.org/mod_fcgid/mod/mod_fcgid.html#fcgidconnecttimeout) says that this directive is not respected on Unix.

FcgidConnectTimeout Directive 
This is the maximum period of time the module will wait while trying to connect to a FastCGI application on Windows. (This directive is not respected on Unix, where AF_UNIX defaults will apply.)

Thanks
Prem





On Tue, Jul 9, 2013 at 9:08 PM, Lincoln Stein <[hidden email]> wrote:
Hi Folks,

With respect to the FastCGI errors, I have tracked the problem down to the following sequence of events:
  1. The FastCGI executive module launches a fresh gbrowse instance.
  2. GBrowse tries to load its default database into memory.
  3. If the default database takes more than 3s to load, then FastCGI times it out.
  4. FastCGI launches a new instance of GBrowse.
  5. GBrowse tries to load its default database into memory.
  6. FastCGI times the new instance out.
  7. Repeat 4-6 indefinitely.
Note that this only happens for databases that are slow to load, typically in-memory databases. For example, the full tutorial database takes ~8s to load on my machine (unreasonably slow for reasons that are unclear to me: I am going to start debugging bioperl next). Once the database is loaded, however, all subsequent accesses are fast.

Here's what to do to fix the problem:
  • for mod_fcgid, add the following directive to /etc/apache2/conf.d/gbrowse2 (in the same section as FcgidIOTimeout)
    •   FcgidConnectTimeout 30
  • for mod_fastcgi, change FastCGIConfig to 
    • FastCGIConfig -startDelay 30 -appConnTimeout 30 -idle-timeout 600 -maxClassProcesses 20  -initial-env GBROWSE_CONF=/etc/gbrowse2
You may also wish to experiment with the current GitHub "master" branch, where I refactored the code so that all default databases are loaded at the time of GBrowse initialization rather than when first needed. This slows down startup under FastCGI, but gives more predictable (and subjectively faster) performance once the processes are loaded and running.

Lincoln


On Mon, Jun 24, 2013 at 11:01 AM, Jim Hu <[hidden email]> wrote:
I was trying to figure out what makes our gbrowse so slow.  I know that part of it is that we turned off FastCGI due to problems we were having with it on (it didn't seem to work for us with the user login system, among other things).  But it also seems like Bio:::DB::SeqFeature::Store is not returning results as quickly as I would expect. Going into mysql, I see the following:

mysql> show full processlist;
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| Id     | User         | Host      | db              | Command | Time  | State | Info                  |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
| 527845 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527846 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527847 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 527855 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527856 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   730 |       | NULL                  |
| 527862 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527905 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   471 |       | NULL                  |
| 527921 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 527935 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 527936 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 527937 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 527948 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   455 |       | NULL                  |
| 527957 | gbrowse_user | localhost | gb_MG1655       | Sleep   |   541 |       | NULL                  |
| 528041 | gbrowse_user | localhost | gb_BW2952       | Sleep   |  1564 |       | NULL                  |
| 528045 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528046 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12536 |       | NULL                  |
| 528054 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12202 |       | NULL                  |
| 528055 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 528069 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12149 |       | NULL                  |
| 528070 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528078 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12536 |       | NULL                  |
| 528094 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12875 |       | NULL                  |
| 528114 | gbrowse_user | localhost | gb_BW2952       | Sleep   | 12118 |       | NULL                  |
| 528130 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12117 |       | NULL                  |
| 528178 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12353 |       | NULL                  |
| 528210 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12203 |       | NULL                  |
| 528274 | gbrowse_user | localhost | gb_W3110        | Sleep   | 12887 |       | NULL                  |
| 529351 | gbrowse_user | localhost | gb_subtilis_168 | Sleep   | 14637 |       | NULL                  |
| 532286 | jimhu        | localhost | NULL            | Query   |     0 | NULL  | show full processlist |
+--------+--------------+-----------+-----------------+---------+-------+-------+-----------------------+
29 rows in set (0.00 sec)

This was late on a Sunday night. When I reloaded gbrowse in my browser, the number went up to 33 sleeping processes.  From my other experience with Mediawiki, it seems like although MySQL can handle lots of simultaneous connections, the performance takes a hit when there are a lot of them.  So, I'm wondering:

- are these processes sleeping because they are waiting for something, or is this due to failure to close a persistent connection when done?
- do people use the default mysql wait_timeout?
- can this be improved by using memcached? On mediawiki, adding memcached give a very dramatic performance increase.  I'm not sure if this is something that would be added at the level of Gbrowse, Bio::DB::SeqFeature::Store, or DBD::MySQL.

Some of this is probably due to EcoliWiki sending lots of requests for gene thumbnails to gbrowse_img.  I'm going to work on caching the images on the wiki side and see if that helps.

Jim

=====================================
Jim Hu
Professor
Dept. of Biochemistry and Biophysics
2128 TAMU
Texas A&M Univ.
College Station, TX 77843-2128
<a href="tel:979-862-4054" value="+19798624054" target="_blank">979-862-4054



------------------------------------------------------------------------------
This SF.net email is sponsored by Windows:

Build for Windows Store.

http://p.sf.net/sfu/windows-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse



--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
<a href="tel:416%20673-8514" value="+14166738514" target="_blank">416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




--
Favorite Quote:
"We all have two choices: We can make a living or we can design a life."

------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: sleeping mysql processes w/Bio::DB::SeqFeature::Store

Lincoln Stein
In reply to this post by Martin Mokrejs
I have just uploaded GBrowse version 2.55 to CPAN. It contains a bioperl patch that fixes slow loading of multiple GFF3 files, and changes the timeout values for mod_fastcgi and mod_fcgid. I have also noticed that mod_fastcgi now seems to be faster than mod_fcgid when the timeout issue is fixed.

I don't think that the FastCGI issues are related to the sleeping mysql process issue, actually, because this was first reported in the context of running GBrowse in CGI mode. I am starting work on CGI mode now.

Lincoln


On Tue, Jul 9, 2013 at 6:49 PM, Martin Mokrejs <[hidden email]> wrote:
Hi,

Lincoln Stein wrote:
> Hi Folks,
>
> With respect to the FastCGI errors, I have tracked the problem down to the following sequence of events:
>
>  1. The FastCGI executive module launches a fresh gbrowse instance.
>  2. GBrowse tries to load its default database into memory.
>  3. If the default database takes more than 3s to load, then FastCGI times it out.
>  4. FastCGI launches a new instance of GBrowse.
>  5. GBrowse tries to load its default database into memory.
>  6. FastCGI times the new instance out.
>  7. Repeat 4-6 indefinitely.
>
> Note that this only happens for databases that are slow to load, typically in-memory databases. For example, the full tutorial database takes ~8s to load on my machine (unreasonably slow for reasons that are unclear to me: I am going to start debugging bioperl next). Once the database is loaded, however, all subsequent accesses are fast.
>
> Here's what to do to fix the problem:
>
>   * for mod_fcgid, add the following directive to /etc/apache2/conf.d/gbrowse2 (in the same section as *FcgidIOTimeout*)
>       o   *FcgidConnectTimeout 30*
>   * for mod_fastcgi, change *FastCGIConfig* to
>       o *FastCGIConfig *-startDelay 30 -appConnTimeout 30 -idle-timeout 600 -maxClassProcesses 20  -initial-env GBROWSE_CONF=/etc/gbrowse2

Hmm, I would call this a workaround instead. Why isn't there something like a lock
in Gbrowse so that multiple connections to mysql wouldn't be established? This
will definitely trick again some user or sysadmin. At least if gbrowse would test
for a timeout value and complain and exit if it would be too short.

In my eyes gbrowse is full of such tricks and reading this lists only ensures me that
the code should be more careful about what user has configured, what is and what is not
available, and ... provide helpful error messages. I don't have the time to contribute
some code in this regard but anything leading to perl unitialized values should be
fixed. At least, I could fish some emails from the archives and emphasize the worst
examples. Or the other way around, jsut take a working setup and start to screw file/dir
permission, place bad filenames/dbnames/dirnames into config, set screw variable names.
It will all generate all kinds of funny messages to apache logs and that should fairly
doable to put a couple of roadblocks along the way.

This would only be helpful to everybody asking here on this list for help with some weird
messages in apache logs.

Just my 2cents, ;-)
Martin



--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
See everything from the browser to the database with AppDynamics
Get end-to-end visibility with application monitoring from AppDynamics
Isolate bottlenecks and diagnose root cause in seconds.
Start your free trial of AppDynamics Pro today!
http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
12