Any colutions to the GBrowse memory leak problem?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Any colutions to the GBrowse memory leak problem?

Jayaraman, Pushkala

Hello,

I am referring to this particular issue that we seem to be experiencing as well…

http://comments.gmane.org/gmane.science.biology.gmod.gbrowse/7572

 

Our Gbrowse process simply hangs even if we have just three tracks open for a range of 1MB. Reloading the page seems to make it okay and everything renders quickly..

Our Gbrowse process is 2.15.

 

The issue ( follow link) reports that it is some sort of memory leak in gbrowse..

 

Would you guys out there know any solution to this problem.. or a fix that we might have to integrate?

 

 

Pushkala Jayaraman

Programmer/Analyst

Rat Genome Database

Human and Molecular Genetics Center

Medical College of Wisconsin

Email: [hidden email]

Work: 414-955-2229

www.rgd.mcw.edu

 


------------------------------------------------------------------------------
Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation -- Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel
Reply | Threaded
Open this post in threaded view
|

Any solutions to the GBrowse memory leak problem?

Jayaraman, Pushkala

RUN(): POST: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/ action=retrieve_multiple;track_ids

=HumanSyntenyBlock;track_ids=RatSyntenyBlock;track_ids=dbSNP_Track;tk_HumanSyntenyBlock=26117e647b7417d62fcfd434df39df4b;tk_RatSyntenyBlock=9b2c1f8a09b0e005fd06638a9645eb78;tk_db

SNP_Track=a653ea6516e414ecf2af259e986b2442;_= at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Bio/Graphics/Browser2/Render.pm line 201., referer: http://test.rgd.mc

w.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: stderr: [11552] add_user_tracks() at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-, referer: http://test.rgd

.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: stderr: thread-multi/Bio/Graphics/Browser2/Render.pm line 209., referer: http://test.rgd.mcw.edu/fgb2/gbrowse

/rgd_904/

[Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: stderr: [11552] testing for asynchronous event() at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-mult

i/Bio/Graphics/Browser2/Render.pm line 214., referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: stderr: [11552] asynchronous exit at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Bio/Graphics/

Browser2/Render.pm line 216., referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:57:44 2011] [warn] [client 141.106.161.52] mod_fcgid: read data timeout in 40 seconds, referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:57:44 2011] [error] [client 141.106.161.52] Premature end of script headers: gbrowse, referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:57:53 2011] [warn] [client 141.106.161.52] mod_fcgid: read data timeout in 40 seconds, referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:57:53 2011] [error] [client 141.106.161.52] Premature end of script headers: gbrowse_img, referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

 

I see these errors in my error logs.. I see a “read data timeout” .. how do I counter that?

 

Pushkala Jayaraman

Programmer/Analyst

Rat Genome Database

Human and Molecular Genetics Center

Medical College of Wisconsin

Email: [hidden email]

Work: 414-955-2229

www.rgd.mcw.edu

 

From: Jayaraman, Pushkala [mailto:[hidden email]]
Sent: Tuesday, April 19, 2011 11:34 AM
To: [hidden email]; [hidden email]
Subject: [Gmod-gbrowse] Any colutions to the GBrowse memory leak problem?

 

Hello,

I am referring to this particular issue that we seem to be experiencing as well…

http://comments.gmane.org/gmane.science.biology.gmod.gbrowse/7572

 

Our Gbrowse process simply hangs even if we have just three tracks open for a range of 1MB. Reloading the page seems to make it okay and everything renders quickly..

Our Gbrowse process is 2.15.

 

The issue ( follow link) reports that it is some sort of memory leak in gbrowse..

 

Would you guys out there know any solution to this problem.. or a fix that we might have to integrate?

 

 

Pushkala Jayaraman

Programmer/Analyst

Rat Genome Database

Human and Molecular Genetics Center

Medical College of Wisconsin

Email: [hidden email]

Work: 414-955-2229

www.rgd.mcw.edu

 


------------------------------------------------------------------------------
Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation -- Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-gbrowse] Any solutions to the GBrowse memory leak problem?

Richard Hayes
Hi,

That was my original bug report. We're using mod_perl 2.0.4 on Apache 2.2.9, perl 5.10.0.

Consistently on our servers, it looks like gbrowse spawned child processes periodically never finish. Our typical apache processes run as:

/usr/sbin/apache2 -f /scratch/www-data/apache2/browser/apache2.conf -k start

whereas long running processes that eventually consume multiple gigs of RAM and >50% CPU show up in the process list as:

/scratch/www-data -f /scratch/www-data/apache2/browser/apache2.conf -k start

When I run strace against one of these runaway processes, this occasionally reports a constant stream of mysql queries against our Bio::SeqFeature::Store database running select statements for subfeatures. We've had to use Apache:Resource in our perl.conf to kill any process consuming more than 5 Gb of RAM:
   PerlSetEnv PERL_RLIMIT_DATA 1024:2048
   PerlSetEnv PERL_RLIMIT_AS   5000:6000
   PerlChildInitHandler Apache2::Resource
and also have a cron job to restart/cycle Apache twice a day, but these are clearly bandaids over a larger problem.

I am in the process of upgrading to BioPerl 1.6.9 and the latest Gbrowse 2 build on a test machine to see if those cumulative bug fixes help. (We're still on Gbrowse 2.16).

Also, I've seen a few reports of potential memory leaks in either DBI or DBD::mysql that could be the culprit, although this will require more testing.

https://rt.cpan.org/Public/Bug/Display.html?id=67435
https://rt.cpan.org/Public/Bug/Display.html?id=66596

Any and all comments welcome, thanks!

--
Richard D. Hayes, Ph.D.
Joint Genome Institute / Lawrence Berkeley National Lab
http://www.phytozome.net

On Tue, Apr 19, 2011 at 10:05 AM, Jayaraman, Pushkala <[hidden email]> wrote:

RUN(): POST: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/ action=retrieve_multiple;track_ids

=HumanSyntenyBlock;track_ids=RatSyntenyBlock;track_ids=dbSNP_Track;tk_HumanSyntenyBlock=26117e647b7417d62fcfd434df39df4b;tk_RatSyntenyBlock=9b2c1f8a09b0e005fd06638a9645eb78;tk_db

SNP_Track=a653ea6516e414ecf2af259e986b2442;_= at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Bio/Graphics/Browser2/Render.pm line 201., referer: http://test.rgd.mc

w.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: stderr: [11552] add_user_tracks() at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-, referer: http://test.rgd

.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: stderr: thread-multi/Bio/Graphics/Browser2/Render.pm line 209., referer: http://test.rgd.mcw.edu/fgb2/gbrowse

/rgd_904/

[Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: stderr: [11552] testing for asynchronous event() at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-mult

i/Bio/Graphics/Browser2/Render.pm line 214., referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:56:46 2011] [warn] [client 141.106.161.52] mod_fcgid: stderr: [11552] asynchronous exit at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/Bio/Graphics/

Browser2/Render.pm line 216., referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:57:44 2011] [warn] [client 141.106.161.52] mod_fcgid: read data timeout in 40 seconds, referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:57:44 2011] [error] [client 141.106.161.52] Premature end of script headers: gbrowse, referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:57:53 2011] [warn] [client 141.106.161.52] mod_fcgid: read data timeout in 40 seconds, referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

[Tue Apr 19 11:57:53 2011] [error] [client 141.106.161.52] Premature end of script headers: gbrowse_img, referer: http://test.rgd.mcw.edu/fgb2/gbrowse/rgd_904/

 

I see these errors in my error logs.. I see a “read data timeout” .. how do I counter that?

 

Pushkala Jayaraman

Programmer/Analyst

Rat Genome Database

Human and Molecular Genetics Center

Medical College of Wisconsin

Email: [hidden email]

Work: <a href="tel:414-955-2229" value="+14149552229" target="_blank">414-955-2229

www.rgd.mcw.edu

 

From: Jayaraman, Pushkala [mailto:[hidden email]]
Sent: Tuesday, April 19, 2011 11:34 AM
To: [hidden email]; [hidden email]
Subject: [Gmod-gbrowse] Any colutions to the GBrowse memory leak problem?

 

Hello,

I am referring to this particular issue that we seem to be experiencing as well…

http://comments.gmane.org/gmane.science.biology.gmod.gbrowse/7572

 

Our Gbrowse process simply hangs even if we have just three tracks open for a range of 1MB. Reloading the page seems to make it okay and everything renders quickly..

Our Gbrowse process is 2.15.

 

The issue ( follow link) reports that it is some sort of memory leak in gbrowse..

 

Would you guys out there know any solution to this problem.. or a fix that we might have to integrate?

 

 

Pushkala Jayaraman

Programmer/Analyst

Rat Genome Database

Human and Molecular Genetics Center

Medical College of Wisconsin

Email: [hidden email]

Work: <a href="tel:414-955-2229" value="+14149552229" target="_blank">414-955-2229

www.rgd.mcw.edu

 


------------------------------------------------------------------------------
Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation -- Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


------------------------------------------------------------------------------
Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation -- Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel