Multiple instances of apache2 processes and MPMs

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Multiple instances of apache2 processes and MPMs

Prem Anand
Hi

Recently I installed GBrowse2 in a clean vm but had issues similar to
the one mentioned by Scooter earlier, multiple instances of apache2
processes hanging around and consuming lots of memory.
http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel

After comparing with other vms where we did not have this issue,
realized that, this issue was there only where the mpm_worker_module
was enabled.  If anyone else has had this issue, can try this.
apache2ctl -M ( will list the modules enabled)

Check in that list if mpm_workder_module is enabled. If so, then
install mpm_prefork_module,
sudo aptitude install apache2-mpm-prefork

and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.

Then again try gbrowse2, this issue would have probably gone.  This
worked for me though.

Thanks
Prem

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Scooter Willis-2
I have been working on the apache config settings to "tune" apache2 to not run out of memory. I think I am making progress and will send a summary when I stop running out of memory. The key appears to limit the number of processes apache2 will start where the default is 16 and not listed in the config file. I have ir currently set to 2 processes because each process consumes 1.7 gb of memory. The other option is to force apache2 to dump the process after x number of requests to avoid memory usage increasing from dirty memory. Apache2 will then spawn a new process.

The values need to balance memory, processors, and number of concurrent users.

Glad to see I am not the only person with this problem and another possible solution.

Thanks

Scooter

----- Reply message -----
From: "Prem Anand" <[hidden email]>
Date: Fri, Jul 9, 2010 9:51 am
Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
To: "[hidden email]" <[hidden email]>

Hi

Recently I installed GBrowse2 in a clean vm but had issues similar to
the one mentioned by Scooter earlier, multiple instances of apache2
processes hanging around and consuming lots of memory.
http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel

After comparing with other vms where we did not have this issue,
realized that, this issue was there only where the mpm_worker_module
was enabled.  If anyone else has had this issue, can try this.
apache2ctl -M ( will list the modules enabled)

Check in that list if mpm_workder_module is enabled. If so, then
install mpm_prefork_module,
sudo aptitude install apache2-mpm-prefork

and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.

Then again try gbrowse2, this issue would have probably gone.  This
worked for me though.

Thanks
Prem

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Scooter Willis-2
<base href="x-msg://74/">

The "tuning" only appears to delay the time until I run out of memory.

I followed Prem's advice and installed the prefork module. Initial testing after loading a couple scaffolds and doing some searches loads about 4GB+ of memory where I am using the following modified settings. The default prefork settings quickly got to 8GB of memory. In my configuration I am loading a fungal genome into memory. Does anyone have modified prefork_module settings in apache2.conf?

<IfModule mpm_prefork_module>
    StartServers          2
    MinSpareServers       2
    MaxSpareServers       2
    MaxClients           10
    MaxRequestsPerChild 1000
</IfModule>


On Jul 9, 2010, at 10:22 AM, Scooter Willis wrote:

I have been working on the apache config settings to "tune" apache2 to not run out of memory. I think I am making progress and will send a summary when I stop running out of memory. The key appears to limit the number of processes apache2 will start where the default is 16 and not listed in the config file. I have ir currently set to 2 processes because each process consumes 1.7 gb of memory. The other option is to force apache2 to dump the process after x number of requests to avoid memory usage increasing from dirty memory. Apache2 will then spawn a new process.

The values need to balance memory, processors, and number of concurrent users.

Glad to see I am not the only person with this problem and another possible solution.

Thanks

Scooter

----- Reply message -----
From: "Prem Anand" <[hidden email]>
Date: Fri, Jul 9, 2010 9:51 am
Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
To: "[hidden email]" <[hidden email]>

Hi

Recently I installed GBrowse2 in a clean vm but had issues similar to
the one mentioned by Scooter earlier, multiple instances of apache2
processes hanging around and consuming lots of memory.
http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel

After comparing with other vms where we did not have this issue,
realized that, this issue was there only where the mpm_worker_module
was enabled.  If anyone else has had this issue, can try this.
apache2ctl -M ( will list the modules enabled)

Check in that list if mpm_workder_module is enabled. If so, then
install mpm_prefork_module,
sudo aptitude install apache2-mpm-prefork

and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.

Then again try gbrowse2, this issue would have probably gone.  This
worked for me though.

Thanks
Prem

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
<ATT00001..txt><ATT00002..txt>


------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Prem Anand
HI Scooter

I have been using this settings for my vm and so far had no issues.
But I have tested only with the example yeast databases on
gbrowse-2.12 and  I have disabled mod_perl. Will be trying enabling
mod_perl and will let you know how it goes.

<IfModule mpm_prefork_module>
    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
    MaxClients           15
    MaxRequestsPerChild  2000
</IfModule>

Thanks
Prem

On Mon, Jul 12, 2010 at 2:10 AM, Scooter Willis <[hidden email]> wrote:

>
> The "tuning" only appears to delay the time until I run out of memory.
> I followed Prem's advice and installed the prefork module. Initial testing
> after loading a couple scaffolds and doing some searches loads about 4GB+ of
> memory where I am using the following modified settings. The default prefork
> settings quickly got to 8GB of memory. In my configuration I am loading a
> fungal genome into memory. Does anyone have modified prefork_module settings
> in apache2.conf?
> <IfModule mpm_prefork_module>
>     StartServers          2
>     MinSpareServers       2
>     MaxSpareServers       2
>     MaxClients           10
>     MaxRequestsPerChild 1000
> </IfModule>
>
> On Jul 9, 2010, at 10:22 AM, Scooter Willis wrote:
>
> I have been working on the apache config settings to "tune" apache2 to not
> run out of memory. I think I am making progress and will send a summary when
> I stop running out of memory. The key appears to limit the number of
> processes apache2 will start where the default is 16 and not listed in the
> config file. I have ir currently set to 2 processes because each process
> consumes 1.7 gb of memory. The other option is to force apache2 to dump the
> process after x number of requests to avoid memory usage increasing from
> dirty memory. Apache2 will then spawn a new process.
>
> The values need to balance memory, processors, and number of concurrent
> users.
>
> Glad to see I am not the only person with this problem and another possible
> solution.
>
> Thanks
>
> Scooter
>
> ----- Reply message -----
> From: "Prem Anand" <[hidden email]>
> Date: Fri, Jul 9, 2010 9:51 am
> Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
> To: "[hidden email]"
> <[hidden email]>
>
> Hi
>
> Recently I installed GBrowse2 in a clean vm but had issues similar to
> the one mentioned by Scooter earlier, multiple instances of apache2
> processes hanging around and consuming lots of memory.
> http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
> http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel
>
> After comparing with other vms where we did not have this issue,
> realized that, this issue was there only where the mpm_worker_module
> was enabled.  If anyone else has had this issue, can try this.
> apache2ctl -M ( will list the modules enabled)
>
> Check in that list if mpm_workder_module is enabled. If so, then
> install mpm_prefork_module,
> sudo aptitude install apache2-mpm-prefork
>
> and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.
>
> Then again try gbrowse2, this issue would have probably gone.  This
> worked for me though.
>
> Thanks
> Prem
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Gmod-gbrowse mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
> <ATT00001..txt><ATT00002..txt>
>

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Scooter Willis-2
After making the change to use mpm_prefork_module the memory usage appears to be under control but the problem now is the performance is very slow. It takes a couple minutes to load the page when visiting the first time and then has reasonable perfomance depending on what you select.

It appears the problem could be related to the latest version of apache2 installed on ubuntu where Prem found the same problem. Is it possible to get this on the radar of those with mod_perl expertise and the ability to look under the hood at the problem? Not sure I can do any more "guessing" and the lab is complaining that they can't use it because of the speed.

Thanks

Scooter




On Jul 12, 2010, at 4:37 AM, Prem Anand wrote:

HI Scooter

I have been using this settings for my vm and so far had no issues.
But I have tested only with the example yeast databases on
gbrowse-2.12 and  I have disabled mod_perl. Will be trying enabling
mod_perl and will let you know how it goes.

<IfModule mpm_prefork_module>
   StartServers          5
   MinSpareServers       5
   MaxSpareServers      10
   MaxClients           15
   MaxRequestsPerChild  2000
</IfModule>

Thanks
Prem

On Mon, Jul 12, 2010 at 2:10 AM, Scooter Willis <[hidden email]> wrote:

The "tuning" only appears to delay the time until I run out of memory.
I followed Prem's advice and installed the prefork module. Initial testing
after loading a couple scaffolds and doing some searches loads about 4GB+ of
memory where I am using the following modified settings. The default prefork
settings quickly got to 8GB of memory. In my configuration I am loading a
fungal genome into memory. Does anyone have modified prefork_module settings
in apache2.conf?
<IfModule mpm_prefork_module>
    StartServers          2
    MinSpareServers       2
    MaxSpareServers       2
    MaxClients           10
    MaxRequestsPerChild 1000
</IfModule>

On Jul 9, 2010, at 10:22 AM, Scooter Willis wrote:

I have been working on the apache config settings to "tune" apache2 to not
run out of memory. I think I am making progress and will send a summary when
I stop running out of memory. The key appears to limit the number of
processes apache2 will start where the default is 16 and not listed in the
config file. I have ir currently set to 2 processes because each process
consumes 1.7 gb of memory. The other option is to force apache2 to dump the
process after x number of requests to avoid memory usage increasing from
dirty memory. Apache2 will then spawn a new process.

The values need to balance memory, processors, and number of concurrent
users.

Glad to see I am not the only person with this problem and another possible
solution.

Thanks

Scooter

----- Reply message -----
From: "Prem Anand" <[hidden email]>
Date: Fri, Jul 9, 2010 9:51 am
Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
To: "[hidden email]"
<[hidden email]>
mpm_worker_module

Hi

Recently I installed GBrowse2 in a clean vm but had issues similar to
the one mentioned by Scooter earlier, multiple instances of apache2
processes hanging around and consuming lots of memory.
http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel

After comparing with other vms where we did not have this issue,
realized that, this issue was there only where the mpm_worker_module
was enabled.  If anyone else has had this issue, can try this.
apache2ctl -M ( will list the modules enabled)

Check in that list if mpm_workder_module is enabled. If so, then
install mpm_prefork_module,
sudo aptitude install apache2-mpm-prefork

and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.

Then again try gbrowse2, this issue would have probably gone.  This
worked for me though.

Thanks
Prem

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
<ATT00001..txt><ATT00002..txt>



------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Prem Anand
Hi Scooter

Try to disable mod_perl and enable only FastCGI.  I had some issues
with mod_perl enabled with mpm_prefork_module.
But with FastCGI, the performance is much better.

Thanks
Prem

On Mon, Jul 12, 2010 at 8:36 PM, Scooter Willis <[hidden email]> wrote:

> After making the change to use mpm_prefork_module the memory usage appears
> to be under control but the problem now is the performance is very slow. It
> takes a couple minutes to load the page when visiting the first time and
> then has reasonable perfomance depending on what you select.
> It appears the problem could be related to the latest version of apache2
> installed on ubuntu where Prem found the same problem. Is it possible to get
> this on the radar of those with mod_perl expertise and the ability to look
> under the hood at the problem? Not sure I can do any more "guessing" and the
> lab is complaining that they can't use it because of the speed.
> Thanks
> Scooter
>
>
>
>
> On Jul 12, 2010, at 4:37 AM, Prem Anand wrote:
>
> HI Scooter
>
> I have been using this settings for my vm and so far had no issues.
> But I have tested only with the example yeast databases on
> gbrowse-2.12 and  I have disabled mod_perl. Will be trying enabling
> mod_perl and will let you know how it goes.
>
> <IfModule mpm_prefork_module>
>    StartServers          5
>    MinSpareServers       5
>    MaxSpareServers      10
>    MaxClients           15
>    MaxRequestsPerChild  2000
> </IfModule>
>
> Thanks
> Prem
>
> On Mon, Jul 12, 2010 at 2:10 AM, Scooter Willis <[hidden email]> wrote:
>
> The "tuning" only appears to delay the time until I run out of memory.
>
> I followed Prem's advice and installed the prefork module. Initial testing
>
> after loading a couple scaffolds and doing some searches loads about 4GB+ of
>
> memory where I am using the following modified settings. The default prefork
>
> settings quickly got to 8GB of memory. In my configuration I am loading a
>
> fungal genome into memory. Does anyone have modified prefork_module settings
>
> in apache2.conf?
>
> <IfModule mpm_prefork_module>
>
>     StartServers          2
>
>     MinSpareServers       2
>
>     MaxSpareServers       2
>
>     MaxClients           10
>
>     MaxRequestsPerChild 1000
>
> </IfModule>
>
> On Jul 9, 2010, at 10:22 AM, Scooter Willis wrote:
>
> I have been working on the apache config settings to "tune" apache2 to not
>
> run out of memory. I think I am making progress and will send a summary when
>
> I stop running out of memory. The key appears to limit the number of
>
> processes apache2 will start where the default is 16 and not listed in the
>
> config file. I have ir currently set to 2 processes because each process
>
> consumes 1.7 gb of memory. The other option is to force apache2 to dump the
>
> process after x number of requests to avoid memory usage increasing from
>
> dirty memory. Apache2 will then spawn a new process.
>
> The values need to balance memory, processors, and number of concurrent
>
> users.
>
> Glad to see I am not the only person with this problem and another possible
>
> solution.
>
> Thanks
>
> Scooter
>
> ----- Reply message -----
>
> From: "Prem Anand" <[hidden email]>
>
> Date: Fri, Jul 9, 2010 9:51 am
>
> Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
>
> To: "[hidden email]"
>
> <[hidden email]>
>
> mpm_worker_module
>
> Hi
>
> Recently I installed GBrowse2 in a clean vm but had issues similar to
>
> the one mentioned by Scooter earlier, multiple instances of apache2
>
> processes hanging around and consuming lots of memory.
>
> http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
>
> http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel
>
> After comparing with other vms where we did not have this issue,
>
> realized that, this issue was there only where the mpm_worker_module
>
> was enabled.  If anyone else has had this issue, can try this.
>
> apache2ctl -M ( will list the modules enabled)
>
> Check in that list if mpm_workder_module is enabled. If so, then
>
> install mpm_prefork_module,
>
> sudo aptitude install apache2-mpm-prefork
>
> and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.
>
> Then again try gbrowse2, this issue would have probably gone.  This
>
> worked for me though.
>
> Thanks
>
> Prem
>
> ------------------------------------------------------------------------------
>
> This SF.net email is sponsored by Sprint
>
> What will you do first with EVO, the first 4G phone?
>
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
>
> _______________________________________________
>
> Gmod-gbrowse mailing list
>
> [hidden email]
>
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>
> <ATT00001..txt><ATT00002..txt>
>
>
>

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Scooter Willis-2
In reply to this post by Prem Anand
With fast_cgi I get internal server error for my dataset but works for the yeast demos. So currenlty I dont have the fast cgi option.

Scooter

----- Reply message -----
From: "Prem Anand" <[hidden email]>
Date: Mon, Jul 12, 2010 3:41 pm
Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
To: "Scooter Willis" <[hidden email]>
Cc: "[hidden email]" <[hidden email]>

Hi Scooter

Try to disable mod_perl and enable only FastCGI.  I had some issues
with mod_perl enabled with mpm_prefork_module.
But with FastCGI, the performance is much better.

Thanks
Prem

On Mon, Jul 12, 2010 at 8:36 PM, Scooter Willis <[hidden email]> wrote:
> After making the change to use mpm_prefork_module the memory usage appears
> to be under control but the problem now is the performance is very slow. It
> takes a couple minutes to load the page when visiting the first time and
> then has reasonable perfomance depending on what you select.
> It appears the problem could be related to the latest version of apache2
> installed on ubuntu where Prem found the same problem. Is it possible to get
> this on the radar of those with mod_perl expertise and the ability to look
> under the hood at the problem? Not sure I can do any more "guessing" and the
> lab is complaining that they can't use it because of the speed.
> Thanks
> Scooter
>
>
>
>
> On Jul 12, 2010, at 4:37 AM, Prem Anand wrote:
>
> HI Scooter
>
> I have been using this settings for my vm and so far had no issues.
> But I have tested only with the example yeast databases on
> gbrowse-2.12 and  I have disabled mod_perl. Will be trying enabling
> mod_perl and will let you know how it goes.
>
> <IfModule mpm_prefork_module>
>    StartServers          5
>    MinSpareServers       5
>    MaxSpareServers      10
>    MaxClients           15
>    MaxRequestsPerChild  2000
> </IfModule>
>
> Thanks
> Prem
>
> On Mon, Jul 12, 2010 at 2:10 AM, Scooter Willis <[hidden email]> wrote:
>
> The "tuning" only appears to delay the time until I run out of memory.
>
> I followed Prem's advice and installed the prefork module. Initial testing
>
> after loading a couple scaffolds and doing some searches loads about 4GB+ of
>
> memory where I am using the following modified settings. The default prefork
>
> settings quickly got to 8GB of memory. In my configuration I am loading a
>
> fungal genome into memory. Does anyone have modified prefork_module settings
>
> in apache2.conf?
>
> <IfModule mpm_prefork_module>
>
>     StartServers          2
>
>     MinSpareServers       2
>
>     MaxSpareServers       2
>
>     MaxClients           10
>
>     MaxRequestsPerChild 1000
>
> </IfModule>
>
> On Jul 9, 2010, at 10:22 AM, Scooter Willis wrote:
>
> I have been working on the apache config settings to "tune" apache2 to not
>
> run out of memory. I think I am making progress and will send a summary when
>
> I stop running out of memory. The key appears to limit the number of
>
> processes apache2 will start where the default is 16 and not listed in the
>
> config file. I have ir currently set to 2 processes because each process
>
> consumes 1.7 gb of memory. The other option is to force apache2 to dump the
>
> process after x number of requests to avoid memory usage increasing from
>
> dirty memory. Apache2 will then spawn a new process.
>
> The values need to balance memory, processors, and number of concurrent
>
> users.
>
> Glad to see I am not the only person with this problem and another possible
>
> solution.
>
> Thanks
>
> Scooter
>
> ----- Reply message -----
>
> From: "Prem Anand" <[hidden email]>
>
> Date: Fri, Jul 9, 2010 9:51 am
>
> Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
>
> To: "[hidden email]"
>
> <[hidden email]>
>
> mpm_worker_module
>
> Hi
>
> Recently I installed GBrowse2 in a clean vm but had issues similar to
>
> the one mentioned by Scooter earlier, multiple instances of apache2
>
> processes hanging around and consuming lots of memory.
>
> http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
>
> http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel
>
> After comparing with other vms where we did not have this issue,
>
> realized that, this issue was there only where the mpm_worker_module
>
> was enabled.  If anyone else has had this issue, can try this.
>
> apache2ctl -M ( will list the modules enabled)
>
> Check in that list if mpm_workder_module is enabled. If so, then
>
> install mpm_prefork_module,
>
> sudo aptitude install apache2-mpm-prefork
>
> and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.
>
> Then again try gbrowse2, this issue would have probably gone.  This
>
> worked for me though.
>
> Thanks
>
> Prem
>
> ------------------------------------------------------------------------------
>
> This SF.net email is sponsored by Sprint
>
> What will you do first with EVO, the first 4G phone?
>
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
>
> _______________________________________________
>
> Gmod-gbrowse mailing list
>
> [hidden email]
>
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>
> <ATT00001..txt><ATT00002..txt>
>
>
>

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Lincoln Stein
Hi Scooter,

What error message appears in your log files when you have fast_cgi going?

Lincoln

On Mon, Jul 12, 2010 at 3:52 PM, Scooter Willis <[hidden email]> wrote:
With fast_cgi I get internal server error for my dataset but works for the yeast demos. So currenlty I dont have the fast cgi option.

Scooter

----- Reply message -----
From: "Prem Anand" <[hidden email]>
Date: Mon, Jul 12, 2010 3:41 pm

Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
To: "Scooter Willis" <[hidden email]>
Cc: "[hidden email]" <[hidden email]>

Hi Scooter

Try to disable mod_perl and enable only FastCGI.  I had some issues
with mod_perl enabled with mpm_prefork_module.
But with FastCGI, the performance is much better.

Thanks
Prem

On Mon, Jul 12, 2010 at 8:36 PM, Scooter Willis <[hidden email]> wrote:
> After making the change to use mpm_prefork_module the memory usage appears
> to be under control but the problem now is the performance is very slow. It
> takes a couple minutes to load the page when visiting the first time and
> then has reasonable perfomance depending on what you select.
> It appears the problem could be related to the latest version of apache2
> installed on ubuntu where Prem found the same problem. Is it possible to get
> this on the radar of those with mod_perl expertise and the ability to look
> under the hood at the problem? Not sure I can do any more "guessing" and the
> lab is complaining that they can't use it because of the speed.
> Thanks
> Scooter
>
>
>
>
> On Jul 12, 2010, at 4:37 AM, Prem Anand wrote:
>
> HI Scooter
>
> I have been using this settings for my vm and so far had no issues.
> But I have tested only with the example yeast databases on
> gbrowse-2.12 and  I have disabled mod_perl. Will be trying enabling
> mod_perl and will let you know how it goes.
>
> <IfModule mpm_prefork_module>
>    StartServers          5
>    MinSpareServers       5
>    MaxSpareServers      10
>    MaxClients           15
>    MaxRequestsPerChild  2000
> </IfModule>
>
> Thanks
> Prem
>
> On Mon, Jul 12, 2010 at 2:10 AM, Scooter Willis <[hidden email]> wrote:
>
> The "tuning" only appears to delay the time until I run out of memory.
>
> I followed Prem's advice and installed the prefork module. Initial testing
>
> after loading a couple scaffolds and doing some searches loads about 4GB+ of
>
> memory where I am using the following modified settings. The default prefork
>
> settings quickly got to 8GB of memory. In my configuration I am loading a
>
> fungal genome into memory. Does anyone have modified prefork_module settings
>
> in apache2.conf?
>
> <IfModule mpm_prefork_module>
>
>     StartServers          2
>
>     MinSpareServers       2
>
>     MaxSpareServers       2
>
>     MaxClients           10
>
>     MaxRequestsPerChild 1000
>
> </IfModule>
>
> On Jul 9, 2010, at 10:22 AM, Scooter Willis wrote:
>
> I have been working on the apache config settings to "tune" apache2 to not
>
> run out of memory. I think I am making progress and will send a summary when
>
> I stop running out of memory. The key appears to limit the number of
>
> processes apache2 will start where the default is 16 and not listed in the
>
> config file. I have ir currently set to 2 processes because each process
>
> consumes 1.7 gb of memory. The other option is to force apache2 to dump the
>
> process after x number of requests to avoid memory usage increasing from
>
> dirty memory. Apache2 will then spawn a new process.
>
> The values need to balance memory, processors, and number of concurrent
>
> users.
>
> Glad to see I am not the only person with this problem and another possible
>
> solution.
>
> Thanks
>
> Scooter
>
> ----- Reply message -----
>
> From: "Prem Anand" <[hidden email]>
>
> Date: Fri, Jul 9, 2010 9:51 am
>
> Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
>
> To: "[hidden email]"
>
> <[hidden email]>
>
> mpm_worker_module
>
> Hi
>
> Recently I installed GBrowse2 in a clean vm but had issues similar to
>
> the one mentioned by Scooter earlier, multiple instances of apache2
>
> processes hanging around and consuming lots of memory.
>
> http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
>
> http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel
>
> After comparing with other vms where we did not have this issue,
>
> realized that, this issue was there only where the mpm_worker_module
>
> was enabled.  If anyone else has had this issue, can try this.
>
> apache2ctl -M ( will list the modules enabled)
>
> Check in that list if mpm_workder_module is enabled. If so, then
>
> install mpm_prefork_module,
>
> sudo aptitude install apache2-mpm-prefork
>
> and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.
>
> Then again try gbrowse2, this issue would have probably gone.  This
>
> worked for me though.
>
> Thanks
>
> Prem
>
> ------------------------------------------------------------------------------
>
> This SF.net email is sponsored by Sprint
>
> What will you do first with EVO, the first 4G phone?
>
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
>
> _______________________________________________
>
> Gmod-gbrowse mailing list
>
> [hidden email]
>
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>
> <ATT00001..txt><ATT00002..txt>
>
>
>

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Lincoln Stein
In reply to this post by Scooter Willis-2
apache2-mpm-worker uses a threaded model which doesn't work well with mod_perl. The prefork version + modperl should be nice and fast on Linux but dreadfully slow under windows. Is this a windows installation?

Lincoln

On Mon, Jul 12, 2010 at 3:36 PM, Scooter Willis <[hidden email]> wrote:
After making the change to use mpm_prefork_module the memory usage appears to be under control but the problem now is the performance is very slow. It takes a couple minutes to load the page when visiting the first time and then has reasonable perfomance depending on what you select.

It appears the problem could be related to the latest version of apache2 installed on ubuntu where Prem found the same problem. Is it possible to get this on the radar of those with mod_perl expertise and the ability to look under the hood at the problem? Not sure I can do any more "guessing" and the lab is complaining that they can't use it because of the speed.

Thanks

Scooter




On Jul 12, 2010, at 4:37 AM, Prem Anand wrote:

HI Scooter

I have been using this settings for my vm and so far had no issues.
But I have tested only with the example yeast databases on
gbrowse-2.12 and  I have disabled mod_perl. Will be trying enabling
mod_perl and will let you know how it goes.

<IfModule mpm_prefork_module>
   StartServers          5
   MinSpareServers       5
   MaxSpareServers      10
   MaxClients           15
   MaxRequestsPerChild  2000
</IfModule>

Thanks
Prem

On Mon, Jul 12, 2010 at 2:10 AM, Scooter Willis <[hidden email]> wrote:

The "tuning" only appears to delay the time until I run out of memory.
I followed Prem's advice and installed the prefork module. Initial testing
after loading a couple scaffolds and doing some searches loads about 4GB+ of
memory where I am using the following modified settings. The default prefork
settings quickly got to 8GB of memory. In my configuration I am loading a
fungal genome into memory. Does anyone have modified prefork_module settings
in apache2.conf?
<IfModule mpm_prefork_module>
    StartServers          2
    MinSpareServers       2
    MaxSpareServers       2
    MaxClients           10
    MaxRequestsPerChild 1000
</IfModule>

On Jul 9, 2010, at 10:22 AM, Scooter Willis wrote:

I have been working on the apache config settings to "tune" apache2 to not
run out of memory. I think I am making progress and will send a summary when
I stop running out of memory. The key appears to limit the number of
processes apache2 will start where the default is 16 and not listed in the
config file. I have ir currently set to 2 processes because each process
consumes 1.7 gb of memory. The other option is to force apache2 to dump the
process after x number of requests to avoid memory usage increasing from
dirty memory. Apache2 will then spawn a new process.

The values need to balance memory, processors, and number of concurrent
users.

Glad to see I am not the only person with this problem and another possible
solution.

Thanks

Scooter

----- Reply message -----
From: "Prem Anand" <[hidden email]>
Date: Fri, Jul 9, 2010 9:51 am
Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
To: "[hidden email]"
<[hidden email]>
mpm_worker_module

Hi

Recently I installed GBrowse2 in a clean vm but had issues similar to
the one mentioned by Scooter earlier, multiple instances of apache2
processes hanging around and consuming lots of memory.
http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel

After comparing with other vms where we did not have this issue,
realized that, this issue was there only where the mpm_worker_module
was enabled.  If anyone else has had this issue, can try this.
apache2ctl -M ( will list the modules enabled)

Check in that list if mpm_workder_module is enabled. If so, then
install mpm_prefork_module,
sudo aptitude install apache2-mpm-prefork

and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.

Then again try gbrowse2, this issue would have probably gone.  This
worked for me though.

Thanks
Prem

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
<ATT00001..txt><ATT00002..txt>



------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Scooter Willis-2
In reply to this post by Prem Anand
Lincoln

It is ubuntu 64 bit latest everything. I tried prefork and it doesnt run out of memory but takes forever to return. The fastcgi error was a timeout which I fixed via adjusting parameters but can't recall the other problem. I am at ISMB if you are around to discuss.

Scooter

----- Reply message -----
From: "Lincoln Stein" <[hidden email]>
Date: Mon, Jul 12, 2010 5:10 pm
Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
To: "Scooter Willis" <[hidden email]>
Cc: "Prem Anand" <[hidden email]>, "[hidden email]" <[hidden email]>

apache2-mpm-worker uses a threaded model which doesn't work well with mod_perl. The prefork version + modperl should be nice and fast on Linux but dreadfully slow under windows. Is this a windows installation?

Lincoln

On Mon, Jul 12, 2010 at 3:36 PM, Scooter Willis <[hidden email]> wrote:
After making the change to use mpm_prefork_module the memory usage appears to be under control but the problem now is the performance is very slow. It takes a couple minutes to load the page when visiting the first time and then has reasonable perfomance depending on what you select.

It appears the problem could be related to the latest version of apache2 installed on ubuntu where Prem found the same problem. Is it possible to get this on the radar of those with mod_perl expertise and the ability to look under the hood at the problem? Not sure I can do any more "guessing" and the lab is complaining that they can't use it because of the speed.

Thanks

Scooter




On Jul 12, 2010, at 4:37 AM, Prem Anand wrote:

HI Scooter

I have been using this settings for my vm and so far had no issues.
But I have tested only with the example yeast databases on
gbrowse-2.12 and  I have disabled mod_perl. Will be trying enabling
mod_perl and will let you know how it goes.

<IfModule mpm_prefork_module>
   StartServers          5
   MinSpareServers       5
   MaxSpareServers      10
   MaxClients           15
   MaxRequestsPerChild  2000
</IfModule>

Thanks
Prem

On Mon, Jul 12, 2010 at 2:10 AM, Scooter Willis <[hidden email]> wrote:

The "tuning" only appears to delay the time until I run out of memory.
I followed Prem's advice and installed the prefork module. Initial testing
after loading a couple scaffolds and doing some searches loads about 4GB+ of
memory where I am using the following modified settings. The default prefork
settings quickly got to 8GB of memory. In my configuration I am loading a
fungal genome into memory. Does anyone have modified prefork_module settings
in apache2.conf?
<IfModule mpm_prefork_module>
    StartServers          2
    MinSpareServers       2
    MaxSpareServers       2
    MaxClients           10
    MaxRequestsPerChild 1000
</IfModule>

On Jul 9, 2010, at 10:22 AM, Scooter Willis wrote:

I have been working on the apache config settings to "tune" apache2 to not
run out of memory. I think I am making progress and will send a summary when
I stop running out of memory. The key appears to limit the number of
processes apache2 will start where the default is 16 and not listed in the
config file. I have ir currently set to 2 processes because each process
consumes 1.7 gb of memory. The other option is to force apache2 to dump the
process after x number of requests to avoid memory usage increasing from
dirty memory. Apache2 will then spawn a new process.

The values need to balance memory, processors, and number of concurrent
users.

Glad to see I am not the only person with this problem and another possible
solution.

Thanks

Scooter

----- Reply message -----
From: "Prem Anand" <[hidden email]>
Date: Fri, Jul 9, 2010 9:51 am
Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
To: "[hidden email]"
<[hidden email]>
mpm_worker_module

Hi

Recently I installed GBrowse2 in a clean vm but had issues similar to
the one mentioned by Scooter earlier, multiple instances of apache2
processes hanging around and consuming lots of memory.
http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel

After comparing with other vms where we did not have this issue,
realized that, this issue was there only where the mpm_worker_module
was enabled.  If anyone else has had this issue, can try this.
apache2ctl -M ( will list the modules enabled)

Check in that list if mpm_workder_module is enabled. If so, then
install mpm_prefork_module,
sudo aptitude install apache2-mpm-prefork

and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.

Then again try gbrowse2, this issue would have probably gone.  This
worked for me though.

Thanks
Prem

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
<ATT00001..txt><ATT00002..txt>



------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Scooter Willis-2
In reply to this post by Lincoln Stein
Lincoln

I reset apache2 for a clean start and ran the fastcgi url. It took 60 seconds to load but then returned a page. I then did a search on a term and after X amount of time get the Internal Server error which is probably related to a time out. Error message below. The mod_perl version where I have reduced the number of processes it can spawn is really fast and works well but keeps consuming memory and the server gets slow as we reach the amount of memory in the server. The performance of fastcgi and the mpm_prefork do not seem to have the memory problem but do not have the mod_perl performance.

Let me know if I should try anything.

Thanks

Scooter



[Mon Jul 12 18:24:28 2010] [error] [client 10.10.250.21] FastCGI: server "/usr/lib/cgi-bin/gb2/gbrowse" stderr: [11996] add_user_tracks() at /usr/local/lib/perl/5.10.1/Bio/Graphics/Browser2/Render.pm line 188., referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:24:28 2010] [error] [client 10.10.250.21] FastCGI: server "/usr/lib/cgi-bin/gb2/gbrowse" stderr: [11996] testing for asynchronous event() at /usr/local/lib/perl/5.10.1/Bio/Graphics/Browser2/Render.pm line 199., referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:24:28 2010] [error] [client 10.10.250.21] FastCGI: server "/usr/lib/cgi-bin/gb2/gbrowse" stderr: [11996] asynchronous exit at /usr/local/lib/perl/5.10.1/Bio/Graphics/Browser2/Render.pm line 201., referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:26:01 2010] [error] [client 10.10.250.21] FastCGI: server "/usr/lib/cgi-bin/gb2/gbrowse" stderr: [11997] RUN(): POST: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/ prefix=GH;action=autocomplete at /usr/local/lib/perl/5.10.1/Bio/Graphics/Browser2/Render.pm line 181., referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:26:01 2010] [error] [client 10.10.250.21] FastCGI: server "/usr/lib/cgi-bin/gb2/gbrowse" stderr: [11997] add_user_tracks() at /usr/local/lib/perl/5.10.1/Bio/Graphics/Browser2/Render.pm line 188., referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:26:01 2010] [error] [client 10.10.250.21] FastCGI: server "/usr/lib/cgi-bin/gb2/gbrowse" stderr: [11997] testing for asynchronous event() at /usr/local/lib/perl/5.10.1/Bio/Graphics/Browser2/Render.pm line 199., referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:26:01 2010] [error] [client 10.10.250.21] FastCGI: server "/usr/lib/cgi-bin/gb2/gbrowse" stderr: [11997] asynchronous exit at /usr/local/lib/perl/5.10.1/Bio/Graphics/Browser2/Render.pm line 201., referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:26:13 2010] [warn] FastCGI: (dynamic) server "/usr/lib/cgi-bin/gb2/gbrowse" (pid 12014) termination signaled
[Mon Jul 12 18:26:13 2010] [notice] GBrowse FastCGI process caught sigTERM. Exiting... (pid 12014)
[Mon Jul 12 18:26:13 2010] [warn] FastCGI: (dynamic) server "/usr/lib/cgi-bin/gb2/gbrowse" (pid 12014) terminated by calling exit with status '0'
[Mon Jul 12 18:26:15 2010] [warn] FastCGI: scheduled the restart of the last (dynamic) server "/usr/lib/cgi-bin/gb2/gbrowse" process: reached dynamicMaxClassProcs (10)
[Mon Jul 12 18:26:15 2010] [warn] FastCGI: (dynamic) server "/usr/lib/cgi-bin/gb2/gbrowse" restarted (pid 12027)
[Mon Jul 12 18:26:55 2010] [error] [client 10.10.250.21] FastCGI: comm with (dynamic) server "/usr/lib/cgi-bin/gb2/gbrowse" aborted: (first read) idle timeout (120 sec), referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:26:55 2010] [error] [client 10.10.250.21] FastCGI: incomplete headers (0 bytes) received from server "/usr/lib/cgi-bin/gb2/gbrowse", referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:26:57 2010] [error] [client 10.10.250.21] FastCGI: comm with (dynamic) server "/usr/lib/cgi-bin/gb2/gbrowse" aborted: (first read) idle timeout (120 sec), referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:26:57 2010] [error] [client 10.10.250.21] FastCGI: incomplete headers (0 bytes) received from server "/usr/lib/cgi-bin/gb2/gbrowse", referer: http://dyadic-c1-server/fgb2/gbrowse/c1scaffold/
[Mon Jul 12 18:27:11 2010] [notice] GBrowse FastCGI process caught sigPIPE. Exiting... (pid 11999)
[Mon Jul 12 18:27:12 2010] [warn] FastCGI: (dynamic) server "/usr/lib/cgi-bin/gb2/gbrowse" (pid 11999) terminated by calling exit with status '0'



On Jul 12, 2010, at 5:10 PM, Lincoln Stein wrote:

apache2-mpm-worker uses a threaded model which doesn't work well with mod_perl. The prefork version + modperl should be nice and fast on Linux but dreadfully slow under windows. Is this a windows installation?

Lincoln

On Mon, Jul 12, 2010 at 3:36 PM, Scooter Willis <[hidden email]> wrote:
After making the change to use mpm_prefork_module the memory usage appears to be under control but the problem now is the performance is very slow. It takes a couple minutes to load the page when visiting the first time and then has reasonable perfomance depending on what you select.

It appears the problem could be related to the latest version of apache2 installed on ubuntu where Prem found the same problem. Is it possible to get this on the radar of those with mod_perl expertise and the ability to look under the hood at the problem? Not sure I can do any more "guessing" and the lab is complaining that they can't use it because of the speed.

Thanks

Scooter




On Jul 12, 2010, at 4:37 AM, Prem Anand wrote:

HI Scooter

I have been using this settings for my vm and so far had no issues.
But I have tested only with the example yeast databases on
gbrowse-2.12 and  I have disabled mod_perl. Will be trying enabling
mod_perl and will let you know how it goes.

<IfModule mpm_prefork_module>
   StartServers          5
   MinSpareServers       5
   MaxSpareServers      10
   MaxClients           15
   MaxRequestsPerChild  2000
</IfModule>

Thanks
Prem

On Mon, Jul 12, 2010 at 2:10 AM, Scooter Willis <[hidden email]> wrote:

The "tuning" only appears to delay the time until I run out of memory.
I followed Prem's advice and installed the prefork module. Initial testing
after loading a couple scaffolds and doing some searches loads about 4GB+ of
memory where I am using the following modified settings. The default prefork
settings quickly got to 8GB of memory. In my configuration I am loading a
fungal genome into memory. Does anyone have modified prefork_module settings
in apache2.conf?
<IfModule mpm_prefork_module>
    StartServers          2
    MinSpareServers       2
    MaxSpareServers       2
    MaxClients           10
    MaxRequestsPerChild 1000
</IfModule>

On Jul 9, 2010, at 10:22 AM, Scooter Willis wrote:

I have been working on the apache config settings to "tune" apache2 to not
run out of memory. I think I am making progress and will send a summary when
I stop running out of memory. The key appears to limit the number of
processes apache2 will start where the default is 16 and not listed in the
config file. I have ir currently set to 2 processes because each process
consumes 1.7 gb of memory. The other option is to force apache2 to dump the
process after x number of requests to avoid memory usage increasing from
dirty memory. Apache2 will then spawn a new process.

The values need to balance memory, processors, and number of concurrent
users.

Glad to see I am not the only person with this problem and another possible
solution.

Thanks

Scooter

----- Reply message -----
From: "Prem Anand" <[hidden email]>
Date: Fri, Jul 9, 2010 9:51 am
Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
To: "[hidden email]"
<[hidden email]>
mpm_worker_module

Hi

Recently I installed GBrowse2 in a clean vm but had issues similar to
the one mentioned by Scooter earlier, multiple instances of apache2
processes hanging around and consuming lots of memory.
http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel

After comparing with other vms where we did not have this issue,
realized that, this issue was there only where the mpm_worker_module
was enabled.  If anyone else has had this issue, can try this.
apache2ctl -M ( will list the modules enabled)

Check in that list if mpm_workder_module is enabled. If so, then
install mpm_prefork_module,
sudo aptitude install apache2-mpm-prefork

and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.

Then again try gbrowse2, this issue would have probably gone.  This
worked for me though.

Thanks
Prem

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
<ATT00001..txt><ATT00002..txt>



------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]>


------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Multiple instances of apache2 processes and MPMs

Scooter Willis-2
In reply to this post by Lincoln Stein
The following settings for the mpm_preform_module seem to be working where performance is good and memory usage is reasonable.

Each process is consuming 670-800 MB of memory. Since I have limited MaxClients to 3 I only have three processes. If it was set to 15 then 15 x 800MB and you easily burn through 8 GB of memory. It is also not clear what the definition of a client is from the apache view where using gbrowse2 ajax capabilities does each request look like a different client? So even with one user connected apache2 would start up 15 processes to server 15 parallel ajax requests. I also found in the docs that the default for MaxMemFree is 0 which means free() is never called and you don't free up any unused memory. Not sure if this is per process or for all processes. The MaxRequestsPerChild 2000 will kill the process after 2000 requests to free up memory. I also have KeepAlive turned off which is recommended for dynamic content.


<IfModule mpm_prefork_module>
    StartServers          2
    MinSpareServers       2
    MaxSpareServers       3
    MaxClients            3
    MaxRequestsPerChild 2000
    MaxMemFree 2000000
</IfModule>

________________________________________
From: Lincoln Stein [[hidden email]]
Sent: Monday, July 12, 2010 5:10 PM
To: Scooter Willis
Cc: Prem Anand; [hidden email]
Subject: Re: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs

apache2-mpm-worker uses a threaded model which doesn't work well with mod_perl. The prefork version + modperl should be nice and fast on Linux but dreadfully slow under windows. Is this a windows installation?

Lincoln

On Mon, Jul 12, 2010 at 3:36 PM, Scooter Willis <[hidden email]<mailto:[hidden email]>> wrote:
After making the change to use mpm_prefork_module the memory usage appears to be under control but the problem now is the performance is very slow. It takes a couple minutes to load the page when visiting the first time and then has reasonable perfomance depending on what you select.

It appears the problem could be related to the latest version of apache2 installed on ubuntu where Prem found the same problem. Is it possible to get this on the radar of those with mod_perl expertise and the ability to look under the hood at the problem? Not sure I can do any more "guessing" and the lab is complaining that they can't use it because of the speed.

Thanks

Scooter




On Jul 12, 2010, at 4:37 AM, Prem Anand wrote:

HI Scooter

I have been using this settings for my vm and so far had no issues.
But I have tested only with the example yeast databases on
gbrowse-2.12 and  I have disabled mod_perl. Will be trying enabling
mod_perl and will let you know how it goes.

<IfModule mpm_prefork_module>
   StartServers          5
   MinSpareServers       5
   MaxSpareServers      10
   MaxClients           15
   MaxRequestsPerChild  2000
</IfModule>

Thanks
Prem

On Mon, Jul 12, 2010 at 2:10 AM, Scooter Willis <[hidden email]<mailto:[hidden email]>> wrote:

The "tuning" only appears to delay the time until I run out of memory.
I followed Prem's advice and installed the prefork module. Initial testing
after loading a couple scaffolds and doing some searches loads about 4GB+ of
memory where I am using the following modified settings. The default prefork
settings quickly got to 8GB of memory. In my configuration I am loading a
fungal genome into memory. Does anyone have modified prefork_module settings
in apache2.conf?
<IfModule mpm_prefork_module>
    StartServers          2
    MinSpareServers       2
    MaxSpareServers       2
    MaxClients           10
    MaxRequestsPerChild 1000
</IfModule>

On Jul 9, 2010, at 10:22 AM, Scooter Willis wrote:

I have been working on the apache config settings to "tune" apache2 to not
run out of memory. I think I am making progress and will send a summary when
I stop running out of memory. The key appears to limit the number of
processes apache2 will start where the default is 16 and not listed in the
config file. I have ir currently set to 2 processes because each process
consumes 1.7 gb of memory. The other option is to force apache2 to dump the
process after x number of requests to avoid memory usage increasing from
dirty memory. Apache2 will then spawn a new process.

The values need to balance memory, processors, and number of concurrent
users.

Glad to see I am not the only person with this problem and another possible
solution.

Thanks

Scooter

----- Reply message -----
From: "Prem Anand" <[hidden email]<mailto:[hidden email]>>
Date: Fri, Jul 9, 2010 9:51 am
Subject: [Gmod-gbrowse] Multiple instances of apache2 processes and MPMs
To: "[hidden email]<mailto:[hidden email]>"
<[hidden email]<mailto:[hidden email]>>
mpm_worker_module

Hi

Recently I installed GBrowse2 in a clean vm but had issues similar to
the one mentioned by Scooter earlier, multiple instances of apache2
processes hanging around and consuming lots of memory.
http://sourceforge.net/mailarchive/forum.php?thread_name=20100630162300.GA574%40dhcp-165-124-223-253.nubic.northwestern.edu&forum_name=gmod-devel
http://sourceforge.net/mailarchive/forum.php?thread_name=FF46E20B7EF40940B038819E2CFBA8523F0A5F05E6%40CCRMB1.fl.ad.scripps.edu&forum_name=gmod-devel

After comparing with other vms where we did not have this issue,
realized that, this issue was there only where the mpm_worker_module
was enabled.  If anyone else has had this issue, can try this.
apache2ctl -M ( will list the modules enabled)

Check in that list if mpm_workder_module is enabled. If so, then
install mpm_prefork_module,
sudo aptitude install apache2-mpm-prefork

and again make sure that if mpm_prefork_module was enabled by apache2ctl -M.

Then again try gbrowse2, this issue would have probably gone.  This
worked for me though.

Thanks
Prem

------------------------------------------------------------------------------
This SF.net<http://SF.net> email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first<http://sprint.com/first> -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]<mailto:[hidden email]>
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
<ATT00001..txt><ATT00002..txt>



------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first<http://sprint.com/first> -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]<mailto:[hidden email]>
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




--
Lincoln D. Stein
Director, Informatics and Biocomputing Platform
Ontario Institute for Cancer Research
101 College St., Suite 800
Toronto, ON, Canada M5G0A3
416 673-8514
Assistant: Renata Musa <[hidden email]<mailto:[hidden email]>>

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse