gbrowse_clean.pl not cleaning sessions

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

gbrowse_clean.pl not cleaning sessions

Wes Barris
I have cron configured to run gbrowse_clean.pl every night.  Typical output
looks something like this:

*** Fri Nov 12 00:05:02 2010 gbrowse_clean.pl starting ***
Expiring sessions older than 1M...0 sessions deleted.
Deleting cache files and directories...1705 directories and 946 files deleted.
***  Fri Nov 12 00:05:03 2010 /htdocs/gbrowse2/share/bin/gbrowse_clean.pl done ***

It never deletes any sessions.  My sessions file is large:

wes@bioweb> ls -l gbrowse2/tmp/sessions/cgisess.db
-rw-r-----  1 nobody nobody 148549632 Nov 15 14:41 gbrowse2/tmp/sessions/cgisess.db

My GBrowse.conf file uses the default session configuration:

# If no session driver is set, then GBrowse will pick one for you.
# It will use db_file for the driver and storable for the serializer
# if available; otherwise falling back to the file driver and default serializer.

# Override driver guessing by setting these options
# The safest, but slowest session driver...
#session driver    = driver:file;serializer:default
#session args      = Directory /htdocs/gbrowse2/tmp/sessions

# to use the berkeley DB driver comment out the previous
# line and uncomment these two
#session driver = driver:db_file;serializer:default
#session args   = FileName /htdocs/gbrowse2/tmp/sessions.db

Is there something wrong with my session configuration or is the
cgisess.db file supposed to be that large?
--
Wes Barris

------------------------------------------------------------------------------
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: gbrowse_clean.pl not cleaning sessions

Jayaraman, Pushkala
I seem to be facing the same problem.. none of the session files seem to
get cleaned up..

Pushkala Jayaraman
Programmer/Analyst
Rat Genome Database
Human and Molecular Genetics Center
Medical College of Wisconsin
Email: [hidden email]
Work: 414-955-2229
www.rgd.mcw.edu


-----Original Message-----
From: Wes Barris [mailto:[hidden email]]
Sent: Sunday, November 14, 2010 11:34 PM
To: Gbrowse Mailing List
Subject: [Gmod-gbrowse] gbrowse_clean.pl not cleaning sessions

I have cron configured to run gbrowse_clean.pl every night.  Typical
output
looks something like this:

*** Fri Nov 12 00:05:02 2010 gbrowse_clean.pl starting ***
Expiring sessions older than 1M...0 sessions deleted.
Deleting cache files and directories...1705 directories and 946 files
deleted.
***  Fri Nov 12 00:05:03 2010
/htdocs/gbrowse2/share/bin/gbrowse_clean.pl done ***

It never deletes any sessions.  My sessions file is large:

wes@bioweb> ls -l gbrowse2/tmp/sessions/cgisess.db
-rw-r-----  1 nobody nobody 148549632 Nov 15 14:41
gbrowse2/tmp/sessions/cgisess.db

My GBrowse.conf file uses the default session configuration:

# If no session driver is set, then GBrowse will pick one for you.
# It will use db_file for the driver and storable for the serializer
# if available; otherwise falling back to the file driver and default
serializer.

# Override driver guessing by setting these options
# The safest, but slowest session driver...
#session driver    = driver:file;serializer:default
#session args      = Directory /htdocs/gbrowse2/tmp/sessions

# to use the berkeley DB driver comment out the previous
# line and uncomment these two
#session driver = driver:db_file;serializer:default
#session args   = FileName /htdocs/gbrowse2/tmp/sessions.db

Is there something wrong with my session configuration or is the
cgisess.db file supposed to be that large?
--
Wes Barris

------------------------------------------------------------------------
------
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

------------------------------------------------------------------------------
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: gbrowse_clean.pl not cleaning sessions

okeeffe
Jayaraman, Pushkala wrote:

> I seem to be facing the same problem.. none of the session files seem to
> get cleaned up..
>
> Pushkala Jayaraman
> Programmer/Analyst
> Rat Genome Database
> Human and Molecular Genetics Center
> Medical College of Wisconsin
> Email: [hidden email]
> Work: 414-955-2229
> www.rgd.mcw.edu
>
>
> -----Original Message-----
> From: Wes Barris [mailto:[hidden email]]
> Sent: Sunday, November 14, 2010 11:34 PM
> To: Gbrowse Mailing List
> Subject: [Gmod-gbrowse] gbrowse_clean.pl not cleaning sessions
>
> I have cron configured to run gbrowse_clean.pl every night.  Typical
> output
> looks something like this:
>
> *** Fri Nov 12 00:05:02 2010 gbrowse_clean.pl starting ***
> Expiring sessions older than 1M...0 sessions deleted.
> Deleting cache files and directories...1705 directories and 946 files
> deleted.
> ***  Fri Nov 12 00:05:03 2010
> /htdocs/gbrowse2/share/bin/gbrowse_clean.pl done ***
>
> It never deletes any sessions.  My sessions file is large:
>
> wes@bioweb> ls -l gbrowse2/tmp/sessions/cgisess.db
> -rw-r-----  1 nobody nobody 148549632 Nov 15 14:41
> gbrowse2/tmp/sessions/cgisess.db
>
> My GBrowse.conf file uses the default session configuration:
>
> # If no session driver is set, then GBrowse will pick one for you.
> # It will use db_file for the driver and storable for the serializer
> # if available; otherwise falling back to the file driver and default
> serializer.
>
> # Override driver guessing by setting these options
> # The safest, but slowest session driver...
> #session driver    = driver:file;serializer:default
> #session args      = Directory /htdocs/gbrowse2/tmp/sessions
>
> # to use the berkeley DB driver comment out the previous
> # line and uncomment these two
> #session driver = driver:db_file;serializer:default
> #session args   = FileName /htdocs/gbrowse2/tmp/sessions.db
>
> Is there something wrong with my session configuration or is the
> cgisess.db file supposed to be that large?
>  
I've set up a list of cronjobs to run a weekly cleanup of various
cache,image, lock directories.
It seems to work fine. modify and add accordingly:

0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/cache/Human_hg18/
-atime +7 -exec rm -rf {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/cache/MeDIP/ -atime
+7 -exec rm -rf {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/cache/Mouse/ -atime
+7 -exec rm -rf {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/cache/Patient1/
-atime +7 -exec rm -rf {} \;

## gbrowse2 internal lock file removal
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/locks/ -atime +7
-exec rm -rf {} \;

## gbrowse2 internal userdata file removal
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/userdata/DS/ -atime
+7 -exec rm -rf {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/userdata/Human/
-atime +7 -exec rm -rf {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/userdata/Human_hg18/
-atime +7 -exec rm -rf {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/userdata/MeDIP/
-atime +7 -exec rm -rf {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/userdata/Mouse/
-atime +7 -exec rm -rf {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/userdata/Patient1/
-atime +7 -exec rm -rf {} \;

## gbrowse2 internal image file removal
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/images/DS/*.png
-atime +7 -exec rm -f {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/images/Human/*.png
-atime +7 -exec rm -f {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/images/Human_hg18/*.png
-atime +7 -exec rm -f {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/images/MeDIP/*.png
-atime +7 -exec rm -f {} \;
0 0 * * 1 /bin/find
/project/gbrowse/httpd_internal/htdocs/gbrowse2/tmp/images/Mouse/*.png
-atime +7 -exec rm -f {} \;




------------------------------------------------------------------------------
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: gbrowse_clean.pl not cleaning sessions

Robert Buels
Reports of a problem, and hacked-up cronjob workarounds are all well and
good, but how about one of you guys offers a patch to gbrowse_clean.pl
for cleaning up session data and ask for a review on it?

Rob



------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today
http://p.sf.net/sfu/msIE9-sfdev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: gbrowse_clean.pl not cleaning sessions

Wes Barris
On 17/11/2010 4:18 AM, Robert Buels wrote:
> Reports of a problem, and hacked-up cronjob workarounds are all well and
> good, but how about one of you guys offers a patch to gbrowse_clean.pl
> for cleaning up session data and ask for a review on it?
>
> Rob

Hi Rob,

I would provide a fix if I could but I do not know how this is supposed
to work.  The comments in GBrowse.conf state this:

# If no session driver is set, then GBrowse will pick one for you.
# It will use db_file for the driver and storable for the serializer
# if available; otherwise falling back to the file driver and default serializer

I can see that this is implemented in the session_driver routine in
Browser2.pm.  I do not believe that gbrowse_clean.pl uses this and I
do not know how to make gbrowse_clean.pl use the routine in Browser2.pm.

The pertinent code in gbrowse_clean.pl is:

logit("Expiring sessions older than $remember_settings_time...");
my $dsn     = $globals->code_setting(general=>'session driver');
my %dsn     = shellwords($globals->code_setting(general=>'session args'));
my $rst     = $globals->time2sec($remember_settings_time);
my $now     = time();
my $deleted = 0;
CGI::Session->find($dsn,
                    sub {
                        my $session = shift;
                        return if $session->is_empty;
                        return if $session->atime + $rst > $now;
                        verbose("session ",$session->id," deleted\n");
                        $session->delete();
                        $session->flush();
                        $deleted++;

                    },\%dsn);
logit("$deleted sessions deleted.\n");

I've debugged this and the find loop is not finding any session.  I
believe this is because $dsn is not set (commented out in GBrowse.conf).

I think that the fix would be to add a line like this:

my $dsn = session_driver(...)

but I don't know what arguments to provide.
--
Wes Barris

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today
http://p.sf.net/sfu/msIE9-sfdev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: gbrowse_clean.pl not cleaning sessions

Matthew Conte
I am curious if anyone has been able to make any progress with this problem?

I've got a cgisess.db file that just recently has grown to 512GB in size!  I'm also curious what exactly is stored in this file and how it could have ballooned to such a size, so I can prevent this from happening again.  None of the cache, images, locks or userdata directories seem to be taking up much space.  Since I can't get the gbrowse_clean.pl script to delete this file, is it safe to manually delete so I can free up the needed space?

Thanks,
Matt


On Tue, Nov 16, 2010 at 8:04 PM, Wes Barris <[hidden email]> wrote:
On 17/11/2010 4:18 AM, Robert Buels wrote:
> Reports of a problem, and hacked-up cronjob workarounds are all well and
> good, but how about one of you guys offers a patch to gbrowse_clean.pl
> for cleaning up session data and ask for a review on it?
>
> Rob

Hi Rob,

I would provide a fix if I could but I do not know how this is supposed
to work.  The comments in GBrowse.conf state this:

# If no session driver is set, then GBrowse will pick one for you.
# It will use db_file for the driver and storable for the serializer
# if available; otherwise falling back to the file driver and default serializer

I can see that this is implemented in the session_driver routine in
Browser2.pm.  I do not believe that gbrowse_clean.pl uses this and I
do not know how to make gbrowse_clean.pl use the routine in Browser2.pm.

The pertinent code in gbrowse_clean.pl is:

logit("Expiring sessions older than $remember_settings_time...");
my $dsn     = $globals->code_setting(general=>'session driver');
my %dsn     = shellwords($globals->code_setting(general=>'session args'));
my $rst     = $globals->time2sec($remember_settings_time);
my $now     = time();
my $deleted = 0;
CGI::Session->find($dsn,
                   sub {
                       my $session = shift;
                       return if $session->is_empty;
                       return if $session->atime + $rst > $now;
                       verbose("session ",$session->id," deleted\n");
                       $session->delete();
                       $session->flush();
                       $deleted++;

                   },\%dsn);
logit("$deleted sessions deleted.\n");

I've debugged this and the find loop is not finding any session.  I
believe this is because $dsn is not set (commented out in GBrowse.conf).

I think that the fix would be to add a line like this:

my $dsn = session_driver(...)

but I don't know what arguments to provide.
--
Wes Barris

------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
Spend less time writing and  rewriting code and more time creating great
experiences on the web. Be a part of the beta today
http://p.sf.net/sfu/msIE9-sfdev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse




------------------------------------------------------------------------------
Enable your software for Intel(R) Active Management Technology to meet the
growing manageability and security demands of your customers. Businesses
are taking advantage of Intel(R) vPro (TM) technology - will your software
be a part of the solution? Download the Intel(R) Manageability Checker
today! http://p.sf.net/sfu/intel-dev2devmar
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: gbrowse_clean.pl not cleaning sessions

Scott Cain
Hi Matt, Wes and everybody else,

Sorry to be responding to a 3 year old thread (well, nearly 4 years old now), but this was brought to my attention recently and I wanted to follow up on it.

It seems to me that the gbrowse_clean script is not quite as smart as it ought to be.  That is, with regards to session variables, the GBrowse.conf file says in the comments that it will pick defaults for you, and while it does seem to do that correctly when the CGI is running (that is, GBrowse correctly stores the session database is a reasonable place), the gbrowse_clean script doesn't seem to know how to do this.  While fixing it shouldn't be too hard (and I will create a bug report so I don't forget about it), the easiest fix in the short term is to uncomment the two session variables that appear first in GBrowse.conf.  That is, uncomment these two lines:

#session driver    = driver:file;serializer:default
#session args      = Directory /var/lib/gbrowse2/sessions

and verify of course that the directory listed is where the session database is actually stored.

Scott




On Fri, Mar 25, 2011 at 3:59 PM, Matthew Conte <[hidden email]> wrote:
>
> I am curious if anyone has been able to make any progress with this problem?
>
> I've got a cgisess.db file that just recently has grown to 512GB in size!  I'm also curious what exactly is stored in this file and how it could have ballooned to such a size, so I can prevent this from happening again.  None of the cache, images, locks or userdata directories seem to be taking up much space.  Since I can't get the gbrowse_clean.pl script to delete this file, is it safe to manually delete so I can free up the needed space?
>
> Thanks,
> Matt
>
>
> On Tue, Nov 16, 2010 at 8:04 PM, Wes Barris <[hidden email]> wrote:
>>
>> On 17/11/2010 4:18 AM, Robert Buels wrote:
>> > Reports of a problem, and hacked-up cronjob workarounds are all well and
>> > good, but how about one of you guys offers a patch to gbrowse_clean.pl
>> > for cleaning up session data and ask for a review on it?
>> >
>> > Rob
>>
>> Hi Rob,
>>
>> I would provide a fix if I could but I do not know how this is supposed
>> to work.  The comments in GBrowse.conf state this:
>>
>> # If no session driver is set, then GBrowse will pick one for you.
>> # It will use db_file for the driver and storable for the serializer
>> # if available; otherwise falling back to the file driver and default serializer
>>
>> I can see that this is implemented in the session_driver routine in
>> Browser2.pm.  I do not believe that gbrowse_clean.pl uses this and I
>> do not know how to make gbrowse_clean.pl use the routine in Browser2.pm.
>>
>> The pertinent code in gbrowse_clean.pl is:
>>
>> logit("Expiring sessions older than $remember_settings_time...");
>> my $dsn     = $globals->code_setting(general=>'session driver');
>> my %dsn     = shellwords($globals->code_setting(general=>'session args'));
>> my $rst     = $globals->time2sec($remember_settings_time);
>> my $now     = time();
>> my $deleted = 0;
>> CGI::Session->find($dsn,
>>                    sub {
>>                        my $session = shift;
>>                        return if $session->is_empty;
>>                        return if $session->atime + $rst > $now;
>>                        verbose("session ",$session->id," deleted\n");
>>                        $session->delete();
>>                        $session->flush();
>>                        $deleted++;
>>
>>                    },\%dsn);
>> logit("$deleted sessions deleted.\n");
>>
>> I've debugged this and the find loop is not finding any session.  I
>> believe this is because $dsn is not set (commented out in GBrowse.conf).
>>
>> I think that the fix would be to add a line like this:
>>
>> my $dsn = session_driver(...)
>>
>> but I don't know what arguments to provide.
>> --
>> Wes Barris
>>
>> ------------------------------------------------------------------------------
>> Beautiful is writing same markup. Internet Explorer 9 supports
>> standards for HTML5, CSS3, SVG 1.1,  ECMAScript5, and DOM L2 & L3.
>> Spend less time writing and  rewriting code and more time creating great
>> experiences on the web. Be a part of the beta today
>> http://p.sf.net/sfu/msIE9-sfdev2dev
>> _______________________________________________
>> Gmod-gbrowse mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>
>
>
>
>
> ------------------------------------------------------------------------------
> Enable your software for Intel(R) Active Management Technology to meet the
> growing manageability and security demands of your customers. Businesses
> are taking advantage of Intel(R) vPro (TM) technology - will your software
> be a part of the solution? Download the Intel(R) Manageability Checker
> today! http://p.sf.net/sfu/intel-dev2devmar
> _______________________________________________
> Gmod-gbrowse mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>



--
------------------------------------------------------------------------
Scott Cain, Ph. D.                                   scott at scottcain dot net
GMOD Coordinator (http://gmod.org/)                     216-392-3087
Ontario Institute for Cancer Research

------------------------------------------------------------------------------
Open source business process management suite built on Java and Eclipse
Turn processes into business applications with Bonita BPM Community Edition
Quickly connect people, data, and systems into organized workflows
Winner of BOSSIE, CODIE, OW2 and Gartner awards
http://p.sf.net/sfu/Bonitasoft
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse