Updating and deleting data in my GBrowse database

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Updating and deleting data in my GBrowse database

Erick Antezana-2
Hi,

We have been populating our GBrowse databases (MySQL) using the
scripts bundled in the BioPerl distribution [1]. Every time the
annotations corresponding to a given database were updated (curation,
wrong data, fixes, etc), we were simply replacing the database
entirely with the new data. As this operation is becoming more and
more frequent and considering that sometimes the loading process could
take a "considerable amount of time", I was wondering whether there is
a script to update (and/or delete) a single feature (a line or set of
related lines of a GFF3 file) without the need of reloading it from
scratch?

cheers,
Erick

[1] http://gmod.org/wiki/GBrowse_Install_HOWTO#Populating_the_Database_.28MySQL.29

------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Updating and deleting data in my GBrowse database

Jason Stajich-4
Erick -

Are you using SeqFeature::Store database?

If so:
 bp_seqfeature_delete.pl for deleting

bp_seqfeature_load.pl

has the option to load without removing the existing data and will be an appending.

I'm not sure if there is a needed periodic reoptimizing or of tables or if this is still done as part of the final step in loading.

Jason

On Jun 23, 2011, at 7:37 AM, Erick Antezana wrote:

> Hi,
>
> We have been populating our GBrowse databases (MySQL) using the
> scripts bundled in the BioPerl distribution [1]. Every time the
> annotations corresponding to a given database were updated (curation,
> wrong data, fixes, etc), we were simply replacing the database
> entirely with the new data. As this operation is becoming more and
> more frequent and considering that sometimes the loading process could
> take a "considerable amount of time", I was wondering whether there is
> a script to update (and/or delete) a single feature (a line or set of
> related lines of a GFF3 file) without the need of reloading it from
> scratch?
>
> cheers,
> Erick
>
> [1] http://gmod.org/wiki/GBrowse_Install_HOWTO#Populating_the_Database_.28MySQL.29
>
> ------------------------------------------------------------------------------
> Simplify data backup and recovery for your virtual environment with vRanger.
> Installation's a snap, and flexible recovery options mean your data is safe,
> secure and there when you need it. Data protection magic?
> Nope - It's vRanger. Get your free trial download today.
> http://p.sf.net/sfu/quest-sfdev2dev
> _______________________________________________
> Gmod-gbrowse mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


------------------------------------------------------------------------------
Simplify data backup and recovery for your virtual environment with vRanger.
Installation's a snap, and flexible recovery options mean your data is safe,
secure and there when you need it. Data protection magic?
Nope - It's vRanger. Get your free trial download today.
http://p.sf.net/sfu/quest-sfdev2dev
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Updating and deleting data in my GBrowse database

Erick Antezana-2
Hi Jason,

thanks for you reply. For some DBs, we use Bio::DB::SeqFeature::Store,
but we still have many repositories using Bio::DB::GFF :-( we're in
the process of migrating some of the latter DBs to
Bio::DB::SeqFeature::Store. So, it seems to be one more reason to
perform such migration...

cheers,
Erick

On 23 June 2011 16:50, Jason Stajich <[hidden email]> wrote:

> Erick -
>
> Are you using SeqFeature::Store database?
>
> If so:
>  bp_seqfeature_delete.pl for deleting
>
> bp_seqfeature_load.pl
>
> has the option to load without removing the existing data and will be an appending.
>
> I'm not sure if there is a needed periodic reoptimizing or of tables or if this is still done as part of the final step in loading.
>
> Jason
>
> On Jun 23, 2011, at 7:37 AM, Erick Antezana wrote:
>
>> Hi,
>>
>> We have been populating our GBrowse databases (MySQL) using the
>> scripts bundled in the BioPerl distribution [1]. Every time the
>> annotations corresponding to a given database were updated (curation,
>> wrong data, fixes, etc), we were simply replacing the database
>> entirely with the new data. As this operation is becoming more and
>> more frequent and considering that sometimes the loading process could
>> take a "considerable amount of time", I was wondering whether there is
>> a script to update (and/or delete) a single feature (a line or set of
>> related lines of a GFF3 file) without the need of reloading it from
>> scratch?
>>
>> cheers,
>> Erick
>>
>> [1] http://gmod.org/wiki/GBrowse_Install_HOWTO#Populating_the_Database_.28MySQL.29
>>
>> ------------------------------------------------------------------------------
>> Simplify data backup and recovery for your virtual environment with vRanger.
>> Installation's a snap, and flexible recovery options mean your data is safe,
>> secure and there when you need it. Data protection magic?
>> Nope - It's vRanger. Get your free trial download today.
>> http://p.sf.net/sfu/quest-sfdev2dev
>> _______________________________________________
>> Gmod-gbrowse mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>
>

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Updating and deleting data in my GBrowse database

Fields, Christopher J
Is there any migration script out there that can at least make an initial migration from Bio::DB::GFF possible (I know that parent-child relationships might be a problem...).

chris

On Jun 27, 2011, at 8:46 AM, Erick Antezana wrote:

> Hi Jason,
>
> thanks for you reply. For some DBs, we use Bio::DB::SeqFeature::Store,
> but we still have many repositories using Bio::DB::GFF :-( we're in
> the process of migrating some of the latter DBs to
> Bio::DB::SeqFeature::Store. So, it seems to be one more reason to
> perform such migration...
>
> cheers,
> Erick
>
> On 23 June 2011 16:50, Jason Stajich <[hidden email]> wrote:
>> Erick -
>>
>> Are you using SeqFeature::Store database?
>>
>> If so:
>>  bp_seqfeature_delete.pl for deleting
>>
>> bp_seqfeature_load.pl
>>
>> has the option to load without removing the existing data and will be an appending.
>>
>> I'm not sure if there is a needed periodic reoptimizing or of tables or if this is still done as part of the final step in loading.
>>
>> Jason
>>
>> On Jun 23, 2011, at 7:37 AM, Erick Antezana wrote:
>>
>>> Hi,
>>>
>>> We have been populating our GBrowse databases (MySQL) using the
>>> scripts bundled in the BioPerl distribution [1]. Every time the
>>> annotations corresponding to a given database were updated (curation,
>>> wrong data, fixes, etc), we were simply replacing the database
>>> entirely with the new data. As this operation is becoming more and
>>> more frequent and considering that sometimes the loading process could
>>> take a "considerable amount of time", I was wondering whether there is
>>> a script to update (and/or delete) a single feature (a line or set of
>>> related lines of a GFF3 file) without the need of reloading it from
>>> scratch?
>>>
>>> cheers,
>>> Erick
>>>
>>> [1] http://gmod.org/wiki/GBrowse_Install_HOWTO#Populating_the_Database_.28MySQL.29
>>>
>>> ------------------------------------------------------------------------------
>>> Simplify data backup and recovery for your virtual environment with vRanger.
>>> Installation's a snap, and flexible recovery options mean your data is safe,
>>> secure and there when you need it. Data protection magic?
>>> Nope - It's vRanger. Get your free trial download today.
>>> http://p.sf.net/sfu/quest-sfdev2dev
>>> _______________________________________________
>>> Gmod-gbrowse mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>>
>>
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Gmod-gbrowse mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse


------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Updating and deleting data in my GBrowse database

Scott Cain
In reply to this post by Erick Antezana-2
Hi Erick,

There are delete functions in Bio::DB::GFF, like delete_features and
delete_groups, though there is not a standard script for doing the
deletes.  I haven't used these methods in a long time, so I can't
assure you that they work, but they probably do :-)

Scott


On Mon, Jun 27, 2011 at 9:46 AM, Erick Antezana
<[hidden email]> wrote:

> Hi Jason,
>
> thanks for you reply. For some DBs, we use Bio::DB::SeqFeature::Store,
> but we still have many repositories using Bio::DB::GFF :-( we're in
> the process of migrating some of the latter DBs to
> Bio::DB::SeqFeature::Store. So, it seems to be one more reason to
> perform such migration...
>
> cheers,
> Erick
>
> On 23 June 2011 16:50, Jason Stajich <[hidden email]> wrote:
>> Erick -
>>
>> Are you using SeqFeature::Store database?
>>
>> If so:
>>  bp_seqfeature_delete.pl for deleting
>>
>> bp_seqfeature_load.pl
>>
>> has the option to load without removing the existing data and will be an appending.
>>
>> I'm not sure if there is a needed periodic reoptimizing or of tables or if this is still done as part of the final step in loading.
>>
>> Jason
>>
>> On Jun 23, 2011, at 7:37 AM, Erick Antezana wrote:
>>
>>> Hi,
>>>
>>> We have been populating our GBrowse databases (MySQL) using the
>>> scripts bundled in the BioPerl distribution [1]. Every time the
>>> annotations corresponding to a given database were updated (curation,
>>> wrong data, fixes, etc), we were simply replacing the database
>>> entirely with the new data. As this operation is becoming more and
>>> more frequent and considering that sometimes the loading process could
>>> take a "considerable amount of time", I was wondering whether there is
>>> a script to update (and/or delete) a single feature (a line or set of
>>> related lines of a GFF3 file) without the need of reloading it from
>>> scratch?
>>>
>>> cheers,
>>> Erick
>>>
>>> [1] http://gmod.org/wiki/GBrowse_Install_HOWTO#Populating_the_Database_.28MySQL.29
>>>
>>> ------------------------------------------------------------------------------
>>> Simplify data backup and recovery for your virtual environment with vRanger.
>>> Installation's a snap, and flexible recovery options mean your data is safe,
>>> secure and there when you need it. Data protection magic?
>>> Nope - It's vRanger. Get your free trial download today.
>>> http://p.sf.net/sfu/quest-sfdev2dev
>>> _______________________________________________
>>> Gmod-gbrowse mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>>
>>
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Gmod-gbrowse mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>



--
------------------------------------------------------------------------
Scott Cain, Ph. D.                                   scott at scottcain dot net
GMOD Coordinator (http://gmod.org/)                     216-392-3087
Ontario Institute for Cancer Research

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse