loading chado database into existing tripal deployment

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

loading chado database into existing tripal deployment

Alex Rice
Hi all,

We have an inhouse chado database that is being populated by various
team members. I got a pg_dump of the database.

Meanwhile, my tripal deployment was created by working through this
tutorial:
http://www.gmod.org/wiki/Tripal_Tutorial_v1.1

I am trying to load our chado database into the tripal database.

I first dropped the chado schemas which were leftover from working the
tutorial (chado, so, genetic_code and frange)

Then I followed these instructions exactly:
http://tripal.info/documentation/migrate_chado

But in my tripal web interface I am now seeing errors like

ERROR: relation "chado.organism_feature_count" does not exist
ERROR: relation "chado.kegg_by_organism" does not exist

And the organisms list is not correct.

- What are the commands or procedure(s) to tell Tripal to get into sync
with the chado database?

- How do tripal users really handle this in the real world? It seems
problematic if you have to re-install and re-build your entire tripal
site every time you load a new chado database?

Thanks in advance!

--
Alex Rice <[hidden email]>
Software Engineer
National Center for Genome Resources (NCGR)
http://ncgr.org
(505)995-4457

------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal
Reply | Threaded
Open this post in threaded view
|

Re: loading chado database into existing tripal deployment

Stephen Ficklin-2
Hi Alex,

The migration documentation you followed had some missing information
for your case.  I have updated the documentation
(http://tripal.info/documentation/migrate_chado) to provide some
additional instructions.  The problem is that Tripal installs some
tables such as materialized views (e.g. organism_feature_count,
kegg_by_organism) into the Chado schema when the modules are installed.  
Because Chado was migrated after the Tripal modules were installed those
tables are missing.

So here's how you can correct the problem:

1) Disable (not uninstall) the Tripal modules (do not uninstall them yet)
2) Rename the 'chado' schema to something else (e.g. 'chado_temp').  We
do this to avoid any side-effects when uninstalling the Tripal modules.
3) Now, uninstall the tripal modules.  You may get some errors about
missing tables, but as long as the module is uninstalled afterwords
there should not be a problem later.
3) Rename the 'chado_temp' schema back to 'chado'.
4) Enable the Tripal modules in the order described in the installation
instructions: http://gmod.org/wiki/Tripal_Tutorial_v1.1#Installation

Let me know how it goes,
Stephen


On 12/20/2013 6:09 PM, Alex Rice wrote:

> Hi all,
>
> We have an inhouse chado database that is being populated by various
> team members. I got a pg_dump of the database.
>
> Meanwhile, my tripal deployment was created by working through this
> tutorial:
> http://www.gmod.org/wiki/Tripal_Tutorial_v1.1
>
> I am trying to load our chado database into the tripal database.
>
> I first dropped the chado schemas which were leftover from working the
> tutorial (chado, so, genetic_code and frange)
>
> Then I followed these instructions exactly:
> http://tripal.info/documentation/migrate_chado
>
> But in my tripal web interface I am now seeing errors like
>
> ERROR: relation "chado.organism_feature_count" does not exist
> ERROR: relation "chado.kegg_by_organism" does not exist
>
> And the organisms list is not correct.
>
> - What are the commands or procedure(s) to tell Tripal to get into sync
> with the chado database?
>
> - How do tripal users really handle this in the real world? It seems
> problematic if you have to re-install and re-build your entire tripal
> site every time you load a new chado database?
>
> Thanks in advance!
>


------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal
Reply | Threaded
Open this post in threaded view
|

Re: loading chado database into existing tripal deployment

Alex Rice
Thanks Stephen, that is helpful. Working through it...
I was also wondering if the pg_dump options should be
  --no-owner --no-acl?

Because if I to load a chado dump into an empty postgres database, I get
errors about missing roles when granting privs for chado, chado_user,
chado_read and pgsql. Unless I add the --no-acl to the pg_dump.

Just wondering how tripal handles the roles & priveleges?

On 12/23/13, 8:27 AM, Stephen Ficklin wrote:

> Hi Alex,
>
> The migration documentation you followed had some missing information
> for your case.  I have updated the documentation
> (http://tripal.info/documentation/migrate_chado) to provide some
> additional instructions.  The problem is that Tripal installs some
> tables such as materialized views (e.g. organism_feature_count,
> kegg_by_organism) into the Chado schema when the modules are installed.
> Because Chado was migrated after the Tripal modules were installed those
> tables are missing.
>
> So here's how you can correct the problem:
>
> 1) Disable (not uninstall) the Tripal modules (do not uninstall them yet)
> 2) Rename the 'chado' schema to something else (e.g. 'chado_temp').  We
> do this to avoid any side-effects when uninstalling the Tripal modules.
> 3) Now, uninstall the tripal modules.  You may get some errors about
> missing tables, but as long as the module is uninstalled afterwords
> there should not be a problem later.
> 3) Rename the 'chado_temp' schema back to 'chado'.
> 4) Enable the Tripal modules in the order described in the installation
> instructions: http://gmod.org/wiki/Tripal_Tutorial_v1.1#Installation
>
> Let me know how it goes,
> Stephen
>

------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal
Reply | Threaded
Open this post in threaded view
|

Re: loading chado database into existing tripal deployment

Stephen Ficklin-2
Hi Alex,

Thanks for the suggestion.  I have updated the documentation to include
your suggestion (http://tripal.info/documentation/migrate_chado).

Tripal itself expects to have access to all tables, sequences, views,
etc. that are part of Chado via the same user account that is used for
the Drupal database (set in the $db_url variable of the settings.php
file).   In all of the Tripal installations we use we do not have any
specific ACLs other than the owner.  All of the Drupal and Chado tables
were owned by the same user with no other ACLs set.   So we hadn't
encountered the problem you had when dumping and restoring the Chado
database.

Thanks,
Stephen

On 12/27/2013 11:45 AM, Alex Rice wrote:

> Thanks Stephen, that is helpful. Working through it...
> I was also wondering if the pg_dump options should be
>    --no-owner --no-acl?
>
> Because if I to load a chado dump into an empty postgres database, I get
> errors about missing roles when granting privs for chado, chado_user,
> chado_read and pgsql. Unless I add the --no-acl to the pg_dump.
>
> Just wondering how tripal handles the roles & priveleges?
>
> On 12/23/13, 8:27 AM, Stephen Ficklin wrote:
>> Hi Alex,
>>
>> The migration documentation you followed had some missing information
>> for your case.  I have updated the documentation
>> (http://tripal.info/documentation/migrate_chado) to provide some
>> additional instructions.  The problem is that Tripal installs some
>> tables such as materialized views (e.g. organism_feature_count,
>> kegg_by_organism) into the Chado schema when the modules are installed.
>> Because Chado was migrated after the Tripal modules were installed those
>> tables are missing.
>>
>> So here's how you can correct the problem:
>>
>> 1) Disable (not uninstall) the Tripal modules (do not uninstall them yet)
>> 2) Rename the 'chado' schema to something else (e.g. 'chado_temp').  We
>> do this to avoid any side-effects when uninstalling the Tripal modules.
>> 3) Now, uninstall the tripal modules.  You may get some errors about
>> missing tables, but as long as the module is uninstalled afterwords
>> there should not be a problem later.
>> 3) Rename the 'chado_temp' schema back to 'chado'.
>> 4) Enable the Tripal modules in the order described in the installation
>> instructions: http://gmod.org/wiki/Tripal_Tutorial_v1.1#Installation
>>
>> Let me know how it goes,
>> Stephen
>>
> ------------------------------------------------------------------------------
> Rapidly troubleshoot problems before they affect your business. Most IT
> organizations don't have a clear picture of how application performance
> affects their revenue. With AppDynamics, you get 100% visibility into your
> Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
> _______________________________________________
> Gmod-tripal mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-tripal


------------------------------------------------------------------------------
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal