Precompiled chado databases

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Precompiled chado databases

Eric Rasche

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chado server admins,

I'm not aware of how many of you regularly have to configure and deploy chado databases, however over time I've found that a significant portion of my time was spent waiting on ontologies to load.

To this end (and in support of some other projects), I set up a jenkins job to

- - clone chado from SVN
- - build
- - load ontologies
- - dump database as sql
- - upload to a publicly accessible webserver

This is done on a weekly basis. I'm making the compiled databases with all 5 ontologies loaded available to the community. They're trivial to use if you're just trying to install chado, as all you need to do is change the username in the precompiled schema

sed -i "s/schema_builder/user_name/g" csb.sql

and load it into the database of your choice.

psql -U {user_name} -d {desintation_db} -f csb.sql

This process takes ~10 seconds, compared to the 1.3 hours (according to my jenkins job) for regular ontology loading.

Hope someone else finds this helpful! Please find the precompiled schema below:

https://cpt.tamu.edu/~docs/chado/

Installed ontologies:

[1] Relationship Ontology
[2] Sequence Ontology
[3] Gene Ontology
[4] Chado Feature Properties
[5] Plant Ontology

If anyone is interested in this, and would like:

- - other builds with different (sub)sets of ontologies
- - archived copies of schemas over time

please feel free to request them and we'll add them to the jenkins job/make them available.

NB: The caveats of this method are, of course, that none of the scripts are installed, and the gmod conf files are not created in GMOD_ROOT. However, for remote access (e.g., via Artemis), and tools that do not make use of GMOD_ROOT locally, this seems to be fine.

Cheers,
Eric

- --
Eric Rasche
Programmer II
Center for Phage Technology
Texas A&M University
College Station, TX 77843
404-692-2048 <tel:4046922048>
[hidden email] [hidden email]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJS8mWsAAoJEMqDXdrsMcpVFgsP/0sN+yDvQuqEot2vZ0q03KK1
swLkYV33GAks7n4x1nHxdnf4oD2HTvabw3WoCbl/qroa3wUiqt162ZSxPyq/fBxx
GbdEHk9yFCXF/EWgZjlfcGQfqr0ac38zyUX4gK/UteZVEbQYnphOQhch/khU9zH8
m1t3k46mIGIMd/UJaEVaekowI23Zv9lEeY3YEqBgUxTRxJ+4Usn2zRp7xeObiZN1
pUPbYiZX/zOdJoTr9glrYNZY7T5zssBwi1V5STx/mPjt7R+qeKsJ1zQgdKZp3ebd
qLW33JaMlqNbBLBw2hDkuXs/nOEmyK1r5nDO/UCKl8AOgiBZCHiQPPUeF5gM8YZv
9BZeZmhWNOOm6wYN/QTt9XSYj5kygChxlMDpXRIxSTF1sDd7y3XtbtdEzJ3fVsuN
18KvHCJJ4N5YreC0rV6SEkNqWqwH2P3MnhQmItzDG67WqMuAIK2FE11E9s6PH9DK
r5YTA8oyIPO5mz0cFjnhHBFi3e/uBq++GkGeVG87Qu2MaZ1Cg9aFIR8mNjAUYwTn
55EkTe4pnWOf7dUPF/23DcTq6V5aSULSJTfWme+tqJ7ndfDcggMJVLf1VQU8Jaq2
iWVZC+SdmkVulR5PUpHfucEglphjgcIVT0LYi4H4bDMT8ZsEUtBTpqZ+vzGFhzy9
tyCYdwhkTDLCAuwajiPX
=63RQ
-----END PGP SIGNATURE-----


------------------------------------------------------------------------------
Managing the Performance of Cloud-Based Applications
Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
Read the Whitepaper.
http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-schema mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-schema
Reply | Threaded
Open this post in threaded view
|

Re: Precompiled chado databases

Siddhartha Basu
Hi Eric,
Interesting project. Could you add a wiki page for this project in
gmod.org, might be something look into it in the future.

thanks,
-sidd

On Wed, 05 Feb 2014, Eric Rasche wrote:

>    -----BEGIN PGP SIGNED MESSAGE-----
>    Hash: SHA1
>
>    Chado server admins,
>
>    I'm not aware of how many of you regularly have to configure and deploy
>    chado databases, however over time I've found that a significant portion
>    of my time was spent waiting on ontologies to load.
>
>    To this end (and in support of some other projects), I set up a jenkins
>    job to
>
>    - - clone chado from SVN
>    - - build
>    - - load ontologies
>    - - dump database as sql
>    - - upload to a publicly accessible webserver
>
>    This is done on a weekly basis. I'm making the compiled databases with all
>    5 ontologies loaded available to the community. They're trivial to use if
>    you're just trying to install chado, as all you need to do is change the
>    username in the precompiled schema
>
>    sed -i "s/schema_builder/user_name/g" csb.sql
>
>    and load it into the database of your choice.
>
>    psql -U {user_name} -d {desintation_db} -f csb.sql
>
>    This process takes ~10 seconds, compared to the 1.3 hours (according to my
>    jenkins job) for regular ontology loading.
>
>    Hope someone else finds this helpful! Please find the precompiled schema
>    below:
>
>    https://cpt.tamu.edu/~docs/chado/
>
>    Installed ontologies:
>
>    [1] Relationship Ontology
>    [2] Sequence Ontology
>    [3] Gene Ontology
>    [4] Chado Feature Properties
>    [5] Plant Ontology
>
>    If anyone is interested in this, and would like:
>
>    - - other builds with different (sub)sets of ontologies
>    - - archived copies of schemas over time
>
>    please feel free to request them and we'll add them to the jenkins
>    job/make them available.
>
>    NB: The caveats of this method are, of course, that none of the scripts
>    are installed, and the gmod conf files are not created in GMOD_ROOT.
>    However, for remote access (e.g., via Artemis), and tools that do not make
>    use of GMOD_ROOT locally, this seems to be fine.
>
>    Cheers,
>    Eric
>
>    - --
>    Eric Rasche
>    Programmer II
>    Center for Phage Technology
>    Texas A&M University
>    College Station, TX 77843
>    404-692-2048 <tel:4046922048>
>    [hidden email] <mailto:[hidden email]>
>    -----BEGIN PGP SIGNATURE-----
>    Version: GnuPG v1.4.11 (GNU/Linux)
>    Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
>    iQIcBAEBAgAGBQJS8mWsAAoJEMqDXdrsMcpVFgsP/0sN+yDvQuqEot2vZ0q03KK1
>    swLkYV33GAks7n4x1nHxdnf4oD2HTvabw3WoCbl/qroa3wUiqt162ZSxPyq/fBxx
>    GbdEHk9yFCXF/EWgZjlfcGQfqr0ac38zyUX4gK/UteZVEbQYnphOQhch/khU9zH8
>    m1t3k46mIGIMd/UJaEVaekowI23Zv9lEeY3YEqBgUxTRxJ+4Usn2zRp7xeObiZN1
>    pUPbYiZX/zOdJoTr9glrYNZY7T5zssBwi1V5STx/mPjt7R+qeKsJ1zQgdKZp3ebd
>    qLW33JaMlqNbBLBw2hDkuXs/nOEmyK1r5nDO/UCKl8AOgiBZCHiQPPUeF5gM8YZv
>    9BZeZmhWNOOm6wYN/QTt9XSYj5kygChxlMDpXRIxSTF1sDd7y3XtbtdEzJ3fVsuN
>    18KvHCJJ4N5YreC0rV6SEkNqWqwH2P3MnhQmItzDG67WqMuAIK2FE11E9s6PH9DK
>    r5YTA8oyIPO5mz0cFjnhHBFi3e/uBq++GkGeVG87Qu2MaZ1Cg9aFIR8mNjAUYwTn
>    55EkTe4pnWOf7dUPF/23DcTq6V5aSULSJTfWme+tqJ7ndfDcggMJVLf1VQU8Jaq2
>    iWVZC+SdmkVulR5PUpHfucEglphjgcIVT0LYi4H4bDMT8ZsEUtBTpqZ+vzGFhzy9
>    tyCYdwhkTDLCAuwajiPX
>    =63RQ
>    -----END PGP SIGNATURE-----

> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk

> _______________________________________________
> Gmod-schema mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-schema


------------------------------------------------------------------------------
Managing the Performance of Cloud-Based Applications
Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
Read the Whitepaper.
http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-schema mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-schema
Reply | Threaded
Open this post in threaded view
|

Re: Precompiled chado databases

Scott Cain
Hi Eric,

I agree with siddhartha: making a wiki page for this would be great. Thanks so much for making an automated tool that does this!  I think plenty of people could make use of this (for example, imagine a Tripal process that just grabbed the pre built database and loaded it!)

Scott


Sent from my iPhone

> On Feb 5, 2014, at 10:58 AM, Siddhartha Basu <[hidden email]> wrote:
>
> Hi Eric,
> Interesting project. Could you add a wiki page for this project in
> gmod.org, might be something look into it in the future.
>
> thanks,
> -sidd
>
>> On Wed, 05 Feb 2014, Eric Rasche wrote:
>>
>>   -----BEGIN PGP SIGNED MESSAGE-----
>>   Hash: SHA1
>>
>>   Chado server admins,
>>
>>   I'm not aware of how many of you regularly have to configure and deploy
>>   chado databases, however over time I've found that a significant portion
>>   of my time was spent waiting on ontologies to load.
>>
>>   To this end (and in support of some other projects), I set up a jenkins
>>   job to
>>
>>   - - clone chado from SVN
>>   - - build
>>   - - load ontologies
>>   - - dump database as sql
>>   - - upload to a publicly accessible webserver
>>
>>   This is done on a weekly basis. I'm making the compiled databases with all
>>   5 ontologies loaded available to the community. They're trivial to use if
>>   you're just trying to install chado, as all you need to do is change the
>>   username in the precompiled schema
>>
>>   sed -i "s/schema_builder/user_name/g" csb.sql
>>
>>   and load it into the database of your choice.
>>
>>   psql -U {user_name} -d {desintation_db} -f csb.sql
>>
>>   This process takes ~10 seconds, compared to the 1.3 hours (according to my
>>   jenkins job) for regular ontology loading.
>>
>>   Hope someone else finds this helpful! Please find the precompiled schema
>>   below:
>>
>>   https://cpt.tamu.edu/~docs/chado/
>>
>>   Installed ontologies:
>>
>>   [1] Relationship Ontology
>>   [2] Sequence Ontology
>>   [3] Gene Ontology
>>   [4] Chado Feature Properties
>>   [5] Plant Ontology
>>
>>   If anyone is interested in this, and would like:
>>
>>   - - other builds with different (sub)sets of ontologies
>>   - - archived copies of schemas over time
>>
>>   please feel free to request them and we'll add them to the jenkins
>>   job/make them available.
>>
>>   NB: The caveats of this method are, of course, that none of the scripts
>>   are installed, and the gmod conf files are not created in GMOD_ROOT.
>>   However, for remote access (e.g., via Artemis), and tools that do not make
>>   use of GMOD_ROOT locally, this seems to be fine.
>>
>>   Cheers,
>>   Eric
>>
>>   - --
>>   Eric Rasche
>>   Programmer II
>>   Center for Phage Technology
>>   Texas A&M University
>>   College Station, TX 77843
>>   404-692-2048 <tel:4046922048>
>>   [hidden email] <mailto:[hidden email]>
>>   -----BEGIN PGP SIGNATURE-----
>>   Version: GnuPG v1.4.11 (GNU/Linux)
>>   Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>>   iQIcBAEBAgAGBQJS8mWsAAoJEMqDXdrsMcpVFgsP/0sN+yDvQuqEot2vZ0q03KK1
>>   swLkYV33GAks7n4x1nHxdnf4oD2HTvabw3WoCbl/qroa3wUiqt162ZSxPyq/fBxx
>>   GbdEHk9yFCXF/EWgZjlfcGQfqr0ac38zyUX4gK/UteZVEbQYnphOQhch/khU9zH8
>>   m1t3k46mIGIMd/UJaEVaekowI23Zv9lEeY3YEqBgUxTRxJ+4Usn2zRp7xeObiZN1
>>   pUPbYiZX/zOdJoTr9glrYNZY7T5zssBwi1V5STx/mPjt7R+qeKsJ1zQgdKZp3ebd
>>   qLW33JaMlqNbBLBw2hDkuXs/nOEmyK1r5nDO/UCKl8AOgiBZCHiQPPUeF5gM8YZv
>>   9BZeZmhWNOOm6wYN/QTt9XSYj5kygChxlMDpXRIxSTF1sDd7y3XtbtdEzJ3fVsuN
>>   18KvHCJJ4N5YreC0rV6SEkNqWqwH2P3MnhQmItzDG67WqMuAIK2FE11E9s6PH9DK
>>   r5YTA8oyIPO5mz0cFjnhHBFi3e/uBq++GkGeVG87Qu2MaZ1Cg9aFIR8mNjAUYwTn
>>   55EkTe4pnWOf7dUPF/23DcTq6V5aSULSJTfWme+tqJ7ndfDcggMJVLf1VQU8Jaq2
>>   iWVZC+SdmkVulR5PUpHfucEglphjgcIVT0LYi4H4bDMT8ZsEUtBTpqZ+vzGFhzy9
>>   tyCYdwhkTDLCAuwajiPX
>>   =63RQ
>>   -----END PGP SIGNATURE-----
>
>> ------------------------------------------------------------------------------
>> Managing the Performance of Cloud-Based Applications
>> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
>> Read the Whitepaper.
>> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
>
>> _______________________________________________
>> Gmod-schema mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/gmod-schema
>
>
> ------------------------------------------------------------------------------
> Managing the Performance of Cloud-Based Applications
> Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> Read the Whitepaper.
> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
> _______________________________________________
> Gmod-schema mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-schema

------------------------------------------------------------------------------
Managing the Performance of Cloud-Based Applications
Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
Read the Whitepaper.
http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-schema mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-schema