adding the GO source

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

adding the GO source

JD Wong
Hi,

When I run project_build.pl, loading my 500,000 genes only takes 4 minutes, but loading the GO source (which references the most recent GO OBO release) takes 43 minutes consistently.  These sources are the first two, and are loaded in this order.  Is this wait typical? 

Thanks,
-JD

_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: adding the GO source

Richard Smith
The GO source uses a library from OBO-Edit to calculate transitive
closure on the ontology (following relationships through the tree).

This is quite memory intensive so it may be that you need to allocate
some more RAM.  I think for us it takes ten minutes or less.

If you have haven't already, you can set an ANT_OPTS environment
variable to control how much memory is allocated to the Java process.

See the Java section here: http://intermine.org/wiki/Prerequisites

I would say you want -Xmx2048m as a minimum on a dev machine, much
more if your running large builds of a production database.  Our
newest servers have 96GB RAM so we set the Java memory to around
32GB (it doesn't need that much but helps performance).

Cheers,
Richard.



On 24/10/2011 17:46, JD Wong wrote:

> Hi,
>
> When I run project_build.pl <http://project_build.pl>, loading my
> 500,000 genes only takes 4 minutes, but loading the GO source (which
> references the most recent GO OBO release) takes 43 minutes
> consistently.  These sources are the first two, and are loaded in this
> order.  Is this wait typical?
>
> Thanks,
> -JD
>
>
> _______________________________________________
> dev mailing list
> [hidden email]
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev