[Gmod-ajax] using multiple sources

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[Gmod-ajax] using multiple sources

Adam Witney
Hi,

Is it possible to use multiple source genomes from a single jbrowse installation?

I have setup a config file for each genome and can run

bin/biodb-to-json.pl --conf config1.js --ref genome1
bin/biodb-to-json.pl --conf config2.js --ref genome2

ok. But I can't seem to be able the second one up in the browser. Is there a specific way I should approach this? I couldn't see anything on the wiki about this

thanks for any help

adam
------------------------------------------------------------------------------

_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-ajax] using multiple sources

Mitch Skinner
On 05/28/2010 07:36 AM, Adam Witney wrote:
> Is it possible to use multiple source genomes from a single jbrowse installation?
>
> I have setup a config file for each genome and can run
>
> bin/biodb-to-json.pl --conf config1.js --ref genome1
> bin/biodb-to-json.pl --conf config2.js --ref genome2
>
> ok. But I can't seem to be able the second one up in the browser. Is there a specific way I should approach this? I couldn't see anything on the wiki about this
>    

Aside: If you have multiple data sources that all describe the same
genome (say, a chado database, and a GFF file, and a wiggle file, all of
which are about the same set of reference sequences) then you can
certainly put them into the same JBrowse instance, like you were trying
to do.

It doesn't sound like that applies to you, but it's an FAQ, so I thought
I'd mention it.

One assumption that JBrowse makes is that each reference sequence has a
unique name, so if your two genomes both have, say, a "chr1" refseq,
then there will be confusion.   So, if each genome has multiple refseqs
(i.e., multiple chromosomes), then usually you want to have one genome
per JBrowse "instance"; an instance is one JBrowse "data" directory
containing the JSON files generated by biodb-to-json.pl and the other
JBrowse scripts.

As usual, the best thing you can do at the moment comes down to what
you're trying to accomplish.

If you make sure that each of the reference sequences has a unique name
(say, if you have multiple species that each only have one reference
sequence, like viruses) then you can have all your genomes in one
JBrowse instance.  All you have to do is make sure that run
bin/prepare-refseqs.pl gets run for all of the ref seqs.

Specifically, start out by doing

$ bin/prepare-refseqs.pl --conf config1.js --refs genome1
$ bin/prepare-refseqs.pl --conf config2.js --refs genome2

and then do the bin/biodb-to-json.pl commands you were doing (you should
be able to leave the "--refs" argument off of the biodb-to-json.pl
commands, if you only have one ref per genome).


On the other hand, if you're working with species with multiple
chromosomes, one simple thing you can do is have a whole separate
JBrowse download for each genome.  It takes up a little more space, but
the space is usually small relative to the size of the genome data.

If you have lots of different genomes and you're concerned about the
space taken up by having separate sets of the JBrowse code, then one
thing you can do is have a separate "data" directory for each genome,
but sharing the same JBrowse code.  Then you'd have to copy the
index.html file and edit these two lines:

<script type="text/javascript" src="data/refSeqs.js"></script>
<script type="text/javascript" src="data/trackInfo.js"></script>

to refer to the specific data directories.

Specifically, I mean doing something like this:

$ bin/prepare-refseqs.pl --conf config1.js --refs chr1,chr2,chr3 --out
genome1-data
$ bin/biodb-to-json.pl --conf config1.js --out genome1-data
$ cp index.html genome1.html

and edit genome1.html so that those two lines above look like this:

<script type="text/javascript" src="genome1-data/refSeqs.js"></script>
<script type="text/javascript" src="genome1-data/trackInfo.js"></script>

And then you'd do the same thing for genome2, with a new genome2-data
directory and a new genome2.html file.

Then you could link to each of those html files separately for each genome.

Comparing two genomes against each other on the same screen in a
synteny-style view is something we're keen to do in the JBrowse renewal,
but it's not currently implemented.

Does that get you where you want to go?

Mitch

------------------------------------------------------------------------------

_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-ajax] using multiple sources

Brenton Graveley
We have very successfully used the method Mitch mentioned of downloading/installing a separate JBrowse instance to do this for several different species (human, mouse, fly, etc.) and simply give the JBrowse directories different names (JBrowse/Human, JBrowse/Mouse, etc.) and it works great.  I agree, with Mitch that the space used up by JBrowse code is minimal with respect to the genome data.  

Brent


On May 28, 2010, at 11:27 AM, Mitch Skinner wrote:

> On 05/28/2010 07:36 AM, Adam Witney wrote:
>> Is it possible to use multiple source genomes from a single jbrowse installation?
>>
>> I have setup a config file for each genome and can run
>>
>> bin/biodb-to-json.pl --conf config1.js --ref genome1
>> bin/biodb-to-json.pl --conf config2.js --ref genome2
>>
>> ok. But I can't seem to be able the second one up in the browser. Is there a specific way I should approach this? I couldn't see anything on the wiki about this
>>
>
> Aside: If you have multiple data sources that all describe the same
> genome (say, a chado database, and a GFF file, and a wiggle file, all of
> which are about the same set of reference sequences) then you can
> certainly put them into the same JBrowse instance, like you were trying
> to do.
>
> It doesn't sound like that applies to you, but it's an FAQ, so I thought
> I'd mention it.
>
> One assumption that JBrowse makes is that each reference sequence has a
> unique name, so if your two genomes both have, say, a "chr1" refseq,
> then there will be confusion.   So, if each genome has multiple refseqs
> (i.e., multiple chromosomes), then usually you want to have one genome
> per JBrowse "instance"; an instance is one JBrowse "data" directory
> containing the JSON files generated by biodb-to-json.pl and the other
> JBrowse scripts.
>
> As usual, the best thing you can do at the moment comes down to what
> you're trying to accomplish.
>
> If you make sure that each of the reference sequences has a unique name
> (say, if you have multiple species that each only have one reference
> sequence, like viruses) then you can have all your genomes in one
> JBrowse instance.  All you have to do is make sure that run
> bin/prepare-refseqs.pl gets run for all of the ref seqs.
>
> Specifically, start out by doing
>
> $ bin/prepare-refseqs.pl --conf config1.js --refs genome1
> $ bin/prepare-refseqs.pl --conf config2.js --refs genome2
>
> and then do the bin/biodb-to-json.pl commands you were doing (you should
> be able to leave the "--refs" argument off of the biodb-to-json.pl
> commands, if you only have one ref per genome).
>
>
> On the other hand, if you're working with species with multiple
> chromosomes, one simple thing you can do is have a whole separate
> JBrowse download for each genome.  It takes up a little more space, but
> the space is usually small relative to the size of the genome data.
>
> If you have lots of different genomes and you're concerned about the
> space taken up by having separate sets of the JBrowse code, then one
> thing you can do is have a separate "data" directory for each genome,
> but sharing the same JBrowse code.  Then you'd have to copy the
> index.html file and edit these two lines:
>
> <script type="text/javascript" src="data/refSeqs.js"></script>
> <script type="text/javascript" src="data/trackInfo.js"></script>
>
> to refer to the specific data directories.
>
> Specifically, I mean doing something like this:
>
> $ bin/prepare-refseqs.pl --conf config1.js --refs chr1,chr2,chr3 --out
> genome1-data
> $ bin/biodb-to-json.pl --conf config1.js --out genome1-data
> $ cp index.html genome1.html
>
> and edit genome1.html so that those two lines above look like this:
>
> <script type="text/javascript" src="genome1-data/refSeqs.js"></script>
> <script type="text/javascript" src="genome1-data/trackInfo.js"></script>
>
> And then you'd do the same thing for genome2, with a new genome2-data
> directory and a new genome2.html file.
>
> Then you could link to each of those html files separately for each genome.
>
> Comparing two genomes against each other on the same screen in a
> synteny-style view is something we're keen to do in the JBrowse renewal,
> but it's not currently implemented.
>
> Does that get you where you want to go?
>
> Mitch
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Gmod-ajax mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-ajax


------------------------------------------------------------------------------

_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-ajax] using multiple sources

Adam Witney
In reply to this post by Mitch Skinner

Hi Mitch,

thanks for your reply. I have multiple bacterial species and i wanted to have a new "view" for each ie each with their own list of tracks on the left hand side. This approach you suggested seems to have done the trick precisely:

> $ bin/prepare-refseqs.pl --conf config1.js --refs chr1,chr2,chr3 --out genome1-data
> $ bin/biodb-to-json.pl --conf config1.js --out genome1-data
> $ cp index.html genome1.html
>
> and edit genome1.html so that those two lines above look like this:
>
> <script type="text/javascript" src="genome1-data/refSeqs.js"></script>
> <script type="text/javascript" src="genome1-data/trackInfo.js"></script>

Although I now cannot seem to get the name searching to work. I have run

bin/generate-names.pl --dir genome1

and the genome1/names does contain the *.json files. I did notice that when i reload the page I get an error in the apache logs:

"File does not exist: /usr/local/apache2/htdocs/jbrowse/data"

and this in firebug:

"Error: Unable to load data/names/root.json status:404"

so it must still be looking for the "data" directory at some point. Do i have to change any other settings to use a non-default data directory?

thanks

adam
------------------------------------------------------------------------------

_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-ajax] using multiple sources

Mitch Skinner
On 05/28/2010 09:30 AM, Adam Witney wrote:

> Although I now cannot seem to get the name searching to work. I have run
> bin/generate-names.pl --dir genome1
>
> and the genome1/names does contain the *.json files. I did notice that when i reload the page I get an error in the apache logs:
>
> "File does not exist: /usr/local/apache2/htdocs/jbrowse/data"
>
> and this in firebug:
>
> "Error: Unable to load data/names/root.json status:404"
>
> so it must still be looking for the "data" directory at some point. Do i have to change any other settings to use a non-default data directory?
>    

Right, I forgot about this part.  There is a setting (the "dataRoot"
option to the Browser constructor), but looking at it just now, it won't
quite work in the setup that we've been discussing (it was originally
aimed at a slightly different setup; it's used by the TWiki plugin).

So, right now you've got two options:

1. have separate JBrowse directories will both code and data, or
2. wait a few days or a week for us to fix up the dataRoot setting to be
more flexible.

If you choose option 1, and if you originally got JBrowse using git,
then you can save some disk space by using "git clone".  For example, if
you originally did a "git clone" to create a "jbrowse" directory, you
can clone from there to a new directory, and the clone will save some
disk space (by using hard-linking).

$ git clone jbrowse genome1
$ git clone jbrowse genome2

will give you new "genome1" and "genome2" directories that don't use
much extra disk space.

Sorry about that,
Mitch



------------------------------------------------------------------------------

_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-ajax] using multiple sources

Adam Witney

On 28 May 2010, at 19:13, Mitch Skinner wrote:

> On 05/28/2010 09:30 AM, Adam Witney wrote:
>> Although I now cannot seem to get the name searching to work. I have run
>> bin/generate-names.pl --dir genome1
>>
>> and the genome1/names does contain the *.json files. I did notice that when i reload the page I get an error in the apache logs:
>>
>> "File does not exist: /usr/local/apache2/htdocs/jbrowse/data"
>>
>> and this in firebug:
>>
>> "Error: Unable to load data/names/root.json status:404"
>>
>> so it must still be looking for the "data" directory at some point. Do i have to change any other settings to use a non-default data directory?
>>  
>
> Right, I forgot about this part.  There is a setting (the "dataRoot" option to the Browser constructor), but looking at it just now, it won't quite work in the setup that we've been discussing (it was originally aimed at a slightly different setup; it's used by the TWiki plugin).
>
> So, right now you've got two options:
>
> 1. have separate JBrowse directories will both code and data, or
> 2. wait a few days or a week for us to fix up the dataRoot setting to be more flexible.
>
> If you choose option 1, and if you originally got JBrowse using git, then you can save some disk space by using "git clone".  For example, if you originally did a "git clone" to create a "jbrowse" directory, you can clone from there to a new directory, and the clone will save some disk space (by using hard-linking).
>
> $ git clone jbrowse genome1
> $ git clone jbrowse genome2
>
> will give you new "genome1" and "genome2" directories that don't use much extra disk space.
>
> Sorry about that,
> Mitch

ok thanks for the tips Mitch, I will try those options

thanks

adam



------------------------------------------------------------------------------

_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: using multiple sources

Mitch Skinner
Hi, sorry it took so long to get back to you; I had a big deadline that
I had to focus on for a while instead.

I spent some time looking at this; I was thinking of changing the
behavior of the dataRoot setting.  But I realized that that would
require people to re-generate their JBrowse JSON.  So don't think we
want to do that in the current "master" branch, although it might make
sense to do it in the current development ("lazyfeatures") branch.

So, assuming you're using the master branch, then there's one more
option that I forgot to mention in my last email, that should make the
name searching work, without requiring you to have multiple copies of
the JBrowse code.  It's a bit awkward, though.

Earlier, I suggested that you have multiple data directories, with
separate html files that pointed to them, something like this:

jbrowse/
    genome1-data/
    genome2-data/
    genome1.html
    genome2.html

The awkward thing that I forgot to suggest earlier is this: take those
genome1-data and genome2-data directories, and rename them and move them
around so that they look like this instead:

jbrowse/
    genome1.html
    genome2.html

genome1/
    data/

genome2/
    data/

That is, take those data directories and make them into directories
named "data", with uniquely named parent directories.  This is because
that "data" string (unfortunately) shows up in several places both in
the JBrowse code and also in the generated JSON.

Then update genome1.html and genome2.html so that the refSeqs.js and
trackInfo.js links point to where you moved those data directories; e.g.:

<script type="text/javascript" src="../genome1/data/refSeqs.js"></script>
<script type="text/javascript" src="../genome1/data/trackInfo.js"></script>


And then add the "dataRoot" option to the Browser constructor in
genome1.html and genome2.html, so that it looks something like this:

            var b = new Browser({
                                    containerID: "GenomeBrowser",
                                    refSeqs: refSeqs,
                                    trackData: trackInfo,
                                    defaultTracks:
"DNA,gene,mRNA,noncodingRNA",
                                    location: queryParams.loc,
                                    tracks: queryParams.tracks,
                                    bookmark: bookmarkCallback,
                                    dataRoot: "../genome1"
                                });


and that ought to do it.  I haven't tested the specific recommendations
in this email, though, so do let me know if it works for you.

Mitch


On 06/01/2010 08:17 AM, Adam Witney wrote:

> On 28 May 2010, at 19:13, Mitch Skinner wrote:
>    
>> So, right now you've got two options:
>>
>> 1. have separate JBrowse directories will both code and data, or
>> 2. wait a few days or a week for us to fix up the dataRoot setting to be more flexible.
>>
>> If you choose option 1, and if you originally got JBrowse using git, then you can save some disk space by using "git clone".  For example, if you originally did a "git clone" to create a "jbrowse" directory, you can clone from there to a new directory, and the clone will save some disk space (by using hard-linking).
>>
>> $ git clone jbrowse genome1
>> $ git clone jbrowse genome2
>>
>> will give you new "genome1" and "genome2" directories that don't use much extra disk space.
>>
>> Sorry about that,
>> Mitch
>>      
> ok thanks for the tips Mitch, I will try those options
>
> thanks
>
> adam
>
>
>    


------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax