[Gmod-ajax] Importing features from GFF3 file

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[Gmod-ajax] Importing features from GFF3 file

Raymond Wan-2
Dear all,

Sorry for such a basic question, but I'm trying to import features
from a GFF3 file using flatfile-to-json.pl.  According to:

http://gmod.org/wiki/JBrowseDev/The_SubfeatureClasses_Option

The SubfeatureClasses option is a list of pairs of keys and values.
The values are (I think) the CSS class options from
http://gmod.org/wiki/JBrowseDev/The_CssClass_Option .

Can I assume that the key is taken from the third column of the GFF3
file (i.e., the "type")? If so, are all types from the GFF3 read in
and, thus, I can edit the trackList.json later to tweak things.  Or,
if I don't list it on the command-line with flatfile-to-json.pl, the
feature gets silently dropped?

Thank you!

Ray

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: Importing features from GFF3 file

Colin
Hi Raymond,
These are good questions. I would check out the flatfile-to-json.pl --help option for a little extra detail on the arguments. For example, here's the one for subfeatureClasses

       --subfeatureClasses '{ JSON-format subfeature class map }'
           CSS classes for each subfeature type, in JSON syntax.  Example:

             --subfeatureClasses '{"CDS": "transcript-CDS", "exon": "transcript-exon"}'

           Only used by HTMLFeatures tracks.The subfeatureClasses concept is a particle


You can see this argument is basically only for HTMLFeatures type tracks, and the reason is that it is for mapping a CSS class to the subfeatures of your loaded track (i.e. specific CSS for your CDS or UTR). If you are more concerned about the type that is being loaded, you would use the --type argument to flatfile-to-json.pl. The --type argument is for loading the specific types from column 3, and the HTMLFeatures tracks prefer to load the "transcript" level, so it is best to use --type mRNA or --type transcript depending on your GFF. Example from docs:

       --type <feature types to process>
           Only process features of the given type.  Can take either single
           type names, e.g. "mRNA", or type names qualified by "source" name,
           for whatever definition of "source" your data file might have.  For
           example, "mRNA:exonerate" will filter for only mRNA features that
           have a source of "exonerate".

           Multiple type names can be specified by separating the type names
           with commas, e.g. "--type mRNA:exonerate,ncRNA". for example. For CanvasFeatures, which has newer capability


Also, this is a random aside, but it is worth noting that the HTMLFeatures is the default, but the CanvasFeatures is actually sort of the "newer" track type, and it can be enabled using --trackType CanvasFeatures argument to flatfile-to-json.pl

Hope that helps

-Colin

On Wed, May 20, 2015 at 11:57 PM, Raymond Wan <[hidden email]> wrote:
Dear all,

Sorry for such a basic question, but I'm trying to import features
from a GFF3 file using flatfile-to-json.pl.  According to:

http://gmod.org/wiki/JBrowseDev/The_SubfeatureClasses_Option

The SubfeatureClasses option is a list of pairs of keys and values.
The values are (I think) the CSS class options from
http://gmod.org/wiki/JBrowseDev/The_CssClass_Option .

Can I assume that the key is taken from the third column of the GFF3
file (i.e., the "type")? If so, are all types from the GFF3 read in
and, thus, I can edit the trackList.json later to tweak things.  Or,
if I don't list it on the command-line with flatfile-to-json.pl, the
feature gets silently dropped?

Thank you!

Ray

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: Importing features from GFF3 file

Raymond Wan-2
Hi Colin,


On Fri, May 22, 2015 at 1:23 AM, Colin <[hidden email]> wrote:

> Hi Raymond,
> These are good questions. I would check out the flatfile-to-json.pl --help
> option for a little extra detail on the arguments. For example, here's the
> one for subfeatureClasses
>
>        --subfeatureClasses '{ JSON-format subfeature class map }'
>            CSS classes for each subfeature type, in JSON syntax.  Example:
>
>              --subfeatureClasses '{"CDS": "transcript-CDS", "exon":
> "transcript-exon"}'


Thank you for clarifying this for me!

I think I got mixed up between the "type" (as stored in the GFF3 file)
and the visual presentation of types by JBrowse.  In short, I just
want to grab the entire GFF3 file and then fiddle with the
presentation (i.e., trackLists.json) later since reading in the GFF3
file takes time.  So, it sounds like to achive this, I would use
--type?  Got it; thanks!

> Also, this is a random aside, but it is worth noting that the HTMLFeatures
> is the default, but the CanvasFeatures is actually sort of the "newer" track
> type, and it can be enabled using --trackType CanvasFeatures argument to
> flatfile-to-json.pl


Oh...  I was not aware of this.  Thanks for this very important aside!

I've been digging through the documentation trying to catch up, after
being away from JBrowse for 1-2 years.  A lot of new features to get
use to!

So, in the tutorial, would this example be incorrect:

http://gmod.org/wiki/JBrowse_Tutorial_PAG_2015#Features_from_GFF3_or_BED_files

The "flatfile-to-json.pl" command has both --trackType CanvasFeatures
and --subfeatureClasses together.  Is subfeatureClasses here
redundant?

Thanks a lot!

Ray

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: Importing features from GFF3 file

Raymond Wan-2

Hi Colin (or anyone else who knows?),


On Friday, May 22, 2015 11:02 AM, Raymond Wan wrote:

> On Fri, May 22, 2015 at 1:23 AM, Colin <[hidden email]> wrote:
>> Hi Raymond,
>> These are good questions. I would check out the flatfile-to-json.pl --help
>> option for a little extra detail on the arguments. For example, here's the
>> one for subfeatureClasses
>>
>>         --subfeatureClasses '{ JSON-format subfeature class map }'
>>             CSS classes for each subfeature type, in JSON syntax.  Example:
>>
>>               --subfeatureClasses '{"CDS": "transcript-CDS", "exon":
>> "transcript-exon"}'
>
>
> Thank you for clarifying this for me!
>
> I think I got mixed up between the "type" (as stored in the GFF3 file)
> and the visual presentation of types by JBrowse.  In short, I just
> want to grab the entire GFF3 file and then fiddle with the
> presentation (i.e., trackLists.json) later since reading in the GFF3
> file takes time.  So, it sounds like to achive this, I would use
> --type?  Got it; thanks!


So, I imported the GFF3 features in using --type
CDS,exon,gene,...<heaps more> without any errors.  However,
when I load the track in, I'm getting errors in both Firefox
and Chrome about the browser being unresponsive.  I'm
guessing that the problem is that loading a single GFF file
into one track is making the browser do a lot of work for a
given region of the chromosome.

 From a performance perspective, would there be an advantage
for the user to change what I've done to multiple calls of
flatfile-to-json.pl?

i.e.,

flatfile-to-json.pl --key "CDS" --type CDS --gff
some-GFF-file ...

flatfile-to-json.pl --key "exon" --type exon --gff
same-GFF-file ...

flatfile-to-json.pl --key "gene" --type gene --gff
same-GFF-file ...

At the very least, the data will end up in different
subdirectories?

Am I on the right track?  Is this what I should be looking
into for a non-responsive JBrowse?  (I'm using the latest
1.11.6 version.)

Thank you!

Ray




------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: Importing features from GFF3 file

Colin
Normally if you load a gene, it will include all the subfeatures too so it is normally not necessary to load exon/CDS separately (unless that is of particular interest to see those features on separate tracks).

Also, in terms of the browser, freezing, you might try using CanvasFeatures because it will be much more efficient for drawing, and can reduce browser freezing

Maybe just try "flatfile-to-json.pl --key "gene" --type gene --gff same-GFF-file --trackType CanvasFeatures --trackLabel mygenetrack" and see how that works?


-Colin

On Fri, May 22, 2015 at 11:14 AM, Raymond Wan <[hidden email]> wrote:

Hi Colin (or anyone else who knows?),


On Friday, May 22, 2015 11:02 AM, Raymond Wan wrote:
On Fri, May 22, 2015 at 1:23 AM, Colin <[hidden email]> wrote:
Hi Raymond,
These are good questions. I would check out the flatfile-to-json.pl --help
option for a little extra detail on the arguments. For example, here's the
one for subfeatureClasses

        --subfeatureClasses '{ JSON-format subfeature class map }'
            CSS classes for each subfeature type, in JSON syntax.  Example:

              --subfeatureClasses '{"CDS": "transcript-CDS", "exon":
"transcript-exon"}'


Thank you for clarifying this for me!

I think I got mixed up between the "type" (as stored in the GFF3 file)
and the visual presentation of types by JBrowse.  In short, I just
want to grab the entire GFF3 file and then fiddle with the
presentation (i.e., trackLists.json) later since reading in the GFF3
file takes time.  So, it sounds like to achive this, I would use
--type?  Got it; thanks!


So, I imported the GFF3 features in using --type CDS,exon,gene,...<heaps more> without any errors.  However, when I load the track in, I'm getting errors in both Firefox and Chrome about the browser being unresponsive.  I'm guessing that the problem is that loading a single GFF file into one track is making the browser do a lot of work for a given region of the chromosome.

>From a performance perspective, would there be an advantage for the user to change what I've done to multiple calls of flatfile-to-json.pl?

i.e.,

flatfile-to-json.pl --key "CDS" --type CDS --gff some-GFF-file ...

flatfile-to-json.pl --key "exon" --type exon --gff same-GFF-file ...

flatfile-to-json.pl --key "gene" --type gene --gff same-GFF-file ...

At the very least, the data will end up in different subdirectories?

Am I on the right track?  Is this what I should be looking into for a non-responsive JBrowse?  (I'm using the latest 1.11.6 version.)

Thank you!

Ray





------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax
Reply | Threaded
Open this post in threaded view
|

Re: Importing features from GFF3 file

Raymond Wan-2
In reply to this post by Colin
Hi Colin and everyone,


On Sat, May 23, 2015 at 4:16 AM, Colin <[hidden email]> wrote:

> Normally if you load a gene, it will include all the subfeatures too so it
> is normally not necessary to load exon/CDS separately (unless that is of
> particular interest to see those features on separate tracks).
>
> Also, in terms of the browser, freezing, you might try using CanvasFeatures
> because it will be much more efficient for drawing, and can reduce browser
> freezing
>
> Maybe just try "flatfile-to-json.pl --key "gene" --type gene --gff
> same-GFF-file --trackType CanvasFeatures --trackLabel mygenetrack" and see
> how that works?


Thanks for the information and sorry if it's located somewhere in the
documentation -- I wasn't aware that if I load "gene" from the GFF
file, all parts of it (I presume it's all lines that have that gene as
a "parent" in the last column of the GFF file?) are loaded too.  I
thought I had to explicitly specify each part.  Silly me!

So, I loaded all of the genes using CanvasFeatures and have even tried
with compression and without.  It seems *with* compression, the
browser freezes less but I'm not sure if it's all "in my head".
(i.e., the difference was small)

With compression, if I click "Continue" a few times in Firefox, the
features do show up at some stage.  It seems I can ask users to make
Firefox wait longer [1], but that doesn't seem like a good idea.

Generally, what is the cause of this?  Is it the amount of data that
the server is providing?  (This is why I thought compression would
help...  Now I think it just saves disk space since the server is
decompressin on-the-fly?)  Or is it the amount of work that the
browser is doing?

Are there any settings I can look into to minimize this from occuring?

Ray

PS:  Thanks with the help with importing!  I think it's fine -- I just
need to play with the glyphs being used a bit since they seemto be all
yellow bars, by default.  I hope I haven't done anything wrong...


[1] https://support.mozilla.org/en-US/kb/warning-unresponsive-script

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gmod-ajax mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-ajax