Re: [Gmod-phendiver] phenovis

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Gmod-phendiver] phenovis

seth redmond
It's been a while since the Hackathon and we've been doing some development at Vectorbase - coming up with a RESTful middleware and visualisation system for the Natural Diversity module.
        The usual caveats apply; this is still very much Under Development but I think it's solid enough to give you an idea of what I've been doing. This is the system we are going to use for Vectorbase, but it's general enough to be used for anything that uses the natural diversity module and could easily be extended to the rest of GMOD.

First off I've a JSON server retrieving data directly from the DB via ActiveRecord - a direct extension of the hackathon work. The JSON structure is still open to modification, but I've tried to keep it as clean as possible:
        http://funcgen.vectorbase.org/PopulationBETA/REST/project/141.json
        http://funcgen.vectorbase.org/PopulationBETA/REST/project/43/stocks.json
Code is in a branch on my github:
        https://github.com/sethnr/natdiv-rest

Then the real work: I'm calling it 'Phenovis' (because G-vis was already taken by google)
        https://github.com/sethnr/phenovis
I've started writing a javascript library to provide standard-ish visualisations based on the above JSON. The script uses jsonPATH (http://goessner.net/articles/JsonPath/) to find objects to iterate over, and variables of those objects, then it plots it via a few protovis classes. For examples of the map visualisations (gmap with a protovis overlay - the pie charts will dynamically re-generate depending on your zoom level) see:  
        http://funcgen.vectorbase.org/PopulationBETA/Project/141       
Grouped bar chart and dotplots are also available, and can be customised with, eg, linear or categorical axes, etc:
        http://funcgen.vectorbase.org/PopulationBETA/project/?id=43

It's worth pointing out that these are actually running off a completely separate instance, ajax calls are made to retrieve the data from the JSON server.
Config is currently hidden in a string of JSON in a projectprop, however there's nothing stopping you rolling your own. There's a sandbox if you want to take a look at the config format or have a play around.
        http://funcgen.vectorbase.org/PopulationBETA/project/sandbox/?id=141
You can also see the same data under alternative visualisations by swapping 'type=geoplot' for type=dot, etc.

Finally, we've also put together a couple of test sets to show how easily this can be used to combine data from different sources; grabbing JSON from 3rd parties concatenating with our data and pushing it through the same visualisations. E.g. our data + a plasmodium db (slow-ish, it's making multiple ajax calls to find geolocations):
        http://funcgen.vectorbase.org/PopulationBETA/xquery/plasmodb/

Obviously I'm hoping this will become more polished over the next few months, but I'm throwing it out there in case it is useful to anyone. And if anyone wants more information or, even better, wants to assist in the development let me know.

-s


--
Seth Redmond
 Scientific Programmer, VectorBase
 Kafatos / Christophides Groups
 Div. Cell and Molecular Biology
 Imperial College, London
[hidden email]
--



------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric
Ries, the creator of the Lean Startup Methodology on "Lean Startup
Secrets Revealed." This video shows you how to validate your ideas,
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Gmod-phendiver mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-phendiver
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Gmod-phendiver] phenovis

Yuri Bendana-3
Nice work Seth!  You've made great progress on the visualizations since the hackathon.  I remember we discussed having a wiki page for describing the phenovis JSON structure, is there one? I still think it's a good idea since it'll be clearer for someone new than looking at the Rails code or raw JSON.  It'll also be a good place to discuss any extensions to the structure needed to support other types of visualizations.

yuri

On Thu, Jul 14, 2011 at 9:59 AM, seth redmond <[hidden email]> wrote:
It's been a while since the Hackathon and we've been doing some development at Vectorbase - coming up with a RESTful middleware and visualisation system for the Natural Diversity module.
       The usual caveats apply; this is still very much Under Development but I think it's solid enough to give you an idea of what I've been doing. This is the system we are going to use for Vectorbase, but it's general enough to be used for anything that uses the natural diversity module and could easily be extended to the rest of GMOD.

First off I've a JSON server retrieving data directly from the DB via ActiveRecord - a direct extension of the hackathon work. The JSON structure is still open to modification, but I've tried to keep it as clean as possible:
       http://funcgen.vectorbase.org/PopulationBETA/REST/project/141.json
       http://funcgen.vectorbase.org/PopulationBETA/REST/project/43/stocks.json
Code is in a branch on my github:
       https://github.com/sethnr/natdiv-rest

Then the real work: I'm calling it 'Phenovis' (because G-vis was already taken by google)
       https://github.com/sethnr/phenovis
I've started writing a javascript library to provide standard-ish visualisations based on the above JSON. The script uses jsonPATH (http://goessner.net/articles/JsonPath/) to find objects to iterate over, and variables of those objects, then it plots it via a few protovis classes. For examples of the map visualisations (gmap with a protovis overlay - the pie charts will dynamically re-generate depending on your zoom level) see:
       http://funcgen.vectorbase.org/PopulationBETA/Project/141
Grouped bar chart and dotplots are also available, and can be customised with, eg, linear or categorical axes, etc:
       http://funcgen.vectorbase.org/PopulationBETA/project/?id=43

It's worth pointing out that these are actually running off a completely separate instance, ajax calls are made to retrieve the data from the JSON server.
Config is currently hidden in a string of JSON in a projectprop, however there's nothing stopping you rolling your own. There's a sandbox if you want to take a look at the config format or have a play around.
       http://funcgen.vectorbase.org/PopulationBETA/project/sandbox/?id=141
You can also see the same data under alternative visualisations by swapping 'type=geoplot' for type=dot, etc.

Finally, we've also put together a couple of test sets to show how easily this can be used to combine data from different sources; grabbing JSON from 3rd parties concatenating with our data and pushing it through the same visualisations. E.g. our data + a plasmodium db (slow-ish, it's making multiple ajax calls to find geolocations):
       http://funcgen.vectorbase.org/PopulationBETA/xquery/plasmodb/

Obviously I'm hoping this will become more polished over the next few months, but I'm throwing it out there in case it is useful to anyone. And if anyone wants more information or, even better, wants to assist in the development let me know.

-s


--
Seth Redmond
 Scientific Programmer, VectorBase
 Kafatos / Christophides Groups
 Div. Cell and Molecular Biology
 Imperial College, London
[hidden email]
--



------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric
Ries, the creator of the Lean Startup Methodology on "Lean Startup
Secrets Revealed." This video shows you how to validate your ideas,
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Gmod-phendiver mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-phendiver


------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric
Ries, the creator of the Lean Startup Methodology on "Lean Startup
Secrets Revealed." This video shows you how to validate your ideas,
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Gmod-phendiver mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-phendiver
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Gmod-phendiver] phenovis

seth redmond
Well, the vis was less a continuation of what was produced at the hackathon than a total rewrite from scratch. I would at some point like to re-implement the bubble plots (which AFAIK was the only working protovis code I had from Nescent) but it's not happened yet. 

The middleware JSON isn't documented. It takes a fairly simple projects-contain-stocks-contain-experiments form, and any kind of JSON notation I saw was more complex than just looking at the ruby code; (it's all in the as_json methods and is clear enough).
If anyone's doing anything similar and wants to try and develop a one-size-fits-all json structure that would be cool, but for the moment the 3-way hashes we're making for most of our visualisations seems to work well enough for our purposes. 


-- 
Seth Redmond
  Scientific Programmer, VectorBase
  Kafatos / Christophides Groups
  Div. Cell and Molecular Biology
  Imperial College, London
[hidden email]
--

On 14 Jul 2011, at 22:08, Yuri Bendana wrote:

Nice work Seth!  You've made great progress on the visualizations since the hackathon.  I remember we discussed having a wiki page for describing the phenovis JSON structure, is there one? I still think it's a good idea since it'll be clearer for someone new than looking at the Rails code or raw JSON.  It'll also be a good place to discuss any extensions to the structure needed to support other types of visualizations.

yuri

On Thu, Jul 14, 2011 at 9:59 AM, seth redmond <[hidden email]> wrote:
It's been a while since the Hackathon and we've been doing some development at Vectorbase - coming up with a RESTful middleware and visualisation system for the Natural Diversity module.
       The usual caveats apply; this is still very much Under Development but I think it's solid enough to give you an idea of what I've been doing. This is the system we are going to use for Vectorbase, but it's general enough to be used for anything that uses the natural diversity module and could easily be extended to the rest of GMOD.

First off I've a JSON server retrieving data directly from the DB via ActiveRecord - a direct extension of the hackathon work. The JSON structure is still open to modification, but I've tried to keep it as clean as possible:
       http://funcgen.vectorbase.org/PopulationBETA/REST/project/141.json
       http://funcgen.vectorbase.org/PopulationBETA/REST/project/43/stocks.json
Code is in a branch on my github:
       https://github.com/sethnr/natdiv-rest

Then the real work: I'm calling it 'Phenovis' (because G-vis was already taken by google)
       https://github.com/sethnr/phenovis
I've started writing a javascript library to provide standard-ish visualisations based on the above JSON. The script uses jsonPATH (http://goessner.net/articles/JsonPath/) to find objects to iterate over, and variables of those objects, then it plots it via a few protovis classes. For examples of the map visualisations (gmap with a protovis overlay - the pie charts will dynamically re-generate depending on your zoom level) see:
       http://funcgen.vectorbase.org/PopulationBETA/Project/141
Grouped bar chart and dotplots are also available, and can be customised with, eg, linear or categorical axes, etc:
       http://funcgen.vectorbase.org/PopulationBETA/project/?id=43

It's worth pointing out that these are actually running off a completely separate instance, ajax calls are made to retrieve the data from the JSON server.
Config is currently hidden in a string of JSON in a projectprop, however there's nothing stopping you rolling your own. There's a sandbox if you want to take a look at the config format or have a play around.
       http://funcgen.vectorbase.org/PopulationBETA/project/sandbox/?id=141
You can also see the same data under alternative visualisations by swapping 'type=geoplot' for type=dot, etc.

Finally, we've also put together a couple of test sets to show how easily this can be used to combine data from different sources; grabbing JSON from 3rd parties concatenating with our data and pushing it through the same visualisations. E.g. our data + a plasmodium db (slow-ish, it's making multiple ajax calls to find geolocations):
       http://funcgen.vectorbase.org/PopulationBETA/xquery/plasmodb/

Obviously I'm hoping this will become more polished over the next few months, but I'm throwing it out there in case it is useful to anyone. And if anyone wants more information or, even better, wants to assist in the development let me know.

-s


--
Seth Redmond
 Scientific Programmer, VectorBase
 Kafatos / Christophides Groups
 Div. Cell and Molecular Biology
 Imperial College, London
[hidden email]
--



------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric
Ries, the creator of the Lean Startup Methodology on "Lean Startup
Secrets Revealed." This video shows you how to validate your ideas,
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Gmod-phendiver mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-phendiver



------------------------------------------------------------------------------
AppSumo Presents a FREE Video for the SourceForge Community by Eric
Ries, the creator of the Lean Startup Methodology on "Lean Startup
Secrets Revealed." This video shows you how to validate your ideas,
optimize your ideas and identify your business strategy.
http://p.sf.net/sfu/appsumosfdev2dev
_______________________________________________
Gmod-phendiver mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-phendiver
Loading...