Tripal/Chado Web Services Update
Locked
Tripal/Chado Web Services Update
 
|
Dear Tripal/Chado Community,
Some progress has been made on the "low-level" Chado API that will provide access to data in a Chado-centric manner, but does not include a design for specific use cases. The current state of this web service API can be found on the Tripal 2.x demo site: http://demo.tripal.info/2.x/web_services Currently, the web services are very simple. As progress is made we will update the web services on that page. There is still much work to do, so what you see is not finalized but is made available in an effort to solicit feedback for suggested changes. Please let us know what you do or don't like so far. Also, on Saturday, Jan 10th at the Plant and Animal Genome Conference (PAG) in San Deigo, several folks interested in the development of web services for Tripal/Chado met for our second meeting to discuss design of the web services as funded by the NSF DIBBs grant. The goal of these meetings is to solicit input in the design of web services such that they can support the needs of a broad set of Tripal sites. Below is a Google doc containing a summary of that meeting. The section containing attendee comments was drawn from memory as we failed to designate an official note taker. https://docs.google.com/document/d/1gdL7Xo-Xoq2ztBVY0FVAHWPv5kNz_ajyNtAlYjCzC4U/edit For anyone interested in web services for Tripal/Chado, please contact me directly if you have comments as the way we have approached soliciting input during the design phase (positive or negative-- I'm happy to hear all) as we want to ensure stake-holders feel heard and the design fits the broadest needs. Thanks much to all who are participating in this! Stephen ------------------------------------------------------------------------------ Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Gmod-schema mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gmod-schema |
Re: Tripal/Chado Web Services Update
|
I haven't been aware of what - if anything - has been happening on the chado API scene. (the web_services link in this mail no longer works. I take this as not a good sign.) But I wanted to pass along to the group some info of a project that I've just learned about in case it is useful. postgrest (www.postgrest.com and https://github.com/begriffs/postgrest) is a project that provides an one-stop REST service for a postgres db. The project is at an early stage and it looks that there much to be done. (I don't know if I trust the authentication just yet.) And the querying seems to be limited to single tables (or views) and stored procedures. I realize there are other similar projects out there. This is just one that I've tried out. (No other involvement in the project, though.) launching is easy: ./postgrest postgres://<database role>:<password>@<dbserver>/<dbname> -a <database role for anonymous connections> A couple examples. A query for a single table (or view): curl 'localhost:3000/organism?species=eq.sapiens' [{"organism_id":1,"abbreviation":"H.sapiens","genus":"Homo","species":"sapiens","common_name":"human","comment":null}] A query on a table uses the endpoint /<table name>. Column constraints on the query are given as "<column>=<operator>.<value>"; in this case species='sapiens' And a stored procedure. genomic_slice is one that I have for returning a segment of a chromosome sequence. ('proteome' is a dbxref... don't worry about that) curl -d '{"proteome":"210","chromosome":"Chr01","fmin":"1000000","len":"10"}' -X POST -H "Content-Type: application/json" 'http://localhost:3000/rpc/genomic_slice' [{"genomic_slice":"TACCAAGGTA"}] (specifying the application/json in the POST headers is important!) I wanted to try the queries on the public flybase db. But that server is running postgres 8.4 and this requires at least 9.0.3 Part of the reason for sending the email is that there will be a meetup with the developer in San Fran next week for those that are local. http://www.meetup.com/postgresql-1/events/229237351 The talk will be video taped and available later, I've heard. Joe On 01/29/2015 08:57 AM, Stephen Ficklin wrote: > Dear Tripal/Chado Community, > > Some progress has been made on the "low-level" Chado API that will > provide access to data in a Chado-centric manner, but does not include a > design for specific use cases. The current state of this web service > API can be found on the Tripal 2.x demo site: > > http://demo.tripal.info/2.x/web_services > > Currently, the web services are very simple. As progress is made we > will update the web services on that page. There is still much work to > do, so what you see is not finalized but is made available in an effort > to solicit feedback for suggested changes. Please let us know what you > do or don't like so far. > > Also, on Saturday, Jan 10th at the Plant and Animal Genome Conference > (PAG) in San Deigo, several folks interested in the development of web > services for Tripal/Chado met for our second meeting to discuss design > of the web services as funded by the NSF DIBBs grant. The goal of > these meetings is to solicit input in the design of web services such > that they can support the needs of a broad set of Tripal sites. Below > is a Google doc containing a summary of that meeting. The section > containing attendee comments was drawn from memory as we failed to > designate an official note taker. > > https://docs.google.com/document/d/1gdL7Xo-Xoq2ztBVY0FVAHWPv5kNz_ajyNtAlYjCzC4U/edit > > For anyone interested in web services for Tripal/Chado, please contact > me directly if you have comments as the way we have approached > soliciting input during the design phase (positive or negative-- I'm > happy to hear all) as we want to ensure stake-holders feel heard and the > design fits the broadest needs. > > Thanks much to all who are participating in this! > Stephen > > > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming. The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > Gmod-schema mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/gmod-schema ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140 _______________________________________________ Gmod-schema mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gmod-schema |
Re: Tripal/Chado Web Services Update
|
Hiya Joe,
Now that you've mentioned this I suppose it's time to share what I've been working on. I was going to keep it under wraps until I was further along but ... oh well ;) On 03/10/2016 05:33 PM, Joe Carlson
wrote:
I'm actually using this since there was going to be a month lead on the "official" one. I was a bit excited to get started.I haven't been aware of what - if anything - has been happening on the chado API scene. (the web_services link in this mail no longer works. I take this as not a good sign.) I haven't figured out the auth yet to be honest. I'm fond of JWT, it's easy to use, etc. Just haven't thrown time at it yet.But I wanted to pass along to the group some info of a project that I've just learned about in case it is useful. postgrest (www.postgrest.com and https://github.com/begriffs/postgrest) is a project that provides an one-stop REST service for a postgres db. The project is at an early stage and it looks that there much to be done. (I don't know if I trust the authentication just yet.) And the querying seems to be limited to single tables (or views) and stored procedures. I realize there are other similar projects out there. This is just one that I've tried out. (No other involvement in the project, though.) That said, between:
It's insanely easy to throw together a responsive/fast Chado
admin interface. Just hit login :) I wrote a small API server to expose a chado database as a
JBrowse REST compatible API. The postgrest server is available here with the yeast data from
one tutorial on the GMOD wiki. I'm actually using a couple of docker images to do this in case anyone else wants to play with it locally
I tried the same, would be great if there were other publicly
exposed chado databases. (I'd love to know how other people store their data. How do people reference intermediate samples that they track? Sequencing files on disk? Etc?) Part of the reason for sending the email is that there will be a meetup with the developer in San Fran next week for those that are local. http://www.meetup.com/postgresql-1/events/229237351 The talk will be video taped and available later, I've heard. Joe On 01/29/2015 08:57 AM, Stephen Ficklin wrote:Dear Tripal/Chado Community, Some progress has been made on the "low-level" Chado API that will provide access to data in a Chado-centric manner, but does not include a design for specific use cases. The current state of this web service API can be found on the Tripal 2.x demo site: http://demo.tripal.info/2.x/web_services Currently, the web services are very simple. As progress is made we will update the web services on that page. There is still much work to do, so what you see is not finalized but is made available in an effort to solicit feedback for suggested changes. Please let us know what you do or don't like so far. Also, on Saturday, Jan 10th at the Plant and Animal Genome Conference (PAG) in San Deigo, several folks interested in the development of web services for Tripal/Chado met for our second meeting to discuss design of the web services as funded by the NSF DIBBs grant. The goal of these meetings is to solicit input in the design of web services such that they can support the needs of a broad set of Tripal sites. Below is a Google doc containing a summary of that meeting. The section containing attendee comments was drawn from memory as we failed to designate an official note taker. https://docs.google.com/document/d/1gdL7Xo-Xoq2ztBVY0FVAHWPv5kNz_ajyNtAlYjCzC4U/edit For anyone interested in web services for Tripal/Chado, please contact me directly if you have comments as the way we have approached soliciting input during the design phase (positive or negative-- I'm happy to hear all) as we want to ensure stake-holders feel heard and the design fits the broadest needs. Thanks much to all who are participating in this! Stephen ------------------------------------------------------------------------------ Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Gmod-schema mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gmod-schema------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140 _______________________________________________ Gmod-schema mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gmod-schema --
Eric Rasche Programmer II Center for Phage Technology Rm 312A, BioBio Texas A&M University College Station, TX 77843 <a href="tel:404-692-2048">404-692-2048 [hidden email] ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140 _______________________________________________ Gmod-schema mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gmod-schema |
Re: Tripal/Chado Web Services Update
|
|
In reply to this post by joe carlson
Hi Joe,
To provide an update as to where we are with web services and Chado... Yes, the link in the earlier email below has been removed. When we first started on the project we had intended to create RESTful web services with direct access to Chado. The link that was in that original email from early last year was a demo of that work. Essentially, you could find genomic features by using the 'feature' resource, and the 'stock' resource for germplasm, etc. However, in meetings that we had with the community of Chado users it was clear that there was a preference for a web services that were not Chado-centric. Folks weren't interested in requiring their users to understand Chado table structure in order to query for information. Some tables, such as the stock table, can be very confusing to users as it's not always clear what's stored in that table. Additionally, some of the sites that use Tripal and Chado have a set of authentication and access rules that would be difficult to enforce via Chado-centric web services. For example, some data is private and only available to specific users, and Chado itself has no account of that access information. Tripal / Drupal does, so we wanted tighter integration with web services and Tripal/Drupal authentication support. Third, it is increasingly common for some data to not be housed in Chado (i.e. BAM alignments, all info in a VCF file, images, etc.), and we wanted those non-Chado (linked to Chado data) to also be available via web services. This ensures we have consistency between the data that is shown on a page (i.e. gene page) and the same data that is available via web services for the same content. Our goal is to provide web services for data in Chado, but we also have the goal of providing web services for the "community database" as well, so we have these additional constraints that need to be built into the web services, and hence they necessitated a change from our original plans. There was some interest in also having Chado-centric web services, but we just didn't have the time to more fully develop both more robust web services and the Chado-centric version at the same time. To meet the requirements described above we redesigned how we are going to offer data in Chado via web services for Tripal v3 (coming soon). It will offer data in a more intuitive manner (e.g. via data types: genes, germplasm, genetic_marker, etc.), and ensures privacy of data. It does support Chado but incorporates non-Chado data as well. So, long story.... we took down the link to the original Chado-based web services we were working on, and unfortunately that added confusion because we failed to replace it with additional information. Our work is progressing and we are close to offering a meeting to provide a demo of our current implementation for web services and to solicit more feedback on our web services design. That should occur in April, and I'll send out a Doodle poll by email soon to schedule a time for interested groups. Stephen On 3/10/2016 9:33 AM, Joe Carlson wrote: > > I haven't been aware of what - if anything - has been happening on the > chado API scene. (the web_services link in this mail no longer works. > I take this as not a good sign.) > > But I wanted to pass along to the group some info of a project that > I've just learned about in case it is useful. postgrest > (www.postgrest.com and https://github.com/begriffs/postgrest) is a > project that provides an one-stop REST service for a postgres db. > > The project is at an early stage and it looks that there much to be > done. (I don't know if I trust the authentication just yet.) And the > querying seems to be limited to single tables (or views) and stored > procedures. I realize there are other similar projects out there. This > is just one that I've tried out. (No other involvement in the project, > though.) > > launching is easy: > > ./postgrest postgres://<database role>:<password>@<dbserver>/<dbname> > -a <database role for anonymous connections> > > A couple examples. A query for a single table (or view): > > curl 'localhost:3000/organism?species=eq.sapiens' > [{"organism_id":1,"abbreviation":"H.sapiens","genus":"Homo","species":"sapiens","common_name":"human","comment":null}] > > > A query on a table uses the endpoint /<table name>. Column constraints > on the query are given as "<column>=<operator>.<value>"; in this case > species='sapiens' > > And a stored procedure. genomic_slice is one that I have for returning > a segment of a chromosome sequence. ('proteome' is a dbxref... don't > worry about that) > > curl -d > '{"proteome":"210","chromosome":"Chr01","fmin":"1000000","len":"10"}' > -X POST -H "Content-Type: application/json" > 'http://localhost:3000/rpc/genomic_slice' > [{"genomic_slice":"TACCAAGGTA"}] > > (specifying the application/json in the POST headers is important!) > > I wanted to try the queries on the public flybase db. But that server > is running postgres 8.4 and this requires at least 9.0.3 > > Part of the reason for sending the email is that there will be a > meetup with the developer in San Fran next week for those that are local. > > http://www.meetup.com/postgresql-1/events/229237351 > > The talk will be video taped and available later, I've heard. > > Joe > > On 01/29/2015 08:57 AM, Stephen Ficklin wrote: >> Dear Tripal/Chado Community, >> >> Some progress has been made on the "low-level" Chado API that will >> provide access to data in a Chado-centric manner, but does not include a >> design for specific use cases. The current state of this web service >> API can be found on the Tripal 2.x demo site: >> >> http://demo.tripal.info/2.x/web_services >> >> Currently, the web services are very simple. As progress is made we >> will update the web services on that page. There is still much work to >> do, so what you see is not finalized but is made available in an effort >> to solicit feedback for suggested changes. Please let us know what you >> do or don't like so far. >> >> Also, on Saturday, Jan 10th at the Plant and Animal Genome Conference >> (PAG) in San Deigo, several folks interested in the development of web >> services for Tripal/Chado met for our second meeting to discuss design >> of the web services as funded by the NSF DIBBs grant. The goal of >> these meetings is to solicit input in the design of web services such >> that they can support the needs of a broad set of Tripal sites. Below >> is a Google doc containing a summary of that meeting. The section >> containing attendee comments was drawn from memory as we failed to >> designate an official note taker. >> >> https://docs.google.com/document/d/1gdL7Xo-Xoq2ztBVY0FVAHWPv5kNz_ajyNtAlYjCzC4U/edit >> >> >> For anyone interested in web services for Tripal/Chado, please contact >> me directly if you have comments as the way we have approached >> soliciting input during the design phase (positive or negative-- I'm >> happy to hear all) as we want to ensure stake-holders feel heard and the >> design fits the broadest needs. >> >> Thanks much to all who are participating in this! >> Stephen >> >> >> >> ------------------------------------------------------------------------------ >> >> Dive into the World of Parallel Programming. The Go Parallel Website, >> sponsored by Intel and developed in partnership with Slashdot Media, >> is your >> hub for all things parallel software development, from weekly thought >> leadership blogs to news, videos, case studies, tutorials and more. >> Take a >> look and join the conversation now. http://goparallel.sourceforge.net/ >> _______________________________________________ >> Gmod-schema mailing list >> [hidden email] >> https://lists.sourceforge.net/lists/listinfo/gmod-schema > ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140 _______________________________________________ Gmod-schema mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gmod-schema |
Re: Tripal/Chado Web Services Update
|
In reply to this post by Eric Rasche-3
Hi Joe,
Great for chipping in about postgrest, really neat piece of software to httpify postgresql access. Will be great to tame chado access with this tool and have a great developer experience. Love those curl snippets! Hi Eric, Well done with Jbrowse REST interface, the enjoyable part is that it's written in golang which is my preferred choice for server side. I think developing web services(REST/unREST anything..) in golang is really straight forward and combining it with postgrest just put you in a no excuse zone. Once again thanks for sharing the code, have quite a bit to learn and code for myself. thanks, -siddhartha On Thu, 10 Mar 2016, Eric Rasche wrote: > Hiya Joe, > > Now that you've mentioned this I suppose it's time to share what I've been > working on. > I was going to keep it under wraps until I was further along but ... oh > well ;) > > On 03/10/2016 05:33 PM, Joe Carlson wrote: > > I haven't been aware of what - if anything - has been happening on the > chado API scene. (the web_services link in this mail no longer works. I > take this as not a good sign.) > > I'm actually using this since there was going to be a month lead on the > "official" one. I was a bit excited to get started. > > But I wanted to pass along to the group some info of a project that I've > just learned about in case it is useful. postgrest (www.postgrest.com > and https://github.com/begriffs/postgrest) is a project that provides an > one-stop REST service for a postgres db. > > The project is at an early stage and it looks that there much to be > done. (I don't know if I trust the authentication just yet.) And the > querying seems to be limited to single tables (or views) and stored > procedures. I realize there are other similar projects out there. This > is just one that I've tried out. (No other involvement in the project, > though.) > > I haven't figured out the auth yet to be honest. I'm fond of JWT, it's > easy to use, etc. Just haven't thrown time at it yet. > > That said, between: > * postgrest > * ng-admin's example postgrest project > > It's insanely easy to throw together a responsive/fast Chado admin > interface. > If y'all will be kind to my servers (writing back doesn't work due to > CORS, otherwise "save" would work), I have a demo here: > Demo: https://erasche.github.io/chado-angular-admin > Source: https://github.com/erasche/chado-angular-admin > > Just hit login :) > > I wrote a small API server to expose a chado database as a JBrowse REST > compatible API. > Demo: http://erasche.github.io/chado-angular-admin/browser.html > Source: https://github.com/erasche/chado-jbrowse-connector > > The postgrest server is available here with the yeast data from one > tutorial on the GMOD wiki. > http://shed.hx42.org:8300/ > > I'm actually using a couple of docker images to do this in case anyone > else wants to play with it locally > > launching is easy: > > ./postgrest postgres://<database role>:<password>@<dbserver>/<dbname> -a > <database role for anonymous connections> > > A couple examples. A query for a single table (or view): > > curl 'localhost:3000/organism?species=eq.sapiens' > [{"organism_id":1,"abbreviation":"H.sapiens","genus":"Homo","species":"sapiens","common_name":"human","comment":null}] > > A query on a table uses the endpoint /<table name>. Column constraints > on the query are given as "<column>=<operator>.<value>"; in this case > species='sapiens' > > And a stored procedure. genomic_slice is one that I have for returning a > segment of a chromosome sequence. ('proteome' is a dbxref... don't > worry about that) > > curl -d > '{"proteome":"210","chromosome":"Chr01","fmin":"1000000","len":"10"}' -X > POST -H "Content-Type: application/json" > 'http://localhost:3000/rpc/genomic_slice' > [{"genomic_slice":"TACCAAGGTA"}] > > (specifying the application/json in the POST headers is important!) > > I wanted to try the queries on the public flybase db. But that server is > running postgres 8.4 and this requires at least 9.0.3 > > I tried the same, would be great if there were other publicly exposed > chado databases. > (I'd love to know how other people store their data. How do people > reference intermediate samples that they track? Sequencing files on disk? > Etc?) > > > Part of the reason for sending the email is that there will be a meetup > with the developer in San Fran next week for those that are local. > > http://www.meetup.com/postgresql-1/events/229237351 > > The talk will be video taped and available later, I've heard. > > Joe > > On 01/29/2015 08:57 AM, Stephen Ficklin wrote: > > Dear Tripal/Chado Community, > > Some progress has been made on the "low-level" Chado API that will > provide access to data in a Chado-centric manner, but does not include a > design for specific use cases. The current state of this web service > API can be found on the Tripal 2.x demo site: > > http://demo.tripal.info/2.x/web_services > > Currently, the web services are very simple. As progress is made we > will update the web services on that page. There is still much work to > do, so what you see is not finalized but is made available in an effort > to solicit feedback for suggested changes. Please let us know what you > do or don't like so far. > > Also, on Saturday, Jan 10th at the Plant and Animal Genome Conference > (PAG) in San Deigo, several folks interested in the development of web > services for Tripal/Chado met for our second meeting to discuss design > of the web services as funded by the NSF DIBBs grant. The goal of > these meetings is to solicit input in the design of web services such > that they can support the needs of a broad set of Tripal sites. Below > is a Google doc containing a summary of that meeting. The section > containing attendee comments was drawn from memory as we failed to > designate an official note taker. > > https://docs.google.com/document/d/1gdL7Xo-Xoq2ztBVY0FVAHWPv5kNz_ajyNtAlYjCzC4U/edit > > For anyone interested in web services for Tripal/Chado, please contact > me directly if you have comments as the way we have approached > soliciting input during the design phase (positive or negative-- I'm > happy to hear all) as we want to ensure stake-holders feel heard and the > design fits the broadest needs. > > Thanks much to all who are participating in this! > Stephen > > > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming. The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > Gmod-schema mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/gmod-schema > > > ------------------------------------------------------------------------------ > Transform Data into Opportunity. > Accelerate data analysis in your applications with > Intel Data Analytics Acceleration Library. > Click to learn more. > http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140 > _______________________________________________ > Gmod-schema mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/gmod-schema > > -- > Eric Rasche > Programmer II > > Center for Phage Technology > Rm 312A, BioBio > Texas A&M University > College Station, TX 77843 > 404-692-2048 > [hidden email] > ------------------------------------------------------------------------------ > Transform Data into Opportunity. > Accelerate data analysis in your applications with > Intel Data Analytics Acceleration Library. > Click to learn more. > http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140 > _______________________________________________ > Gmod-schema mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/gmod-schema ------------------------------------------------------------------------------ Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140 _______________________________________________ Gmod-schema mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gmod-schema |
«
Return to Chado
|
1 view|%1 views
Loading...
| Free forum by Nabble | Edit this page |