RDF support for the API

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

RDF support for the API

Maxime Déraspe
Hi intermine's dev,

I am just starting familiarizing myself with the codebase of Intermine.

I would like to add some functionalities to the API so that it could
return RDF format as output.
A first step would be to convert on the fly the tabular results into
triples, but it could eventually become more sophisticated than that.
I would like to get the opinion of someone who already made some
development in the API.

I believe later on we would have to add some kind of parameter in the
API clients but for now I'm mostly interested on the server side
transformation.

Since I'd want to respect the code standard from the ground up is there
some documentation for code contribution ?

Thank you very much for your help,

Maxime


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: RDF support for the API

Justin Clark-Casey
Hi Maxime.  I haven't yet done any work on the webservices (I presume
this is what you mean by the API?) but I am a developer in the InterMine
group and am becoming familiar with the code in this area (but if
anybody is more familiar, please do feel free to chime in).

I can say that InterMine has received requests for RDF services so it is
on the radar.  If you are willing to spend time in this area then that
will be great - I am also interested and available (at varying degrees)
to help on various questions and quite possibly with bits of code.  Then
if things go well we can look to integrate changes with InterMine.  
There are also other groups who have either already started this road or
are thinking about it if they want to chime in.

I'm also a beginner in RDF/SPARQL though rapidly learning more.  What
kind of triple transformation are you thinking about - presumably
something like (<entity-id>, <column-name>, <column-value>)?  Are you
think about adapting the existing path query endpoints with an RDF
request parameter or a SPARQL endpoint?  The latter strikes me as
possibly a better long term path but maybe more challenging.  How about
the question of URI entities in RDF resolving to valid entities on the
mine?  Have you given any thought to performance impact?  I'm wondering
if generating large numbers of triples on the fly may be costly,
particularly as they are not coming from some dedicated triplestore.

The general doc on InterMine contributions is at [1].  I'm not sure we
have a coding standard per se (can someone correct me?), the best bet
right now is probably to keep in the style of the existing code.  Also,
this list is a good place to keep discussing things.

[1] http://intermine.readthedocs.org/en/latest/about/get-involved/

Best Regards,

--
Justin Clark-Casey
http://twitter.com/justincc

On 2016-01-06 21:46, Maxime Déraspe wrote:

> Hi intermine's dev,
>
> I am just starting familiarizing myself with the codebase of Intermine.
>
> I would like to add some functionalities to the API so that it could
> return RDF format as output.
> A first step would be to convert on the fly the tabular results into
> triples, but it could eventually become more sophisticated than that.
> I would like to get the opinion of someone who already made some
> development in the API.
>
> I believe later on we would have to add some kind of parameter in the
> API clients but for now I'm mostly interested on the server side
> transformation.
>
> Since I'd want to respect the code standard from the ground up is there
> some documentation for code contribution ?
>
> Thank you very much for your help,
>
> Maxime
>
>
> _______________________________________________
> dev mailing list
> [hidden email]
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev

_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: RDF support for the API

Maxime Déraspe
Hi Justin,


On 01/07/2016 02:04 PM, J. Clark-Casey wrote:
> Hi Maxime.  I haven't yet done any work on the webservices (I presume
> this is what you mean by the API?) but I am a developer in the
> InterMine group and am becoming familiar with the code in this area
> (but if anybody is more familiar, please do feel free to chime in).
>

Indeed I am talking about the webservices - that is a RESTful API with
some RPC calls as I have seen in the doc.


> I can say that InterMine has received requests for RDF services so it
> is on the radar.  If you are willing to spend time in this area then
> that will be great - I am also interested and available (at varying
> degrees) to help on various questions and quite possibly with bits of
> code.  Then if things go well we can look to integrate changes with
> InterMine.  There are also other groups who have either already
> started this road or are thinking about it if they want to chime in.

I am currently working in the group of Michel Dumontier at Stanford via
a supplement to SGD (Mike Cherry) to bring Linked Data to Intermine
deployments.


>
> I'm also a beginner in RDF/SPARQL though rapidly learning more.  What
> kind of triple transformation are you thinking about - presumably
> something like (<entity-id>, <column-name>, <column-value>)?  

Yes that could work and would be a good proof of concept. The think to
keep in mind is to correctly resolve the entities that come from a
second table via a join clauses (in SQL) so they can refer to the right
resource in the other table. We might want to define a certain schema in
the first place to correctly map the RDF triples from the SQL database.


> Are you think about adapting the existing path query endpoints with an
> RDF request parameter or a SPARQL endpoint?  The latter strikes me as
> possibly a better long term path but maybe more challenging.  

I am currently working that way with external tools but I think it is
more challenging and more work to incorporate everything in the
intermine webservices and that would be better in the long term for
maintenance.


> How about the question of URI entities in RDF resolving to valid
> entities on the mine?  

Right the URI should be built in a way that they can easily resolve the
resource in the mine (I mean redirection in the front-end to browse the
actual resource) but there might be some work to be done in other
components of intermine (probably in the webapp to resolve the URL path)
to make this work.


> Have you given any thought to performance impact?  I'm wondering if
> generating large numbers of triples on the fly may be costly,
> particularly as they are not coming from some dedicated triplestore.

I think that this would only be a matter of transformation on the server
side before sending the answer back to the client.

I am not yet familiar with everything that goes on in the back-end to
process a query, for the webservices I assume that the client API makes
a query that is received by the object model of intermine which make
some calls to the SQL database, then it retrieves the answer and format
it in a tabular format to send it back to the client.. so it might be
negligible to transform into RDF, but we'll have to run some test.
Having a prior configuration so that the service automatically knows how
to transform the data into RDF - like by knowing the links between the
SQL tables (primary and foreign keys), etc. it shouldn't be that hard to
maintained and make the conversion on the fly IMO but I need to learn
more how the webservices work and how it communicates with API client.

That's true that a triplestore would probably be more efficient for
complex sparql queries, but the computer resources and the configuration
needed for deployment would almost doubled and it might be in the long
run painful to maintained.


>
> The general doc on InterMine contributions is at [1].  I'm not sure we
> have a coding standard per se (can someone correct me?), the best bet
> right now is probably to keep in the style of the existing code.
> Also, this list is a good place to keep discussing things.
>
> [1] http://intermine.readthedocs.org/en/latest/about/get-involved/
>

OK thank you I'll have a look into it.

I'll be in touch for further question once I try more technical stuff in the code.


> Best Regards,
>
> --
> Justin Clark-Casey
> http://twitter.com/justincc
>
> On 2016-01-06 21:46, Maxime Déraspe wrote:
>> Hi intermine's dev,
>>
>> I am just starting familiarizing myself with the codebase of Intermine.
>>
>> I would like to add some functionalities to the API so that it could
>> return RDF format as output.
>> A first step would be to convert on the fly the tabular results into
>> triples, but it could eventually become more sophisticated than that.
>> I would like to get the opinion of someone who already made some
>> development in the API.
>>
>> I believe later on we would have to add some kind of parameter in the
>> API clients but for now I'm mostly interested on the server side
>> transformation.
>>
>> Since I'd want to respect the code standard from the ground up is there
>> some documentation for code contribution ?
>>
>> Thank you very much for your help,
>>
>> Maxime
>>
>>
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>
> _______________________________________________
> dev mailing list
> [hidden email]
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev