is anyone using the Perl items API?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

is anyone using the Perl items API?

Jayaraman, Pushkala

It looks like the Perl Ietms API doesn’t create the xml correctly.. I can never get any of my objects to integrate with the SOTerms when they are generated by the Perl API.. the JAVA api on the other hand helps integrate the objects to the soterms correctly..

 

Has anyone else come across this ?

 

If so.. do we need to continue supporting the Perl Items API if it doesn’t create the objects correctly and integrate them correctly..?

 

Thanks,

Pushkala Jayaraman

Programmer/Analyst - Rat Genome Database

Human and Molecular Genetics Center

Medical College of Wisconsin

414-955-2229

http://rgd.mcw.edu

 


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: is anyone using the Perl items API?

Julie Sullivan
Hi Pushkala

The Java converter does a lot of things for you - automatically creates SO
terms / data sets and sources, there are several convenience methods for
you etc etc.

See the Bio*Converter.java files, e.g.:
https://github.com/intermine/intermine/blob/dev/bio/core/main/src/org/intermine/bio/dataconversion/BioFileConverter.java

And of course if you use the Java converter, you'll use the Java API to
create InterMine objects. Even so, you can create InterMine Items XML in
any language you want. To help with this, we created a Perl Items API. (I
know we have some folks who have Python scripts and don't use any APIs at
all!)

However if you write a Perl script using the Perl API, you'll have to do
all the things the Java converter does manually.

You and I have talked about Andrew's script before, and I had thought that
it wasn't a merging issue. If you have two genes only one with a SO term
reference, they should merge correctly as empty references aren't
considered a valid value.

Is it possible for me to see your webapp? Are your keys set correctly for
all sources? Last time we talked, we tracked down which sources were
creating the bad genes by running this query:

select * from tracker where objectid = N;

1. Can you give an example of a "bad" gene (or pair of duplicate genes)
and why it's wrong?
2. Which sources created this gene?
3. Can you send me the keys for these sources - or even better can you
update the ratmine github repo?

That being said, I would recommend you update Andrew's script to create SO
terms as it's bad practice to depend on another source - there might not
be one! If I remember correctly you were going to run Andrew's unit test
and get that passing first.

You were also considering moving to the Java converter - or was that for
another source?

Sorry you're having so much trouble - we'll get this sorted out!

Cheers
Julie

> It looks like the Perl Ietms API doesn't create the xml correctly.. I can
> never get any of my objects to integrate with the SOTerms when they are
> generated by the Perl API.. the JAVA api on the other hand helps integrate
> the objects to the soterms correctly..
>
> Has anyone else come across this ?
>
> If so.. do we need to continue supporting the Perl Items API if it doesn't
> create the objects correctly and integrate them correctly..?
>
> Thanks,
> Pushkala Jayaraman
> Programmer/Analyst - Rat Genome Database
> Human and Molecular Genetics Center
> Medical College of Wisconsin
> 414-955-2229
> http://rgd.mcw.edu
>
> _______________________________________________
> dev mailing list
> [hidden email]
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>

_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: is anyone using the Perl items API?

Jayaraman, Pushkala
My response is inline..

-----Original Message-----
From: www-data [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Thursday, September 26, 2013 3:15 AM
To: Jayaraman, Pushkala
Cc: [hidden email]
Subject: Re: [InterMine Dev] is anyone using the Perl items API?

Hi Pushkala

The Java converter does a lot of things for you - automatically creates SO terms / data sets and sources, there are several convenience methods for you etc etc.

See the Bio*Converter.java files, e.g.:
https://github.com/intermine/intermine/blob/dev/bio/core/main/src/org/intermine/bio/dataconversion/BioFileConverter.java

And of course if you use the Java converter, you'll use the Java API to create InterMine objects. Even so, you can create InterMine Items XML in any language you want. To help with this, we created a Perl Items API. (I know we have some folks who have Python scripts and don't use any APIs at
all!)

However if you write a Perl script using the Perl API, you'll have to do all the things the Java converter does manually.

--which I do..


You and I have talked about Andrew's script before, and I had thought that it wasn't a merging issue. If you have two genes only one with a SO term reference, they should merge correctly as empty references aren't considered a valid value.

--yup and that happens with the genes.. you helped me figure out that my genes weren't integrating correctly and I corrected that.. since then my genes have been integrating correctly..
The issue was that the homology xml created via the Perl api and the genes xml created via the Perl API were both not creating the soterms.. the xml file had one id 0_4 always missing after ids 0_1 0_2 and 0_3 being used for the organisms.. 0_4 should be used to define the soterm gene object.. but it wasn't happening and all the other records in the xml file were referring to the id 0_4 which didn't exist..  but when I switched to the Java Gff3 Converter to create my genes they then ended up merging the data correctly because my java converter was now creating soterms correctly and the sequenceontologytermid in the genes table now no longer contained "null" elements.. it all referenced back to the id of the gene object in the soterms table..

That's how I know the integrating is happening correctly..

Now in case of QTLs it's the same, I have the perl API that creates the QTLs xml.. which does not have an entry of the soterm object QTL but then every single qtl record references the soterm object and that causes improper integration of the soterms.. so the sequenceontologytermid in the qtls table is also null..
The gff3 file for qtls (loaded by the Java Gff3 converter ) then integrated with the qtls and added the seuenceontologytermids.

If that was the case then we should fix the Perl API to correctly create the soterm..

I could paste my perl code that shows that there is a line in the perl code (that creates the xml file) that I am manually calling the function to create the "gene" object or the QTL object.. but it just doesn't create that in the xml file..
:(

Is it possible for me to see your webapp? Are your keys set correctly for all sources? Last time we talked, we tracked down which sources were creating the bad genes by running this query:

select * from tracker where objectid = N;

1. Can you give an example of a "bad" gene (or pair of duplicate genes) and why it's wrong?
2. Which sources created this gene?
3. Can you send me the keys for these sources - or even better can you update the ratmine github repo?

-- yes that really helped me understand how to debug.. Thanks for that Julie! :) that's how I came about figuring this out .


That being said, I would recommend you update Andrew's script to create SO terms as it's bad practice to depend on another source - there might not be one! If I remember correctly you were going to run Andrew's unit test and get that passing first.

-- the trick is to get the perl code to create that tag in the xml file.. ill send you my code.. youll see what im talking about..

You were also considering moving to the Java converter - or was that for another source?

-- yup already did.. that show I got the gemes to integrate correctly.. the qtls are causing me as much worry now..

Sorry you're having so much trouble - we'll get this sorted out!

Lemme know when yore available on Campfire.. (BTW it kicked me out again! :(  could I please get an invite back to the tech support room?)

Cheers
Julie

> It looks like the Perl Ietms API doesn't create the xml correctly.. I
> can never get any of my objects to integrate with the SOTerms when
> they are generated by the Perl API.. the JAVA api on the other hand
> helps integrate the objects to the soterms correctly..
>
> Has anyone else come across this ?
>
> If so.. do we need to continue supporting the Perl Items API if it
> doesn't create the objects correctly and integrate them correctly..?
>
> Thanks,
> Pushkala Jayaraman
> Programmer/Analyst - Rat Genome Database Human and Molecular Genetics
> Center Medical College of Wisconsin
> 414-955-2229
> http://rgd.mcw.edu
>
> _______________________________________________
> dev mailing list
> [hidden email]
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>

_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev