Importing txt files

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Importing txt files

siwelb

Is it possible to upload data to a mine in txt format or do these files first need to be converted to an XML format?

 

Thanks,

Brian Lewis

 


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Importing txt files

Julie Sullivan
Hi Brian

You can do either.  You can write a Java data loader which will parse your
text file and create and store objects directly to the database.  Or you
can write a Perl script to parse your data file and write an XML file,
which can then be loaded into the mine.

This doc might help:

http://intermine.org/wiki/SourceHowto

I also would recommend going through our tutorial, it might make the data
integration system a little more clear:

http://intermine.org/wiki/GettingStarted

If you let me know which one you prefer, I can point you to some more
examples.  I think most people choose whichever language they like best.

Cheers,
Julie

> Is it possible to upload data to a mine in txt format or do these files
> first need to be converted to an XML format?
>
> Thanks,
> Brian Lewis
>
> _______________________________________________
> dev mailing list
> [hidden email]
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Importing txt files

siwelb
Julie -

Thanks for the reply and the links.  I'm quite a bit more familiar with Java so I think I'd rather go with the Java data loader approach, especially since I've done this with other applications before.  I would definitely be interested in seeing some examples of this approach.

Thanks,
~ Brian

-----Original Message-----
From: www-data [mailto:[hidden email]] On Behalf Of [hidden email]
Sent: Thursday, August 04, 2011 3:55 PM
To: Lewis, Brian Andrew
Cc: [hidden email]
Subject: Re: [InterMine Dev] Importing txt files

Hi Brian

You can do either.  You can write a Java data loader which will parse your text file and create and store objects directly to the database.  Or you can write a Perl script to parse your data file and write an XML file, which can then be loaded into the mine.

This doc might help:

http://intermine.org/wiki/SourceHowto

I also would recommend going through our tutorial, it might make the data integration system a little more clear:

http://intermine.org/wiki/GettingStarted

If you let me know which one you prefer, I can point you to some more examples.  I think most people choose whichever language they like best.

Cheers,
Julie

> Is it possible to upload data to a mine in txt format or do these
> files first need to be converted to an XML format?
>
> Thanks,
> Brian Lewis
>
> _______________________________________________
> dev mailing list
> [hidden email]
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Importing txt files

Julie Sullivan
Hi Brian

I think our KEGG loaders are fairly basic, especially kegg-orthologues:
       
http://intrac.flymine.org/browser/trunk/bio/sources/kegg-orthologues/main/src/org/intermine/bio/dataconversion/KeggOrthologuesConverter.java

The code loops through the text file and creates gene, homologue and evidence
objects.

It only creates genes for organisms set in the project XML file; this is
important because the KEGG file has all organisms.  (You can disregard the
config and resolver sections, you won't need those.)

On the wiki page I linked to it explains how to set the keys and update the data
model for your new source.

Let me know if you need more help!

Cheers,
Julie

On 05/08/11 12:36, Lewis, Brian Andrew wrote:

> Julie -
>
> Thanks for the reply and the links.  I'm quite a bit more familiar with Java so I think I'd rather go with the Java data loader approach, especially since I've done this with other applications before.  I would definitely be interested in seeing some examples of this approach.
>
> Thanks,
> ~ Brian
>
> -----Original Message-----
> From: www-data [mailto:[hidden email]] On Behalf Of [hidden email]
> Sent: Thursday, August 04, 2011 3:55 PM
> To: Lewis, Brian Andrew
> Cc: [hidden email]
> Subject: Re: [InterMine Dev] Importing txt files
>
> Hi Brian
>
> You can do either.  You can write a Java data loader which will parse your text file and create and store objects directly to the database.  Or you can write a Perl script to parse your data file and write an XML file, which can then be loaded into the mine.
>
> This doc might help:
>
> http://intermine.org/wiki/SourceHowto
>
> I also would recommend going through our tutorial, it might make the data integration system a little more clear:
>
> http://intermine.org/wiki/GettingStarted
>
> If you let me know which one you prefer, I can point you to some more examples.  I think most people choose whichever language they like best.
>
> Cheers,
> Julie
>
>> Is it possible to upload data to a mine in txt format or do these
>> files first need to be converted to an XML format?
>>
>> Thanks,
>> Brian Lewis
>>
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>>
>
>

_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev