Intermine build errors

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Intermine build errors

Paulo Nuin
Hi everyone

This is my first message to the list. I am currently working on deploying a build of Intermine on AWS and I am running into some problems on the build process. I will add a couple of errors below and some background on the process to solve them and what work and what didn’t. Any help is appreciated.


The first blocking error was

starting command: ant -v -Dsource=celegans-gff3-gene


BUILD FAILED
/mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
/mnt/data/intermine/imbuild/source.xml:256: java.io.FileNotFoundException: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-gff3/mapping/id_mapping.tab (No such file or directory)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:146)


as no mapping file was available in the required directory. I tried searching the repository online, googling it and wasn’t able to find this file. Todd Harris, whom I am helping setting this up, found mapping files in a section of the source code, but we are not sure if these files are created or not during the process. At that point I was transferring all files from most of the release 246 of worm base, but with the mapping file used, only C elegans was able to be used in the build. The error we were getting with all sequences was

BUILD FAILED
/mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
/mnt/data/intermine/imbuild/source.xml:256: java.lang.RuntimeException: IOException while getting next GFF record
        at org.intermine.bio.io.gff3.GFF3Parser$1.next(GFF3Parser.java:78)


with the workaround being just transferring C elegans. The build went on a couple more steps and failed at

/mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
/mnt/data/intermine/imbuild/source.xml:145: failed to set value for mappingFile to /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@41e0c4c4

Caused by: java.io.FileNotFoundException: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties (No such file or directory)

The only similar file I found that was able to make the process continue was

 /mnt/data/website-intermine-dev/acedb-dev/intermine/testlab/perl/OBSOLETE/gen_db_tests/wormbase-acedb-gene.properties

and this brings us to the current error, that I wasn’t able to make the build proceed:

BUILD FAILED
/mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
/mnt/data/intermine/imbuild/source.xml:145: javax.xml.transform.TransformerException: Unknown nodetype: node-name
        at org.apache.xpath.compiler.XPathParser.error(XPathParser.java:608)


with a build.log tail with

ant] -retrieve-tgt-from-custom-file:
 [ant] [integrate] DataSet set to:AceDB XML (Gene)
 [ant] [integrate] Processed key file: [../../bio/sources/wormbase-acedb/resources/wormbase-acedb_keys.properties]
 [ant] [integrate] Processed mapping file: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties
 [ant] [integrate] Processing file: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/XML/prepped_gene.xml
 [ant] [integrate] JDJDLOG:: XML rejects file set to:/mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/wormbase-acedb-gene-rejects.xml
 [ant]      [null] Exiting /mnt/data/intermine/bio/sources/wormbase-acedb/build.xml.
Fri Mar  6 05:36:04 UTC 2015

finished


failed with exit code 0: ant -v -Dsource=wb-acedb-gene


I don’t know if these errors during the build process are related, but I am looking to any possible explanation and help. The build to get to this stage takes a couple of hours to complete, so I have a modified version of the build.xml just to build this step.

Thanks in advance

Paulo










_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Intermine build errors

Todd Harris-2
Hi Paulo  -

Quick notes.

1. We supply those mapping files, not Intermine. I’m not sure *why* they are stashed in the intermine repository?  I haven’t found the scripts for generating those files for current releases (and other speceis) but they would be easy to draft. I can do so.

2. I found a different version of the gene properties mapping file. I just copied the whole directory over like the previous one as there are likely to be other files that need to be scrounged.  Please ensure these find their way into the git repo. Obviously would have been easier for us if these files had been committed initially…

The variant is at:

/mnt/data/archive/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties

3. I think the last error is probably due to the XPath entries in the properties file do not match the XML dump. This may be due to a) an obsolete version of the properties file or b) divergent XML — the acedb model changed, and thus the resulting XML dumps have changed.

(The whole process uses xpath queries to extract data from the xml dumps)

Todd

> On Mar 12, 2015, at 7:51 PM, Paulo Nuin <[hidden email]> wrote:
>
> Hi everyone
>
> This is my first message to the list. I am currently working on deploying a build of Intermine on AWS and I am running into some problems on the build process. I will add a couple of errors below and some background on the process to solve them and what work and what didn’t. Any help is appreciated.
>
>
> The first blocking error was
>
> starting command: ant -v -Dsource=celegans-gff3-gene
>
>
> BUILD FAILED
> /mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
> /mnt/data/intermine/imbuild/source.xml:256: java.io.FileNotFoundException: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-gff3/mapping/id_mapping.tab (No such file or directory)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.<init>(FileInputStream.java:146)
> …
>
> as no mapping file was available in the required directory. I tried searching the repository online, googling it and wasn’t able to find this file. Todd Harris, whom I am helping setting this up, found mapping files in a section of the source code, but we are not sure if these files are created or not during the process. At that point I was transferring all files from most of the release 246 of worm base, but with the mapping file used, only C elegans was able to be used in the build. The error we were getting with all sequences was
>
> BUILD FAILED
> /mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
> /mnt/data/intermine/imbuild/source.xml:256: java.lang.RuntimeException: IOException while getting next GFF record
> at org.intermine.bio.io.gff3.GFF3Parser$1.next(GFF3Parser.java:78)
> …
>
> with the workaround being just transferring C elegans. The build went on a couple more steps and failed at
>
> /mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
> /mnt/data/intermine/imbuild/source.xml:145: failed to set value for mappingFile to /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@41e0c4c4
>
> Caused by: java.io.FileNotFoundException: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties (No such file or directory)
>
> The only similar file I found that was able to make the process continue was
>
> /mnt/data/website-intermine-dev/acedb-dev/intermine/testlab/perl/OBSOLETE/gen_db_tests/wormbase-acedb-gene.properties
>
> and this brings us to the current error, that I wasn’t able to make the build proceed:
>
> BUILD FAILED
> /mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
> /mnt/data/intermine/imbuild/source.xml:145: javax.xml.transform.TransformerException: Unknown nodetype: node-name
> at org.apache.xpath.compiler.XPathParser.error(XPathParser.java:608)
> …
>
> with a build.log tail with
>
> ant] -retrieve-tgt-from-custom-file:
> [ant] [integrate] DataSet set to:AceDB XML (Gene)
> [ant] [integrate] Processed key file: [../../bio/sources/wormbase-acedb/resources/wormbase-acedb_keys.properties]
> [ant] [integrate] Processed mapping file: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties
> [ant] [integrate] Processing file: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/XML/prepped_gene.xml
> [ant] [integrate] JDJDLOG:: XML rejects file set to:/mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/wormbase-acedb-gene-rejects.xml
> [ant]      [null] Exiting /mnt/data/intermine/bio/sources/wormbase-acedb/build.xml.
> Fri Mar  6 05:36:04 UTC 2015
>
> finished
>
>
> failed with exit code 0: ant -v -Dsource=wb-acedb-gene
>
>
> I don’t know if these errors during the build process are related, but I am looking to any possible explanation and help. The build to get to this stage takes a couple of hours to complete, so I have a modified version of the build.xml just to build this step.
>
> Thanks in advance
>
> Paulo
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> dev mailing list
> [hidden email]
> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Intermine build errors

Paulo Nuin
Hi Todd

I added the previous errors as I didn’t know if they were somewhat connected.

> On Mar 12, 2015, at 9:30 PM, Todd Harris <[hidden email]> wrote:
>
> Hi Paulo  -
>
> Quick notes.
>
> 1. We supply those mapping files, not Intermine. I’m not sure *why* they are stashed in the intermine repository?  I haven’t found the scripts for generating those files for current releases (and other speceis) but they would be easy to draft. I can do so.
>

Give me some pointers and I will work on generating then, I can add the procedure to the deployment build script that I am working on.


> 2. I found a different version of the gene properties mapping file. I just copied the whole directory over like the previous one as there are likely to be other files that need to be scrounged.  Please ensure these find their way into the git repo. Obviously would have been easier for us if these files had been committed initially…
>
> The variant is at:
>
> /mnt/data/archive/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties
>

Thanks, I am checking them.

> 3. I think the last error is probably due to the XPath entries in the properties file do not match the XML dump. This may be due to a) an obsolete version of the properties file or b) divergent XML — the acedb model changed, and thus the resulting XML dumps have changed.
>
> (The whole process uses xpath queries to extract data from the xml dumps)
>

I have the limited project.xml file with the section that causes the error, I will try to find the culprit.

Cheers
Paulo




> Todd
>
>> On Mar 12, 2015, at 7:51 PM, Paulo Nuin <[hidden email]> wrote:
>>
>> Hi everyone
>>
>> This is my first message to the list. I am currently working on deploying a build of Intermine on AWS and I am running into some problems on the build process. I will add a couple of errors below and some background on the process to solve them and what work and what didn’t. Any help is appreciated.
>>
>>
>> The first blocking error was
>>
>> starting command: ant -v -Dsource=celegans-gff3-gene
>>
>>
>> BUILD FAILED
>> /mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
>> /mnt/data/intermine/imbuild/source.xml:256: java.io.FileNotFoundException: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-gff3/mapping/id_mapping.tab (No such file or directory)
>> at java.io.FileInputStream.open(Native Method)
>> at java.io.FileInputStream.<init>(FileInputStream.java:146)
>> …
>>
>> as no mapping file was available in the required directory. I tried searching the repository online, googling it and wasn’t able to find this file. Todd Harris, whom I am helping setting this up, found mapping files in a section of the source code, but we are not sure if these files are created or not during the process. At that point I was transferring all files from most of the release 246 of worm base, but with the mapping file used, only C elegans was able to be used in the build. The error we were getting with all sequences was
>>
>> BUILD FAILED
>> /mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
>> /mnt/data/intermine/imbuild/source.xml:256: java.lang.RuntimeException: IOException while getting next GFF record
>> at org.intermine.bio.io.gff3.GFF3Parser$1.next(GFF3Parser.java:78)
>> …
>>
>> with the workaround being just transferring C elegans. The build went on a couple more steps and failed at
>>
>> /mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
>> /mnt/data/intermine/imbuild/source.xml:145: failed to set value for mappingFile to /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@41e0c4c4
>>
>> Caused by: java.io.FileNotFoundException: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties (No such file or directory)
>>
>> The only similar file I found that was able to make the process continue was
>>
>> /mnt/data/website-intermine-dev/acedb-dev/intermine/testlab/perl/OBSOLETE/gen_db_tests/wormbase-acedb-gene.properties
>>
>> and this brings us to the current error, that I wasn’t able to make the build proceed:
>>
>> BUILD FAILED
>> /mnt/data/intermine/imbuild/integrate.xml:54: The following error occurred while executing this line:
>> /mnt/data/intermine/imbuild/source.xml:145: javax.xml.transform.TransformerException: Unknown nodetype: node-name
>> at org.apache.xpath.compiler.XPathParser.error(XPathParser.java:608)
>> …
>>
>> with a build.log tail with
>>
>> ant] -retrieve-tgt-from-custom-file:
>> [ant] [integrate] DataSet set to:AceDB XML (Gene)
>> [ant] [integrate] Processed key file: [../../bio/sources/wormbase-acedb/resources/wormbase-acedb_keys.properties]
>> [ant] [integrate] Processed mapping file: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/mapping/wormbase-acedb-gene.properties
>> [ant] [integrate] Processing file: /mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/XML/prepped_gene.xml
>> [ant] [integrate] JDJDLOG:: XML rejects file set to:/mnt/data/website-intermine/acedb-dev/intermine/datadir/wormbase-acedb/gene/wormbase-acedb-gene-rejects.xml
>> [ant]      [null] Exiting /mnt/data/intermine/bio/sources/wormbase-acedb/build.xml.
>> Fri Mar  6 05:36:04 UTC 2015
>>
>> finished
>>
>>
>> failed with exit code 0: ant -v -Dsource=wb-acedb-gene
>>
>>
>> I don’t know if these errors during the build process are related, but I am looking to any possible explanation and help. The build to get to this stage takes a couple of hours to complete, so I have a modified version of the build.xml just to build this step.
>>
>> Thanks in advance
>>
>> Paulo
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> http://mail.intermine.org/cgi-bin/mailman/listinfo/dev
>


_______________________________________________
dev mailing list
[hidden email]
http://mail.intermine.org/cgi-bin/mailman/listinfo/dev