Strange error on build

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Strange error on build

Paulo Nuin
Hi everyone

I am working on build WormMine and getting a strange error that haven’t occurred before, at least in this condition. I discussed with Julie, and we look at some common possibilities, and I tried extra things to no success. Basically, I am getting an error on a step where the build is looking for a mapping file and not finding it, but the file is there.

starting command: ant -v -Dsource=wb-acedb-species


BUILD FAILED
/mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error occurred while executing this line:
/mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value for mappingFile to ../../datadir/wormbase-acedb/species/species_m
apping.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@56a3d7ca

and

➜  ls ../../datadir/wormbase-acedb/species/species_mapping.properties
../../datadir/wormbase-acedb/species/species_mapping.properties

Things we tried:

permissions for file
permissions for directory
change the file location (dirtree and project.xml)
creating a new file
running the step independently
new copy of the code
and other smaller things that didn’t work.

All other loading steps that required a mapping file worked perfectly, as this is our last step in the load.

Any help is appreciated.

Thanks

Paulo

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange error on build

Justin Clark-Casey-2
Hi Paulo,

If this is via the method at [1] then my guess is that it is actually finding the file but failing in the load itself.  In my experience, InterMine is not doing
a good job of surfacing exceptions that are actually causing build tasks to fail - I haven't yet had an opportunity to look into our Ant setup to find out why
this is, though I suspect it might be something to do with the way that InterMine is calling sub-projects to perform the build, integrate, etc. tasks

At the moment I'm restoring to pumping out the exception on System.err at which point I think it shows up in pbuild.log.  My second guess here is that your
mappingFile is malformed in some way since it looks like your code is expecting to ingest something in java.util.Properties format.

[1] https://github.com/WormBase/intermine/blob/dev/bio/sources/wormbase-acedb/main/src/org/intermine/bio/dataconversion/WormbaseAcedbConverter.java#L539

On 17/09/16 15:16, Paulo Nuin wrote:

> Hi everyone
>
> I am working on build WormMine and getting a strange error that haven’t occurred before, at least in this condition. I discussed with Julie, and we look at some common possibilities, and I tried extra things to no success. Basically, I am getting an error on a step where the build is looking for a mapping file and not finding it, but the file is there.
>
> starting command: ant -v -Dsource=wb-acedb-species
>
>
> BUILD FAILED
> /mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error occurred while executing this line:
> /mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value for mappingFile to ../../datadir/wormbase-acedb/species/species_m
> apping.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@56a3d7ca
>
> and
>
> ➜  ls ../../datadir/wormbase-acedb/species/species_mapping.properties
> ../../datadir/wormbase-acedb/species/species_mapping.properties
>
> Things we tried:
>
> permissions for file
> permissions for directory
> change the file location (dirtree and project.xml)
> creating a new file
> running the step independently
> new copy of the code
> and other smaller things that didn’t work.
>
> All other loading steps that required a mapping file worked perfectly, as this is our last step in the load.
>
> Any help is appreciated.
>
> Thanks
>
> Paulo
>
> _______________________________________________
> dev mailing list
> [hidden email]
> https://lists.intermine.org/mailman/listinfo/dev
>
_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange error on build

Paulo Nuin
Thanks, Justin.

I currently have a test build with a small dataset and our action development build. I checked both mapping files and they are identical, and our test build worked perfectly.

And yes, that’s the method we’re using to load the data. Is there any thing that I can use to debug the error? In running the source by itself, I get the same error output.

Thanks again

Paulo




> On Sep 19, 2016, at 8:55 AM, Justin Clark-Casey <[hidden email]> wrote:
>
> Hi Paulo,
>
> If this is via the method at [1] then my guess is that it is actually finding the file but failing in the load itself.  In my experience, InterMine is not doing a good job of surfacing exceptions that are actually causing build tasks to fail - I haven't yet had an opportunity to look into our Ant setup to find out why this is, though I suspect it might be something to do with the way that InterMine is calling sub-projects to perform the build, integrate, etc. tasks
>
> At the moment I'm restoring to pumping out the exception on System.err at which point I think it shows up in pbuild.log.  My second guess here is that your mappingFile is malformed in some way since it looks like your code is expecting to ingest something in java.util.Properties format.
>
> [1] https://github.com/WormBase/intermine/blob/dev/bio/sources/wormbase-acedb/main/src/org/intermine/bio/dataconversion/WormbaseAcedbConverter.java#L539
>
> On 17/09/16 15:16, Paulo Nuin wrote:
>> Hi everyone
>>
>> I am working on build WormMine and getting a strange error that haven’t occurred before, at least in this condition. I discussed with Julie, and we look at some common possibilities, and I tried extra things to no success. Basically, I am getting an error on a step where the build is looking for a mapping file and not finding it, but the file is there.
>>
>> starting command: ant -v -Dsource=wb-acedb-species
>>
>>
>> BUILD FAILED
>> /mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error occurred while executing this line:
>> /mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value for mappingFile to ../../datadir/wormbase-acedb/species/species_m
>> apping.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@56a3d7ca
>>
>> and
>>
>> ➜  ls ../../datadir/wormbase-acedb/species/species_mapping.properties
>> ../../datadir/wormbase-acedb/species/species_mapping.properties
>>
>> Things we tried:
>>
>> permissions for file
>> permissions for directory
>> change the file location (dirtree and project.xml)
>> creating a new file
>> running the step independently
>> new copy of the code
>> and other smaller things that didn’t work.
>>
>> All other loading steps that required a mapping file worked perfectly, as this is our last step in the load.
>>
>> Any help is appreciated.
>>
>> Thanks
>>
>> Paulo
>>
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> https://lists.intermine.org/mailman/listinfo/dev
>>
> _______________________________________________
> dev mailing list
> [hidden email]
> https://lists.intermine.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange error on build

Justin Clark-Casey-2
Perhaps you could temporarily change setMappingFile to be

     public void setMappingFile(String mappingFile) throws Exception{
         try {
             dataMapping = new DataMapper();
             try {
                 dataMapping.load(new FileReader(mappingFile));
             } catch (FileNotFoundException e) {
                 wmd.debug("ERROR: "+mappingFile+" not found");
                 throw e;
             }
         } catch (Exception e) {
             System.err.println(e);
             throw e;
         }

         System.out.println("Processed mapping file: "+mappingFile);
     }

This would show or eliminate the possibility that an exception is being generated at the property parsing stage.

If it was failing at this stage due to not finding the file, then we would expect to see the FileNotFoundException caught with the log message "ERROR:
"+mappingFile+" not found".  Otherwise, hunting for the error gets pushed back to DynamicAttributeTask.configureDynamicAttributes() (where you can see the
BuildException message that you're seeing being generated at the end).  This is pretty generic code so I am doubtful your error is happening due to the code not
finding the file.

--
Justin Clark-Casey, Synbiomine/InterMine Developer
http://synbiomine.org
http://twitter.com/justincc

On 19/09/16 16:20, Paulo Nuin wrote:

> Thanks, Justin.
>
> I currently have a test build with a small dataset and our action development build. I checked both mapping files and they are identical, and our test build worked perfectly.
>
> And yes, that’s the method we’re using to load the data. Is there any thing that I can use to debug the error? In running the source by itself, I get the same error output.
>
> Thanks again
>
> Paulo
>
>
>
>
>> On Sep 19, 2016, at 8:55 AM, Justin Clark-Casey <[hidden email]> wrote:
>>
>> Hi Paulo,
>>
>> If this is via the method at [1] then my guess is that it is actually finding the file but failing in the load itself.  In my experience, InterMine is not doing a good job of surfacing exceptions that are actually causing build tasks to fail - I haven't yet had an opportunity to look into our Ant setup to find out why this is, though I suspect it might be something to do with the way that InterMine is calling sub-projects to perform the build, integrate, etc. tasks
>>
>> At the moment I'm restoring to pumping out the exception on System.err at which point I think it shows up in pbuild.log.  My second guess here is that your mappingFile is malformed in some way since it looks like your code is expecting to ingest something in java.util.Properties format.
>>
>> [1] https://github.com/WormBase/intermine/blob/dev/bio/sources/wormbase-acedb/main/src/org/intermine/bio/dataconversion/WormbaseAcedbConverter.java#L539
>>
>> On 17/09/16 15:16, Paulo Nuin wrote:
>>> Hi everyone
>>>
>>> I am working on build WormMine and getting a strange error that haven’t occurred before, at least in this condition. I discussed with Julie, and we look at some common possibilities, and I tried extra things to no success. Basically, I am getting an error on a step where the build is looking for a mapping file and not finding it, but the file is there.
>>>
>>> starting command: ant -v -Dsource=wb-acedb-species
>>>
>>>
>>> BUILD FAILED
>>> /mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error occurred while executing this line:
>>> /mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value for mappingFile to ../../datadir/wormbase-acedb/species/species_m
>>> apping.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@56a3d7ca
>>>
>>> and
>>>
>>> ➜  ls ../../datadir/wormbase-acedb/species/species_mapping.properties
>>> ../../datadir/wormbase-acedb/species/species_mapping.properties
>>>
>>> Things we tried:
>>>
>>> permissions for file
>>> permissions for directory
>>> change the file location (dirtree and project.xml)
>>> creating a new file
>>> running the step independently
>>> new copy of the code
>>> and other smaller things that didn’t work.
>>>
>>> All other loading steps that required a mapping file worked perfectly, as this is our last step in the load.
>>>
>>> Any help is appreciated.
>>>
>>> Thanks
>>>
>>> Paulo
>>>
>>> _______________________________________________
>>> dev mailing list
>>> [hidden email]
>>> https://lists.intermine.org/mailman/listinfo/dev
>>>
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> https://lists.intermine.org/mailman/listinfo/dev
>
_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange error on build

Paulo Nuin
Hi Justin

It seems that the error is actually a file not found problem:

-retrieve-tgt-from-custom-file:
[integrate] Processed key file: [../../bio/sources/wormbase-acedb/resources/wormbase-acedb_keys.properties]
[integrate] error############################################
[integrate] java.io.FileNotFoundException: ../../datadir/wormbase-acedb/species/mapping/species_mapping.properties (No such file or directory)
     [null] Exiting /mnt/data2/intermine_dev/bio/sources/wormbase-acedb/build.xml.

BUILD FAILED
/mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error occurred while executing this line:
/mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value for mappingFile to ../../datadir/wormbase-acedb/species/mapping/species_mapping.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@d95671e
        at org.intermine.task.DynamicAttributeTask.configureDynamicAttributes(DynamicAttributeTask.java:102)


I added the error#### line just to see if I was actually on the right function. Any other ideas?

Cheers
Paulo



> On Sep 19, 2016, at 9:44 AM, Justin Clark-Casey <[hidden email]> wrote:
>
> Perhaps you could temporarily change setMappingFile to be
>
>    public void setMappingFile(String mappingFile) throws Exception{
>        try {
>            dataMapping = new DataMapper();
>            try {
>                dataMapping.load(new FileReader(mappingFile));
>            } catch (FileNotFoundException e) {
>                wmd.debug("ERROR: "+mappingFile+" not found");
>                throw e;
>            }
>        } catch (Exception e) {
>            System.err.println(e);
>            throw e;
>        }
>
>        System.out.println("Processed mapping file: "+mappingFile);
>    }
>
> This would show or eliminate the possibility that an exception is being generated at the property parsing stage.
>
> If it was failing at this stage due to not finding the file, then we would expect to see the FileNotFoundException caught with the log message "ERROR: "+mappingFile+" not found".  Otherwise, hunting for the error gets pushed back to DynamicAttributeTask.configureDynamicAttributes() (where you can see the BuildException message that you're seeing being generated at the end).  This is pretty generic code so I am doubtful your error is happening due to the code not finding the file.
>
> --
> Justin Clark-Casey, Synbiomine/InterMine Developer
> http://synbiomine.org
> http://twitter.com/justincc
>
> On 19/09/16 16:20, Paulo Nuin wrote:
>> Thanks, Justin.
>>
>> I currently have a test build with a small dataset and our action development build. I checked both mapping files and they are identical, and our test build worked perfectly.
>>
>> And yes, that’s the method we’re using to load the data. Is there any thing that I can use to debug the error? In running the source by itself, I get the same error output.
>>
>> Thanks again
>>
>> Paulo
>>
>>
>>
>>
>>> On Sep 19, 2016, at 8:55 AM, Justin Clark-Casey <[hidden email]> wrote:
>>>
>>> Hi Paulo,
>>>
>>> If this is via the method at [1] then my guess is that it is actually finding the file but failing in the load itself.  In my experience, InterMine is not doing a good job of surfacing exceptions that are actually causing build tasks to fail - I haven't yet had an opportunity to look into our Ant setup to find out why this is, though I suspect it might be something to do with the way that InterMine is calling sub-projects to perform the build, integrate, etc. tasks
>>>
>>> At the moment I'm restoring to pumping out the exception on System.err at which point I think it shows up in pbuild.log.  My second guess here is that your mappingFile is malformed in some way since it looks like your code is expecting to ingest something in java.util.Properties format.
>>>
>>> [1] https://github.com/WormBase/intermine/blob/dev/bio/sources/wormbase-acedb/main/src/org/intermine/bio/dataconversion/WormbaseAcedbConverter.java#L539
>>>
>>> On 17/09/16 15:16, Paulo Nuin wrote:
>>>> Hi everyone
>>>>
>>>> I am working on build WormMine and getting a strange error that haven’t occurred before, at least in this condition. I discussed with Julie, and we look at some common possibilities, and I tried extra things to no success. Basically, I am getting an error on a step where the build is looking for a mapping file and not finding it, but the file is there.
>>>>
>>>> starting command: ant -v -Dsource=wb-acedb-species
>>>>
>>>>
>>>> BUILD FAILED
>>>> /mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error occurred while executing this line:
>>>> /mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value for mappingFile to ../../datadir/wormbase-acedb/species/species_m
>>>> apping.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@56a3d7ca
>>>>
>>>> and
>>>>
>>>> ➜  ls ../../datadir/wormbase-acedb/species/species_mapping.properties
>>>> ../../datadir/wormbase-acedb/species/species_mapping.properties
>>>>
>>>> Things we tried:
>>>>
>>>> permissions for file
>>>> permissions for directory
>>>> change the file location (dirtree and project.xml)
>>>> creating a new file
>>>> running the step independently
>>>> new copy of the code
>>>> and other smaller things that didn’t work.
>>>>
>>>> All other loading steps that required a mapping file worked perfectly, as this is our last step in the load.
>>>>
>>>> Any help is appreciated.
>>>>
>>>> Thanks
>>>>
>>>> Paulo
>>>>
>>>> _______________________________________________
>>>> dev mailing list
>>>> [hidden email]
>>>> https://lists.intermine.org/mailman/listinfo/dev
>>>>
>>> _______________________________________________
>>> dev mailing list
>>> [hidden email]
>>> https://lists.intermine.org/mailman/listinfo/dev
>>

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange error on build

Paulo Nuin
Hi

It seems that the error is with the location (disk mount) itself. We have on AWS three mounts, one for the data directory for WormMine and there’s where we have the files located. All other mapping files were successfully read, but as I mentioned, not the species. I decided to move it to another mount point and at least the independent build worked fine this time. I am assuming, that particular directory in the mount wasn’t being properly accessed by the build.

Cheers
Paulo


> On Sep 19, 2016, at 11:13 AM, Paulo Nuin <[hidden email]> wrote:
>
> Hi Justin
>
> It seems that the error is actually a file not found problem:
>
> -retrieve-tgt-from-custom-file:
> [integrate] Processed key file: [../../bio/sources/wormbase-acedb/resources/wormbase-acedb_keys.properties]
> [integrate] error############################################
> [integrate] java.io.FileNotFoundException: ../../datadir/wormbase-acedb/species/mapping/species_mapping.properties (No such file or directory)
>     [null] Exiting /mnt/data2/intermine_dev/bio/sources/wormbase-acedb/build.xml.
>
> BUILD FAILED
> /mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error occurred while executing this line:
> /mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value for mappingFile to ../../datadir/wormbase-acedb/species/mapping/species_mapping.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@d95671e
>        at org.intermine.task.DynamicAttributeTask.configureDynamicAttributes(DynamicAttributeTask.java:102)
>
>
> I added the error#### line just to see if I was actually on the right function. Any other ideas?
>
> Cheers
> Paulo
>
>
>
>> On Sep 19, 2016, at 9:44 AM, Justin Clark-Casey <[hidden email]> wrote:
>>
>> Perhaps you could temporarily change setMappingFile to be
>>
>>   public void setMappingFile(String mappingFile) throws Exception{
>>       try {
>>           dataMapping = new DataMapper();
>>           try {
>>               dataMapping.load(new FileReader(mappingFile));
>>           } catch (FileNotFoundException e) {
>>               wmd.debug("ERROR: "+mappingFile+" not found");
>>               throw e;
>>           }
>>       } catch (Exception e) {
>>           System.err.println(e);
>>           throw e;
>>       }
>>
>>       System.out.println("Processed mapping file: "+mappingFile);
>>   }
>>
>> This would show or eliminate the possibility that an exception is being generated at the property parsing stage.
>>
>> If it was failing at this stage due to not finding the file, then we would expect to see the FileNotFoundException caught with the log message "ERROR: "+mappingFile+" not found".  Otherwise, hunting for the error gets pushed back to DynamicAttributeTask.configureDynamicAttributes() (where you can see the BuildException message that you're seeing being generated at the end).  This is pretty generic code so I am doubtful your error is happening due to the code not finding the file.
>>
>> --
>> Justin Clark-Casey, Synbiomine/InterMine Developer
>> http://synbiomine.org
>> http://twitter.com/justincc
>>
>> On 19/09/16 16:20, Paulo Nuin wrote:
>>> Thanks, Justin.
>>>
>>> I currently have a test build with a small dataset and our action development build. I checked both mapping files and they are identical, and our test build worked perfectly.
>>>
>>> And yes, that’s the method we’re using to load the data. Is there any thing that I can use to debug the error? In running the source by itself, I get the same error output.
>>>
>>> Thanks again
>>>
>>> Paulo
>>>
>>>
>>>
>>>
>>>> On Sep 19, 2016, at 8:55 AM, Justin Clark-Casey <[hidden email]> wrote:
>>>>
>>>> Hi Paulo,
>>>>
>>>> If this is via the method at [1] then my guess is that it is actually finding the file but failing in the load itself.  In my experience, InterMine is not doing a good job of surfacing exceptions that are actually causing build tasks to fail - I haven't yet had an opportunity to look into our Ant setup to find out why this is, though I suspect it might be something to do with the way that InterMine is calling sub-projects to perform the build, integrate, etc. tasks
>>>>
>>>> At the moment I'm restoring to pumping out the exception on System.err at which point I think it shows up in pbuild.log.  My second guess here is that your mappingFile is malformed in some way since it looks like your code is expecting to ingest something in java.util.Properties format.
>>>>
>>>> [1] https://github.com/WormBase/intermine/blob/dev/bio/sources/wormbase-acedb/main/src/org/intermine/bio/dataconversion/WormbaseAcedbConverter.java#L539
>>>>
>>>> On 17/09/16 15:16, Paulo Nuin wrote:
>>>>> Hi everyone
>>>>>
>>>>> I am working on build WormMine and getting a strange error that haven’t occurred before, at least in this condition. I discussed with Julie, and we look at some common possibilities, and I tried extra things to no success. Basically, I am getting an error on a step where the build is looking for a mapping file and not finding it, but the file is there.
>>>>>
>>>>> starting command: ant -v -Dsource=wb-acedb-species
>>>>>
>>>>>
>>>>> BUILD FAILED
>>>>> /mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error occurred while executing this line:
>>>>> /mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value for mappingFile to ../../datadir/wormbase-acedb/species/species_m
>>>>> apping.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@56a3d7ca
>>>>>
>>>>> and
>>>>>
>>>>> ➜  ls ../../datadir/wormbase-acedb/species/species_mapping.properties
>>>>> ../../datadir/wormbase-acedb/species/species_mapping.properties
>>>>>
>>>>> Things we tried:
>>>>>
>>>>> permissions for file
>>>>> permissions for directory
>>>>> change the file location (dirtree and project.xml)
>>>>> creating a new file
>>>>> running the step independently
>>>>> new copy of the code
>>>>> and other smaller things that didn’t work.
>>>>>
>>>>> All other loading steps that required a mapping file worked perfectly, as this is our last step in the load.
>>>>>
>>>>> Any help is appreciated.
>>>>>
>>>>> Thanks
>>>>>
>>>>> Paulo
>>>>>
>>>>> _______________________________________________
>>>>> dev mailing list
>>>>> [hidden email]
>>>>> https://lists.intermine.org/mailman/listinfo/dev
>>>>>
>>>> _______________________________________________
>>>> dev mailing list
>>>> [hidden email]
>>>> https://lists.intermine.org/mailman/listinfo/dev
>>>
>

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange error on build

Justin Clark-Casey-2
How odd.  Glad you found the error, Paulo!  Sounds like it might have
been a generic problem rather than anything InterMine specific?

-- Justin

On 2016-09-19 18:34, Paulo Nuin wrote:

> Hi
>
> It seems that the error is with the location (disk mount) itself. We
> have on AWS three mounts, one for the data directory for WormMine and
> there’s where we have the files located. All other mapping files were
> successfully read, but as I mentioned, not the species. I decided to
> move it to another mount point and at least the independent build
> worked fine this time. I am assuming, that particular directory in the
> mount wasn’t being properly accessed by the build.
>
> Cheers
> Paulo
>
>
>> On Sep 19, 2016, at 11:13 AM, Paulo Nuin <[hidden email]> wrote:
>>
>> Hi Justin
>>
>> It seems that the error is actually a file not found problem:
>>
>> -retrieve-tgt-from-custom-file:
>> [integrate] Processed key file:
>> [../../bio/sources/wormbase-acedb/resources/wormbase-acedb_keys.properties]
>> [integrate] error############################################
>> [integrate] java.io.FileNotFoundException:
>> ../../datadir/wormbase-acedb/species/mapping/species_mapping.properties
>> (No such file or directory)
>>     [null] Exiting
>> /mnt/data2/intermine_dev/bio/sources/wormbase-acedb/build.xml.
>>
>> BUILD FAILED
>> /mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error
>> occurred while executing this line:
>> /mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value
>> for mappingFile to
>> ../../datadir/wormbase-acedb/species/mapping/species_mapping.properties
>> in org.intermine.bio.dataconversion.WormbaseAcedbConverter@d95671e
>>        at
>> org.intermine.task.DynamicAttributeTask.configureDynamicAttributes(DynamicAttributeTask.java:102)
>>
>>
>> I added the error#### line just to see if I was actually on the right
>> function. Any other ideas?
>>
>> Cheers
>> Paulo
>>
>>
>>
>>> On Sep 19, 2016, at 9:44 AM, Justin Clark-Casey
>>> <[hidden email]> wrote:
>>>
>>> Perhaps you could temporarily change setMappingFile to be
>>>
>>>   public void setMappingFile(String mappingFile) throws Exception{
>>>       try {
>>>           dataMapping = new DataMapper();
>>>           try {
>>>               dataMapping.load(new FileReader(mappingFile));
>>>           } catch (FileNotFoundException e) {
>>>               wmd.debug("ERROR: "+mappingFile+" not found");
>>>               throw e;
>>>           }
>>>       } catch (Exception e) {
>>>           System.err.println(e);
>>>           throw e;
>>>       }
>>>
>>>       System.out.println("Processed mapping file: "+mappingFile);
>>>   }
>>>
>>> This would show or eliminate the possibility that an exception is
>>> being generated at the property parsing stage.
>>>
>>> If it was failing at this stage due to not finding the file, then we
>>> would expect to see the FileNotFoundException caught with the log
>>> message "ERROR: "+mappingFile+" not found".  Otherwise, hunting for
>>> the error gets pushed back to
>>> DynamicAttributeTask.configureDynamicAttributes() (where you can see
>>> the BuildException message that you're seeing being generated at the
>>> end).  This is pretty generic code so I am doubtful your error is
>>> happening due to the code not finding the file.
>>>
>>> --
>>> Justin Clark-Casey, Synbiomine/InterMine Developer
>>> http://synbiomine.org
>>> http://twitter.com/justincc
>>>
>>> On 19/09/16 16:20, Paulo Nuin wrote:
>>>> Thanks, Justin.
>>>>
>>>> I currently have a test build with a small dataset and our action
>>>> development build. I checked both mapping files and they are
>>>> identical, and our test build worked perfectly.
>>>>
>>>> And yes, that’s the method we’re using to load the data. Is there
>>>> any thing that I can use to debug the error? In running the source
>>>> by itself, I get the same error output.
>>>>
>>>> Thanks again
>>>>
>>>> Paulo
>>>>
>>>>
>>>>
>>>>
>>>>> On Sep 19, 2016, at 8:55 AM, Justin Clark-Casey
>>>>> <[hidden email]> wrote:
>>>>>
>>>>> Hi Paulo,
>>>>>
>>>>> If this is via the method at [1] then my guess is that it is
>>>>> actually finding the file but failing in the load itself.  In my
>>>>> experience, InterMine is not doing a good job of surfacing
>>>>> exceptions that are actually causing build tasks to fail - I
>>>>> haven't yet had an opportunity to look into our Ant setup to find
>>>>> out why this is, though I suspect it might be something to do with
>>>>> the way that InterMine is calling sub-projects to perform the
>>>>> build, integrate, etc. tasks
>>>>>
>>>>> At the moment I'm restoring to pumping out the exception on
>>>>> System.err at which point I think it shows up in pbuild.log.  My
>>>>> second guess here is that your mappingFile is malformed in some way
>>>>> since it looks like your code is expecting to ingest something in
>>>>> java.util.Properties format.
>>>>>
>>>>> [1]
>>>>> https://github.com/WormBase/intermine/blob/dev/bio/sources/wormbase-acedb/main/src/org/intermine/bio/dataconversion/WormbaseAcedbConverter.java#L539
>>>>>
>>>>> On 17/09/16 15:16, Paulo Nuin wrote:
>>>>>> Hi everyone
>>>>>>
>>>>>> I am working on build WormMine and getting a strange error that
>>>>>> haven’t occurred before, at least in this condition. I discussed
>>>>>> with Julie, and we look at some common possibilities, and I tried
>>>>>> extra things to no success. Basically, I am getting an error on a
>>>>>> step where the build is looking for a mapping file and not finding
>>>>>> it, but the file is there.
>>>>>>
>>>>>> starting command: ant -v -Dsource=wb-acedb-species
>>>>>>
>>>>>>
>>>>>> BUILD FAILED
>>>>>> /mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following
>>>>>> error occurred while executing this line:
>>>>>> /mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set
>>>>>> value for mappingFile to
>>>>>> ../../datadir/wormbase-acedb/species/species_m
>>>>>> apping.properties in
>>>>>> org.intermine.bio.dataconversion.WormbaseAcedbConverter@56a3d7ca
>>>>>>
>>>>>> and
>>>>>>
>>>>>> ➜  ls
>>>>>> ../../datadir/wormbase-acedb/species/species_mapping.properties
>>>>>> ../../datadir/wormbase-acedb/species/species_mapping.properties
>>>>>>
>>>>>> Things we tried:
>>>>>>
>>>>>> permissions for file
>>>>>> permissions for directory
>>>>>> change the file location (dirtree and project.xml)
>>>>>> creating a new file
>>>>>> running the step independently
>>>>>> new copy of the code
>>>>>> and other smaller things that didn’t work.
>>>>>>
>>>>>> All other loading steps that required a mapping file worked
>>>>>> perfectly, as this is our last step in the load.
>>>>>>
>>>>>> Any help is appreciated.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Paulo
>>>>>>
>>>>>> _______________________________________________
>>>>>> dev mailing list
>>>>>> [hidden email]
>>>>>> https://lists.intermine.org/mailman/listinfo/dev
>>>>>>
>>>>> _______________________________________________
>>>>> dev mailing list
>>>>> [hidden email]
>>>>> https://lists.intermine.org/mailman/listinfo/dev
>>>>
>>

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Strange error on build

Paulo Nuin
Hi Justin

I don’t think it’s a InterMine error, maybe more Java related and the way the file system is mounted in the instance. I sent to the list in case you guys saw this on different systems.

Cheers
Paulo


> On Sep 19, 2016, at 4:41 PM, [hidden email] wrote:
>
> How odd.  Glad you found the error, Paulo!  Sounds like it might have been a generic problem rather than anything InterMine specific?
>
> -- Justin
>
> On 2016-09-19 18:34, Paulo Nuin wrote:
>> Hi
>> It seems that the error is with the location (disk mount) itself. We
>> have on AWS three mounts, one for the data directory for WormMine and
>> there’s where we have the files located. All other mapping files were
>> successfully read, but as I mentioned, not the species. I decided to
>> move it to another mount point and at least the independent build
>> worked fine this time. I am assuming, that particular directory in the
>> mount wasn’t being properly accessed by the build.
>> Cheers
>> Paulo
>>> On Sep 19, 2016, at 11:13 AM, Paulo Nuin <[hidden email]> wrote:
>>> Hi Justin
>>> It seems that the error is actually a file not found problem:
>>> -retrieve-tgt-from-custom-file:
>>> [integrate] Processed key file: [../../bio/sources/wormbase-acedb/resources/wormbase-acedb_keys.properties]
>>> [integrate] error############################################
>>> [integrate] java.io.FileNotFoundException: ../../datadir/wormbase-acedb/species/mapping/species_mapping.properties (No such file or directory)
>>>    [null] Exiting /mnt/data2/intermine_dev/bio/sources/wormbase-acedb/build.xml.
>>> BUILD FAILED
>>> /mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error occurred while executing this line:
>>> /mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value for mappingFile to ../../datadir/wormbase-acedb/species/mapping/species_mapping.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@d95671e
>>>       at org.intermine.task.DynamicAttributeTask.configureDynamicAttributes(DynamicAttributeTask.java:102)
>>> I added the error#### line just to see if I was actually on the right function. Any other ideas?
>>> Cheers
>>> Paulo
>>>> On Sep 19, 2016, at 9:44 AM, Justin Clark-Casey <[hidden email]> wrote:
>>>> Perhaps you could temporarily change setMappingFile to be
>>>>  public void setMappingFile(String mappingFile) throws Exception{
>>>>      try {
>>>>          dataMapping = new DataMapper();
>>>>          try {
>>>>              dataMapping.load(new FileReader(mappingFile));
>>>>          } catch (FileNotFoundException e) {
>>>>              wmd.debug("ERROR: "+mappingFile+" not found");
>>>>              throw e;
>>>>          }
>>>>      } catch (Exception e) {
>>>>          System.err.println(e);
>>>>          throw e;
>>>>      }
>>>>      System.out.println("Processed mapping file: "+mappingFile);
>>>>  }
>>>> This would show or eliminate the possibility that an exception is being generated at the property parsing stage.
>>>> If it was failing at this stage due to not finding the file, then we would expect to see the FileNotFoundException caught with the log message "ERROR: "+mappingFile+" not found".  Otherwise, hunting for the error gets pushed back to DynamicAttributeTask.configureDynamicAttributes() (where you can see the BuildException message that you're seeing being generated at the end).  This is pretty generic code so I am doubtful your error is happening due to the code not finding the file.
>>>> --
>>>> Justin Clark-Casey, Synbiomine/InterMine Developer
>>>> http://synbiomine.org
>>>> http://twitter.com/justincc
>>>> On 19/09/16 16:20, Paulo Nuin wrote:
>>>>> Thanks, Justin.
>>>>> I currently have a test build with a small dataset and our action development build. I checked both mapping files and they are identical, and our test build worked perfectly.
>>>>> And yes, that’s the method we’re using to load the data. Is there any thing that I can use to debug the error? In running the source by itself, I get the same error output.
>>>>> Thanks again
>>>>> Paulo
>>>>>> On Sep 19, 2016, at 8:55 AM, Justin Clark-Casey <[hidden email]> wrote:
>>>>>> Hi Paulo,
>>>>>> If this is via the method at [1] then my guess is that it is actually finding the file but failing in the load itself.  In my experience, InterMine is not doing a good job of surfacing exceptions that are actually causing build tasks to fail - I haven't yet had an opportunity to look into our Ant setup to find out why this is, though I suspect it might be something to do with the way that InterMine is calling sub-projects to perform the build, integrate, etc. tasks
>>>>>> At the moment I'm restoring to pumping out the exception on System.err at which point I think it shows up in pbuild.log.  My second guess here is that your mappingFile is malformed in some way since it looks like your code is expecting to ingest something in java.util.Properties format.
>>>>>> [1] https://github.com/WormBase/intermine/blob/dev/bio/sources/wormbase-acedb/main/src/org/intermine/bio/dataconversion/WormbaseAcedbConverter.java#L539
>>>>>> On 17/09/16 15:16, Paulo Nuin wrote:
>>>>>>> Hi everyone
>>>>>>> I am working on build WormMine and getting a strange error that haven’t occurred before, at least in this condition. I discussed with Julie, and we look at some common possibilities, and I tried extra things to no success. Basically, I am getting an error on a step where the build is looking for a mapping file and not finding it, but the file is there.
>>>>>>> starting command: ant -v -Dsource=wb-acedb-species
>>>>>>> BUILD FAILED
>>>>>>> /mnt/data2/intermine_dev/imbuild/integrate.xml:54: The following error occurred while executing this line:
>>>>>>> /mnt/data2/intermine_dev/imbuild/source.xml:145: failed to set value for mappingFile to ../../datadir/wormbase-acedb/species/species_m
>>>>>>> apping.properties in org.intermine.bio.dataconversion.WormbaseAcedbConverter@56a3d7ca
>>>>>>> and
>>>>>>> ➜  ls ../../datadir/wormbase-acedb/species/species_mapping.properties
>>>>>>> ../../datadir/wormbase-acedb/species/species_mapping.properties
>>>>>>> Things we tried:
>>>>>>> permissions for file
>>>>>>> permissions for directory
>>>>>>> change the file location (dirtree and project.xml)
>>>>>>> creating a new file
>>>>>>> running the step independently
>>>>>>> new copy of the code
>>>>>>> and other smaller things that didn’t work.
>>>>>>> All other loading steps that required a mapping file worked perfectly, as this is our last step in the load.
>>>>>>> Any help is appreciated.
>>>>>>> Thanks
>>>>>>> Paulo
>>>>>>> _______________________________________________
>>>>>>> dev mailing list
>>>>>>> [hidden email]
>>>>>>> https://lists.intermine.org/mailman/listinfo/dev
>>>>>> _______________________________________________
>>>>>> dev mailing list
>>>>>> [hidden email]
>>>>>> https://lists.intermine.org/mailman/listinfo/dev
>

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Loading...