Debugging Entrez species step

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Debugging Entrez species step

Paulo Nuin
Hi everyone

We are having issues (again) with the Entrez step to load species features in the organism DB. It’s out last step of the build and earlier when we had the same problem we added species/taxon IDs to our XML that were loaded in the DB and not listed in it. At this time, it seems that all species are listed and loaded from our XML, but the Entrez step is still failing after the build is complete.

Running the step by itself doesn’t output any problems, just at the end of the build. Is there a straightforward to debug the Entrez step?

Thanks

Paulo

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Debugging Entrez species step

sergio contrino-2
hi paul, could you please send the logs of when it failed?
the entrez step involves calls to ncbi, so there could be external
factors as well.
thanks
sergio

On 18/12/2018 22:47, Paulo Nuin wrote:

> Hi everyone
>
> We are having issues (again) with the Entrez step to load species features in the organism DB. It’s out last step of the build and earlier when we had the same problem we added species/taxon IDs to our XML that were loaded in the DB and not listed in it. At this time, it seems that all species are listed and loaded from our XML, but the Entrez step is still failing after the build is complete.
>
> Running the step by itself doesn’t output any problems, just at the end of the build. Is there a straightforward to debug the Entrez step?
>
> Thanks
>
> Paulo
>
> _______________________________________________
> dev mailing list
> [hidden email]
> https://lists.intermine.org/mailman/listinfo/dev
>

--
sergio contrino                  InterMine, University of Cambridge
https://sergiocontrino.github.io           http://www.intermine.org
_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Debugging Entrez species step

Paulo Nuin
Hi Sergio

I don’t have the log anymore, but it basically showed that the step failed. The XML file created by it has just

<item>

in it, nothing else.

Cheers
Paulo



> On Dec 19, 2018, at 5:29 AM, sergio contrino <[hidden email]> wrote:
>
> hi paul, could you please send the logs of when it failed?
> the entrez step involves calls to ncbi, so there could be external factors as well.
> thanks
> sergio
>
> On 18/12/2018 22:47, Paulo Nuin wrote:
>> Hi everyone
>> We are having issues (again) with the Entrez step to load species features in the organism DB. It’s out last step of the build and earlier when we had the same problem we added species/taxon IDs to our XML that were loaded in the DB and not listed in it. At this time, it seems that all species are listed and loaded from our XML, but the Entrez step is still failing after the build is complete.
>> Running the step by itself doesn’t output any problems, just at the end of the build. Is there a straightforward to debug the Entrez step?
>> Thanks
>> Paulo
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> https://lists.intermine.org/mailman/listinfo/dev
>
> --
> sergio contrino                  InterMine, University of Cambridge
> https://sergiocontrino.github.io           http://www.intermine.org
> _______________________________________________
> dev mailing list
> [hidden email]
> https://lists.intermine.org/mailman/listinfo/dev

_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Debugging Entrez species step

sergio contrino-2
hi paulo,
would it be possible for you to run it again?
i would suggest you add an intermediate dump just before the
entrez-organism step, so you can retry the step without losing all the
build up to that moment (see below for example)
then you could share the integration log file (and also test if it is a
temporary network issue)
thanks!
sergio


in your project.xml file:
[..]
<source name="ensembl-compara-yeast" type="ensembl-compara" dump="true">
...
<source name="entrez-organism" type="entrez-organism">
[..]

you can then restart your build after failure from the entrez-organism
step (project_build script with -r switch)


On 19/12/2018 18:28, Paulo Nuin wrote:

> Hi Sergio
>
> I don’t have the log anymore, but it basically showed that the step failed. The XML file created by it has just
>
> <item>
>
> in it, nothing else.
>
> Cheers
> Paulo
>
>
>
>> On Dec 19, 2018, at 5:29 AM, sergio contrino <[hidden email]> wrote:
>>
>> hi paul, could you please send the logs of when it failed?
>> the entrez step involves calls to ncbi, so there could be external factors as well.
>> thanks
>> sergio
>>
>> On 18/12/2018 22:47, Paulo Nuin wrote:
>>> Hi everyone
>>> We are having issues (again) with the Entrez step to load species features in the organism DB. It’s out last step of the build and earlier when we had the same problem we added species/taxon IDs to our XML that were loaded in the DB and not listed in it. At this time, it seems that all species are listed and loaded from our XML, but the Entrez step is still failing after the build is complete.
>>> Running the step by itself doesn’t output any problems, just at the end of the build. Is there a straightforward to debug the Entrez step?
>>> Thanks
>>> Paulo
>>> _______________________________________________
>>> dev mailing list
>>> [hidden email]
>>> https://lists.intermine.org/mailman/listinfo/dev
>>
>> --
>> sergio contrino                  InterMine, University of Cambridge
>> https://sergiocontrino.github.io           http://www.intermine.org
>> _______________________________________________
>> dev mailing list
>> [hidden email]
>> https://lists.intermine.org/mailman/listinfo/dev
>

--
sergio contrino                  InterMine, University of Cambridge
https://sergiocontrino.github.io           http://www.intermine.org
_______________________________________________
dev mailing list
[hidden email]
https://lists.intermine.org/mailman/listinfo/dev