Problems with latest Galaxy 8167:b0c916349070

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Problems with latest Galaxy 8167:b0c916349070

Scooter Willis-2
Running a CloudMan created instance on Amazon.

Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070

Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location
galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls
ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq  DistanceEst   MergePaths   PathConsensus  SimpleGraph
abyss2ace  abyss-fac       ABYSS-P         AdjList        KAligner      Overlap      PathOverlap
abyss2afg  abyss-fixmate   abyss-pe        Consensus      MergeContigs  ParseAligns  PopBubbles

Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented. 

Thanks

Scooter


Bowtie

Stdout

Sequence file aligned.
ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt'

Stderr
WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml'
WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'



Using toolshed installed Abyss and get the same error in Stdout.



Stdout

ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt'

Stderr
COMMAND FAILURE: /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_tool/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrapper.sh: line 9: ABYSS: command not found
WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml'
WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'

Looking at the util directory based on the expected path of the read build file

galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls
aliaspickler.py   docutils_ext           heartbeat.py    json.pyc       odict.py           streamball.py
aliaspickler.pyc  docutils_template.txt  inflection.py   lrucache.py    odict.pyc          streamball.pyc
backports         expressions.py         inflection.pyc  lrucache.pyc   sanitize_html.py   template.py
bunch.py          expressions.pyc        __init__.py     memdump.py     sanitize_html.pyc  template.pyc
bunch.pyc         hash_util.py           __init__.pyc    none_like.py   shed_util.py       topsort.py
debugging.py      hash_util.pyc          json.py         none_like.pyc  shed_util.pyc      topsort.pyc











___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Dannon Baker
Unfortunately the cloud instance upgrade path requires some manual intervention here due to tool migrations.  SSH in to your instance, edit /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any references to BlastXML.  Save the file, restart galaxy, and you should be good to go.

-Dannon

On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]> wrote:

> Running a CloudMan created instance on Amazon.
>
> Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central to version 8167:b0c916349070
>
> Tried to run bowtie and get the following error. Searched the message archives an someone else reported the same problem but thought it was related to a new version of cufflinks. Same problem also exists when running abyss. Given the nature of the error message not even sure what the actual path should be. Also for the abyss install from toolshed the toolshed.g2.bx.psu.edu directory doesn't exist or the six directories underneath. However Abyss was/is installed at the following location
>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls
>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq  DistanceEst   MergePaths   PathConsensus  SimpleGraph
>> abyss2ace  abyss-fac       ABYSS-P         AdjList        KAligner      Overlap      PathOverlap
>> abyss2afg  abyss-fixmate   abyss-pe        Consensus      MergeContigs  ParseAligns  PopBubbles
>
> Would be great if someone could give some guidance on how to resolve the problems. I understand if I need to do something under the covers but the expectation is that the one click install/updates should not then require major command line work that isn't documented.
>
> Thanks
>
> Scooter
>
>
> Bowtie
>
> Stdout
>
>> Sequence file aligned.
>> ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt'
>> ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt'
>> ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt'
>
> Stderr
>> WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml'
>> WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
>
>
>
> Using toolshed installed Abyss and get the same error in Stdout.
>
>
>
> Stdout
>
>> ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ucsc/publicbuilds.txt'
>> ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ensembl/builds.txt'
>> ERROR: Unable to read builds file: [Errno 2] No such file or directory: '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/ncbi/builds.txt'
>
> Stderr
>> COMMAND FAILURE: /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_tool/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrapper.sh: line 9: ABYSS: command not found
>> WARNING:galaxy.datatypes.registry:Error loading datatype with extension 'blastxml': 'module' object has no attribute 'BlastXml'
>> WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no attribute 'BlastXml'
>
> Looking at the util directory based on the expected path of the read build file
>
>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls
>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc       odict.py           streamball.py
>> aliaspickler.pyc  docutils_template.txt  inflection.py   lrucache.py    odict.pyc          streamball.pyc
>> backports         expressions.py         inflection.pyc  lrucache.pyc   sanitize_html.py   template.py
>> bunch.py          expressions.pyc        __init__.py     memdump.py     sanitize_html.pyc  template.pyc
>> bunch.pyc         hash_util.py           __init__.pyc    none_like.py   shed_util.py       topsort.py
>> debugging.py      hash_util.pyc          json.py         none_like.pyc  shed_util.pyc      topsort.pyc
>
>
>
>
>
>
>
>
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>  http://lists.bx.psu.edu/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Scooter Willis-2
Ok that fixed Bowtie.

Any thoughts on expected state of installing Abyss from toolshed. It
indicates no requirements so my expectation is that installing from
toolshed should install the appropriate abyss package in a location the
tool is expecting to find it.

In looking at the toolshed looks like two wrappers have been configured.
One by msjeon and the other edward-kirton. I did the install for msjeon.
Will try the other.

Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin
from prior testing and should be deleted. Current release version of abyss
as of May 30 2012 is Abyss 1.3.4




On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:

>Unfortunately the cloud instance upgrade path requires some manual
>intervention here due to tool migrations.  SSH in to your instance, edit
>/mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
>references to BlastXML.  Save the file, restart galaxy, and you should be
>good to go.
>
>-Dannon
>
>On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]> wrote:
>
>> Running a CloudMan created instance on Amazon.
>>
>> Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central
>>to version 8167:b0c916349070
>>
>> Tried to run bowtie and get the following error. Searched the message
>>archives an someone else reported the same problem but thought it was
>>related to a new version of cufflinks. Same problem also exists when
>>running abyss. Given the nature of the error message not even sure what
>>the actual path should be. Also for the abyss install from toolshed the
>>toolshed.g2.bx.psu.edu directory doesn't exist or the six directories
>>underneath. However Abyss was/is installed at the following location
>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls
>>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq  DistanceEst
>>> MergePaths   PathConsensus  SimpleGraph
>>> abyss2ace  abyss-fac       ABYSS-P         AdjList        KAligner
>>> Overlap      PathOverlap
>>> abyss2afg  abyss-fixmate   abyss-pe        Consensus      MergeContigs
>>> ParseAligns  PopBubbles
>>
>> Would be great if someone could give some guidance on how to resolve
>>the problems. I understand if I need to do something under the covers
>>but the expectation is that the one click install/updates should not
>>then require major command line work that isn't documented.
>>
>> Thanks
>>
>> Scooter
>>
>>
>> Bowtie
>>
>> Stdout
>>
>>> Sequence file aligned.
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ucsc/publicbuilds.txt'
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ensembl/builds.txt'
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ncbi/builds.txt'
>>
>> Stderr
>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype
>>>'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no
>>>attribute 'BlastXml'
>>
>>
>>
>> Using toolshed installed Abyss and get the same error in Stdout.
>>
>>
>>
>> Stdout
>>
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ucsc/publicbuilds.txt'
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ensembl/builds.txt'
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ncbi/builds.txt'
>>
>> Stderr
>>> COMMAND FAILURE:
>>>/mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_to
>>>ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrap
>>>per.sh: line 9: ABYSS: command not found
>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype
>>>'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no
>>>attribute 'BlastXml'
>>
>> Looking at the util directory based on the expected path of the read
>>build file
>>
>>>
>>>galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$
>>> ls
>>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
>>> odict.py           streamball.py
>>> aliaspickler.pyc  docutils_template.txt  inflection.py   lrucache.py
>>> odict.pyc          streamball.pyc
>>> backports         expressions.py         inflection.pyc  lrucache.pyc
>>> sanitize_html.py   template.py
>>> bunch.py          expressions.pyc        __init__.py     memdump.py
>>> sanitize_html.pyc  template.pyc
>>> bunch.pyc         hash_util.py           __init__.pyc    none_like.py
>>> shed_util.py       topsort.py
>>> debugging.py      hash_util.pyc          json.py         none_like.pyc
>>> shed_util.pyc      topsort.pyc
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>
>>  http://lists.bx.psu.edu/
>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Scooter Willis-2
In reply to this post by Dannon Baker
I removed two BlastXML references and restarted galaxy and a new compute
node.

This problem still exists and is found at stdout.

Sequence file aligned.
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/
ucsc/publicbuilds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/
ensembl/builds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/
ncbi/builds.txt'


On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:

>Unfortunately the cloud instance upgrade path requires some manual
>intervention here due to tool migrations.  SSH in to your instance, edit
>/mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
>references to BlastXML.  Save the file, restart galaxy, and you should be
>good to go.
>
>-Dannon
>
>On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]> wrote:
>
>> Running a CloudMan created instance on Amazon.
>>
>> Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central
>>to version 8167:b0c916349070
>>
>> Tried to run bowtie and get the following error. Searched the message
>>archives an someone else reported the same problem but thought it was
>>related to a new version of cufflinks. Same problem also exists when
>>running abyss. Given the nature of the error message not even sure what
>>the actual path should be. Also for the abyss install from toolshed the
>>toolshed.g2.bx.psu.edu directory doesn't exist or the six directories
>>underneath. However Abyss was/is installed at the following location
>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls
>>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq  DistanceEst
>>> MergePaths   PathConsensus  SimpleGraph
>>> abyss2ace  abyss-fac       ABYSS-P         AdjList        KAligner
>>> Overlap      PathOverlap
>>> abyss2afg  abyss-fixmate   abyss-pe        Consensus      MergeContigs
>>> ParseAligns  PopBubbles
>>
>> Would be great if someone could give some guidance on how to resolve
>>the problems. I understand if I need to do something under the covers
>>but the expectation is that the one click install/updates should not
>>then require major command line work that isn't documented.
>>
>> Thanks
>>
>> Scooter
>>
>>
>> Bowtie
>>
>> Stdout
>>
>>> Sequence file aligned.
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ucsc/publicbuilds.txt'
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ensembl/builds.txt'
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ncbi/builds.txt'
>>
>> Stderr
>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype
>>>'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no
>>>attribute 'BlastXml'
>>
>>
>>
>> Using toolshed installed Abyss and get the same error in Stdout.
>>
>>
>>
>> Stdout
>>
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ucsc/publicbuilds.txt'
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ensembl/builds.txt'
>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>directory:
>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>ed/ncbi/builds.txt'
>>
>> Stderr
>>> COMMAND FAILURE:
>>>/mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_to
>>>ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrap
>>>per.sh: line 9: ABYSS: command not found
>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype
>>>'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no
>>>attribute 'BlastXml'
>>
>> Looking at the util directory based on the expected path of the read
>>build file
>>
>>>
>>>galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$
>>> ls
>>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
>>> odict.py           streamball.py
>>> aliaspickler.pyc  docutils_template.txt  inflection.py   lrucache.py
>>> odict.pyc          streamball.pyc
>>> backports         expressions.py         inflection.pyc  lrucache.pyc
>>> sanitize_html.py   template.py
>>> bunch.py          expressions.pyc        __init__.py     memdump.py
>>> sanitize_html.pyc  template.pyc
>>> bunch.pyc         hash_util.py           __init__.pyc    none_like.py
>>> shed_util.py       topsort.py
>>> debugging.py      hash_util.pyc          json.py         none_like.pyc
>>> shed_util.pyc      topsort.pyc
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ___________________________________________________________
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>
>>  http://lists.bx.psu.edu/
>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Dannon Baker
In reply to this post by Scooter Willis-2
Neither abyss wrapper in the toolshed installs binaries (you'd see a tool_dependencies.xml in the repository), that's left up to the end user.  You might contact the authors of the wrappers via the toolshed to see if they were going to add dependency management and had a newer version, but otherwise you'll have to install binaries on your own.  The cloud volumes do have a version of the Abyss binaries preinstalled, but it's certainly possible to add a newer version to your particular instance and persist the changes to the tool volume (see the cloud admin panel).


On Nov 8, 2012, at 11:20 AM, Scooter Willis <[hidden email]> wrote:

> Ok that fixed Bowtie.
>
> Any thoughts on expected state of installing Abyss from toolshed. It
> indicates no requirements so my expectation is that installing from
> toolshed should install the appropriate abyss package in a location the
> tool is expecting to find it.
>
> In looking at the toolshed looks like two wrappers have been configured.
> One by msjeon and the other edward-kirton. I did the install for msjeon.
> Will try the other.
>
> Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin
> from prior testing and should be deleted. Current release version of abyss
> as of May 30 2012 is Abyss 1.3.4
>
>
>
>
> On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:
>
>> Unfortunately the cloud instance upgrade path requires some manual
>> intervention here due to tool migrations.  SSH in to your instance, edit
>> /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
>> references to BlastXML.  Save the file, restart galaxy, and you should be
>> good to go.
>>
>> -Dannon
>>
>> On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]> wrote:
>>
>>> Running a CloudMan created instance on Amazon.
>>>
>>> Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central
>>> to version 8167:b0c916349070
>>>
>>> Tried to run bowtie and get the following error. Searched the message
>>> archives an someone else reported the same problem but thought it was
>>> related to a new version of cufflinks. Same problem also exists when
>>> running abyss. Given the nature of the error message not even sure what
>>> the actual path should be. Also for the abyss install from toolshed the
>>> toolshed.g2.bx.psu.edu directory doesn't exist or the six directories
>>> underneath. However Abyss was/is installed at the following location
>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls
>>>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq  DistanceEst
>>>> MergePaths   PathConsensus  SimpleGraph
>>>> abyss2ace  abyss-fac       ABYSS-P         AdjList        KAligner
>>>> Overlap      PathOverlap
>>>> abyss2afg  abyss-fixmate   abyss-pe        Consensus      MergeContigs
>>>> ParseAligns  PopBubbles
>>>
>>> Would be great if someone could give some guidance on how to resolve
>>> the problems. I understand if I need to do something under the covers
>>> but the expectation is that the one click install/updates should not
>>> then require major command line work that isn't documented.
>>>
>>> Thanks
>>>
>>> Scooter
>>>
>>>
>>> Bowtie
>>>
>>> Stdout
>>>
>>>> Sequence file aligned.
>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>> directory:
>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>> ed/ucsc/publicbuilds.txt'
>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>> directory:
>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>> ed/ensembl/builds.txt'
>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>> directory:
>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>> ed/ncbi/builds.txt'
>>>
>>> Stderr
>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype
>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no
>>>> attribute 'BlastXml'
>>>
>>>
>>>
>>> Using toolshed installed Abyss and get the same error in Stdout.
>>>
>>>
>>>
>>> Stdout
>>>
>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>> directory:
>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>> ed/ucsc/publicbuilds.txt'
>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>> directory:
>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>> ed/ensembl/builds.txt'
>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>> directory:
>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shar
>>>> ed/ncbi/builds.txt'
>>>
>>> Stderr
>>>> COMMAND FAILURE:
>>>> /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_to
>>>> ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wrap
>>>> per.sh: line 9: ABYSS: command not found
>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for datatype
>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has no
>>>> attribute 'BlastXml'
>>>
>>> Looking at the util directory based on the expected path of the read
>>> build file
>>>
>>>>
>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$
>>>> ls
>>>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
>>>> odict.py           streamball.py
>>>> aliaspickler.pyc  docutils_template.txt  inflection.py   lrucache.py
>>>> odict.pyc          streamball.pyc
>>>> backports         expressions.py         inflection.pyc  lrucache.pyc
>>>> sanitize_html.py   template.py
>>>> bunch.py          expressions.pyc        __init__.py     memdump.py
>>>> sanitize_html.pyc  template.pyc
>>>> bunch.pyc         hash_util.py           __init__.pyc    none_like.py
>>>> shed_util.py       topsort.py
>>>> debugging.py      hash_util.pyc          json.py         none_like.pyc
>>>> shed_util.pyc      topsort.pyc
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ___________________________________________________________
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>
>>> http://lists.bx.psu.edu/
>>
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Scooter Willis-2
Dannon

If I install abyss to the galaxy tools volume and use cloud admin to do a
snapshot I assume I then need to remove and then add compute nodes? From
the EC2 management console I see two 15GiB EBS volumes from the same
snapshot and I have a master and one compute node running. The other
approach is that the compute nodes have a NFS mount back into the
galaxy_tools volume. Trying to make sure I understand the need to restart
the compute nodes after an install of third party tools.

>From looking at the abyss wrapper it is expecting ABYSS to be found on the
path via a .sh file that gets called. Where would I update the path to a
particular installed tool such that it will be picked up for each compute
node that gets started? I couldn't find any examples of doing a third
party install with discussion about how that install would persist to
future compute nodes. The missing element for me at the moment is where to
make path related changes that will be picked up by compute nodes. I could
hard code the path in the .sh file used by the wrapper. Just trying to
figure out how to use the software as designed.

Thanks

Scooter


On 11/8/12 11:49 AM, "Dannon Baker" <[hidden email]> wrote:

>Neither abyss wrapper in the toolshed installs binaries (you'd see a
>tool_dependencies.xml in the repository), that's left up to the end user.
> You might contact the authors of the wrappers via the toolshed to see if
>they were going to add dependency management and had a newer version, but
>otherwise you'll have to install binaries on your own.  The cloud volumes
>do have a version of the Abyss binaries preinstalled, but it's certainly
>possible to add a newer version to your particular instance and persist
>the changes to the tool volume (see the cloud admin panel).
>
>
>On Nov 8, 2012, at 11:20 AM, Scooter Willis <[hidden email]> wrote:
>
>> Ok that fixed Bowtie.
>>
>> Any thoughts on expected state of installing Abyss from toolshed. It
>> indicates no requirements so my expectation is that installing from
>> toolshed should install the appropriate abyss package in a location the
>> tool is expecting to find it.
>>
>> In looking at the toolshed looks like two wrappers have been configured.
>> One by msjeon and the other edward-kirton. I did the install for msjeon.
>> Will try the other.
>>
>> Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin
>> from prior testing and should be deleted. Current release version of
>>abyss
>> as of May 30 2012 is Abyss 1.3.4
>>
>>
>>
>>
>> On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:
>>
>>> Unfortunately the cloud instance upgrade path requires some manual
>>> intervention here due to tool migrations.  SSH in to your instance,
>>>edit
>>> /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
>>> references to BlastXML.  Save the file, restart galaxy, and you should
>>>be
>>> good to go.
>>>
>>> -Dannon
>>>
>>> On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]>
>>>wrote:
>>>
>>>> Running a CloudMan created instance on Amazon.
>>>>
>>>> Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central
>>>> to version 8167:b0c916349070
>>>>
>>>> Tried to run bowtie and get the following error. Searched the message
>>>> archives an someone else reported the same problem but thought it was
>>>> related to a new version of cufflinks. Same problem also exists when
>>>> running abyss. Given the nature of the error message not even sure
>>>>what
>>>> the actual path should be. Also for the abyss install from toolshed
>>>>the
>>>> toolshed.g2.bx.psu.edu directory doesn't exist or the six directories
>>>> underneath. However Abyss was/is installed at the following location
>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls
>>>>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq  DistanceEst
>>>>> MergePaths   PathConsensus  SimpleGraph
>>>>> abyss2ace  abyss-fac       ABYSS-P         AdjList        KAligner
>>>>> Overlap      PathOverlap
>>>>> abyss2afg  abyss-fixmate   abyss-pe        Consensus
>>>>>MergeContigs
>>>>> ParseAligns  PopBubbles
>>>>
>>>> Would be great if someone could give some guidance on how to resolve
>>>> the problems. I understand if I need to do something under the covers
>>>> but the expectation is that the one click install/updates should not
>>>> then require major command line work that isn't documented.
>>>>
>>>> Thanks
>>>>
>>>> Scooter
>>>>
>>>>
>>>> Bowtie
>>>>
>>>> Stdout
>>>>
>>>>> Sequence file aligned.
>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>> directory:
>>>>>
>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>ar
>>>>> ed/ucsc/publicbuilds.txt'
>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>> directory:
>>>>>
>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>ar
>>>>> ed/ensembl/builds.txt'
>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>> directory:
>>>>>
>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>ar
>>>>> ed/ncbi/builds.txt'
>>>>
>>>> Stderr
>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>datatype
>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has
>>>>>no
>>>>> attribute 'BlastXml'
>>>>
>>>>
>>>>
>>>> Using toolshed installed Abyss and get the same error in Stdout.
>>>>
>>>>
>>>>
>>>> Stdout
>>>>
>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>> directory:
>>>>>
>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>ar
>>>>> ed/ucsc/publicbuilds.txt'
>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>> directory:
>>>>>
>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>ar
>>>>> ed/ensembl/builds.txt'
>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>> directory:
>>>>>
>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>ar
>>>>> ed/ncbi/builds.txt'
>>>>
>>>> Stderr
>>>>> COMMAND FAILURE:
>>>>>
>>>>>/mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_
>>>>>to
>>>>>
>>>>>ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wr
>>>>>ap
>>>>> per.sh: line 9: ABYSS: command not found
>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>datatype
>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has
>>>>>no
>>>>> attribute 'BlastXml'
>>>>
>>>> Looking at the util directory based on the expected path of the read
>>>> build file
>>>>
>>>>>
>>>>>
>>>>>galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/uti
>>>>>l$
>>>>> ls
>>>>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
>>>>> odict.py           streamball.py
>>>>> aliaspickler.pyc  docutils_template.txt  inflection.py   lrucache.py
>>>>> odict.pyc          streamball.pyc
>>>>> backports         expressions.py         inflection.pyc  lrucache.pyc
>>>>> sanitize_html.py   template.py
>>>>> bunch.py          expressions.pyc        __init__.py     memdump.py
>>>>> sanitize_html.pyc  template.pyc
>>>>> bunch.pyc         hash_util.py           __init__.pyc    none_like.py
>>>>> shed_util.py       topsort.py
>>>>> debugging.py      hash_util.pyc          json.py
>>>>>none_like.pyc
>>>>> shed_util.pyc      topsort.pyc
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ___________________________________________________________
>>>> Please keep all replies on the list by using "reply all"
>>>> in your mail client.  To manage your subscriptions to this
>>>> and other Galaxy lists, please use the interface at:
>>>>
>>>> http://lists.bx.psu.edu/
>>>
>>
>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Dannon Baker
You do not need to restart or add/remove worker nodes, the master's tool and data volumes are shared via NFS.

The easiest way to make this work for you in the cloud is probably going to be installing the version of the Abyss binaries you'd like to use manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency management (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and then, after it has been installed, modify the abyss wrapper to include a <requirement type="package">abyss</requirement> tag, and finally persist changes to galaxyTools (which will also restart Galaxy, reloading the tools).

-Dannon


On Nov 8, 2012, at 12:09 PM, Scooter Willis <[hidden email]> wrote:

> Dannon
>
> If I install abyss to the galaxy tools volume and use cloud admin to do a
> snapshot I assume I then need to remove and then add compute nodes? From
> the EC2 management console I see two 15GiB EBS volumes from the same
> snapshot and I have a master and one compute node running. The other
> approach is that the compute nodes have a NFS mount back into the
> galaxy_tools volume. Trying to make sure I understand the need to restart
> the compute nodes after an install of third party tools.
>
> From looking at the abyss wrapper it is expecting ABYSS to be found on the
> path via a .sh file that gets called. Where would I update the path to a
> particular installed tool such that it will be picked up for each compute
> node that gets started? I couldn't find any examples of doing a third
> party install with discussion about how that install would persist to
> future compute nodes. The missing element for me at the moment is where to
> make path related changes that will be picked up by compute nodes. I could
> hard code the path in the .sh file used by the wrapper. Just trying to
> figure out how to use the software as designed.
>
> Thanks
>
> Scooter
>
>
> On 11/8/12 11:49 AM, "Dannon Baker" <[hidden email]> wrote:
>
>> Neither abyss wrapper in the toolshed installs binaries (you'd see a
>> tool_dependencies.xml in the repository), that's left up to the end user.
>> You might contact the authors of the wrappers via the toolshed to see if
>> they were going to add dependency management and had a newer version, but
>> otherwise you'll have to install binaries on your own.  The cloud volumes
>> do have a version of the Abyss binaries preinstalled, but it's certainly
>> possible to add a newer version to your particular instance and persist
>> the changes to the tool volume (see the cloud admin panel).
>>
>>
>> On Nov 8, 2012, at 11:20 AM, Scooter Willis <[hidden email]> wrote:
>>
>>> Ok that fixed Bowtie.
>>>
>>> Any thoughts on expected state of installing Abyss from toolshed. It
>>> indicates no requirements so my expectation is that installing from
>>> toolshed should install the appropriate abyss package in a location the
>>> tool is expecting to find it.
>>>
>>> In looking at the toolshed looks like two wrappers have been configured.
>>> One by msjeon and the other edward-kirton. I did the install for msjeon.
>>> Will try the other.
>>>
>>> Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin
>>> from prior testing and should be deleted. Current release version of
>>> abyss
>>> as of May 30 2012 is Abyss 1.3.4
>>>
>>>
>>>
>>>
>>> On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:
>>>
>>>> Unfortunately the cloud instance upgrade path requires some manual
>>>> intervention here due to tool migrations.  SSH in to your instance,
>>>> edit
>>>> /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
>>>> references to BlastXML.  Save the file, restart galaxy, and you should
>>>> be
>>>> good to go.
>>>>
>>>> -Dannon
>>>>
>>>> On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]>
>>>> wrote:
>>>>
>>>>> Running a CloudMan created instance on Amazon.
>>>>>
>>>>> Updated using cloudman from http://bitbucket.org/galaxy/galaxy-central
>>>>> to version 8167:b0c916349070
>>>>>
>>>>> Tried to run bowtie and get the following error. Searched the message
>>>>> archives an someone else reported the same problem but thought it was
>>>>> related to a new version of cufflinks. Same problem also exists when
>>>>> running abyss. Given the nature of the error message not even sure
>>>>> what
>>>>> the actual path should be. Also for the abyss install from toolshed
>>>>> the
>>>>> toolshed.g2.bx.psu.edu directory doesn't exist or the six directories
>>>>> underneath. However Abyss was/is installed at the following location
>>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls
>>>>>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq  DistanceEst
>>>>>> MergePaths   PathConsensus  SimpleGraph
>>>>>> abyss2ace  abyss-fac       ABYSS-P         AdjList        KAligner
>>>>>> Overlap      PathOverlap
>>>>>> abyss2afg  abyss-fixmate   abyss-pe        Consensus
>>>>>> MergeContigs
>>>>>> ParseAligns  PopBubbles
>>>>>
>>>>> Would be great if someone could give some guidance on how to resolve
>>>>> the problems. I understand if I need to do something under the covers
>>>>> but the expectation is that the one click install/updates should not
>>>>> then require major command line work that isn't documented.
>>>>>
>>>>> Thanks
>>>>>
>>>>> Scooter
>>>>>
>>>>>
>>>>> Bowtie
>>>>>
>>>>> Stdout
>>>>>
>>>>>> Sequence file aligned.
>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>> directory:
>>>>>>
>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>> ar
>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>> directory:
>>>>>>
>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>> ar
>>>>>> ed/ensembl/builds.txt'
>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>> directory:
>>>>>>
>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>> ar
>>>>>> ed/ncbi/builds.txt'
>>>>>
>>>>> Stderr
>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>> datatype
>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has
>>>>>> no
>>>>>> attribute 'BlastXml'
>>>>>
>>>>>
>>>>>
>>>>> Using toolshed installed Abyss and get the same error in Stdout.
>>>>>
>>>>>
>>>>>
>>>>> Stdout
>>>>>
>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>> directory:
>>>>>>
>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>> ar
>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>> directory:
>>>>>>
>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>> ar
>>>>>> ed/ensembl/builds.txt'
>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>> directory:
>>>>>>
>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/sh
>>>>>> ar
>>>>>> ed/ncbi/builds.txt'
>>>>>
>>>>> Stderr
>>>>>> COMMAND FAILURE:
>>>>>>
>>>>>> /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abyss_
>>>>>> to
>>>>>>
>>>>>> ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_wr
>>>>>> ap
>>>>>> per.sh: line 9: ABYSS: command not found
>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>> datatype
>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has
>>>>>> no
>>>>>> attribute 'BlastXml'
>>>>>
>>>>> Looking at the util directory based on the expected path of the read
>>>>> build file
>>>>>
>>>>>>
>>>>>>
>>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/uti
>>>>>> l$
>>>>>> ls
>>>>>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
>>>>>> odict.py           streamball.py
>>>>>> aliaspickler.pyc  docutils_template.txt  inflection.py   lrucache.py
>>>>>> odict.pyc          streamball.pyc
>>>>>> backports         expressions.py         inflection.pyc  lrucache.pyc
>>>>>> sanitize_html.py   template.py
>>>>>> bunch.py          expressions.pyc        __init__.py     memdump.py
>>>>>> sanitize_html.pyc  template.pyc
>>>>>> bunch.pyc         hash_util.py           __init__.pyc    none_like.py
>>>>>> shed_util.py       topsort.py
>>>>>> debugging.py      hash_util.pyc          json.py
>>>>>> none_like.pyc
>>>>>> shed_util.pyc      topsort.pyc
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ___________________________________________________________
>>>>> Please keep all replies on the list by using "reply all"
>>>>> in your mail client.  To manage your subscriptions to this
>>>>> and other Galaxy lists, please use the interface at:
>>>>>
>>>>> http://lists.bx.psu.edu/
>>>>
>>>
>>
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Scooter Willis-2
Thanks! That was the explanation I was looking for and the link for how to

Do you have any suggestions for fixing the error that are still occurring
after running Bowtie?


Sequence file aligned.
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/
ucsc/publicbuilds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/
ensembl/builds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/
ncbi/builds.txt'



On 11/8/12 12:44 PM, "Dannon Baker" <[hidden email]> wrote:

>You do not need to restart or add/remove worker nodes, the master's tool
>and data volumes are shared via NFS.
>
>The easiest way to make this work for you in the cloud is probably going
>to be installing the version of the Abyss binaries you'd like to use
>manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency
>management
>(http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and
>then, after it has been installed, modify the abyss wrapper to include a
><requirement type="package">abyss</requirement> tag, and finally persist
>changes to galaxyTools (which will also restart Galaxy, reloading the
>tools).
>
>-Dannon
>
>
>On Nov 8, 2012, at 12:09 PM, Scooter Willis <[hidden email]> wrote:
>
>> Dannon
>>
>> If I install abyss to the galaxy tools volume and use cloud admin to do
>>a
>> snapshot I assume I then need to remove and then add compute nodes? From
>> the EC2 management console I see two 15GiB EBS volumes from the same
>> snapshot and I have a master and one compute node running. The other
>> approach is that the compute nodes have a NFS mount back into the
>> galaxy_tools volume. Trying to make sure I understand the need to
>>restart
>> the compute nodes after an install of third party tools.
>>
>> From looking at the abyss wrapper it is expecting ABYSS to be found on
>>the
>> path via a .sh file that gets called. Where would I update the path to a
>> particular installed tool such that it will be picked up for each
>>compute
>> node that gets started? I couldn't find any examples of doing a third
>> party install with discussion about how that install would persist to
>> future compute nodes. The missing element for me at the moment is where
>>to
>> make path related changes that will be picked up by compute nodes. I
>>could
>> hard code the path in the .sh file used by the wrapper. Just trying to
>> figure out how to use the software as designed.
>>
>> Thanks
>>
>> Scooter
>>
>>
>> On 11/8/12 11:49 AM, "Dannon Baker" <[hidden email]> wrote:
>>
>>> Neither abyss wrapper in the toolshed installs binaries (you'd see a
>>> tool_dependencies.xml in the repository), that's left up to the end
>>>user.
>>> You might contact the authors of the wrappers via the toolshed to see
>>>if
>>> they were going to add dependency management and had a newer version,
>>>but
>>> otherwise you'll have to install binaries on your own.  The cloud
>>>volumes
>>> do have a version of the Abyss binaries preinstalled, but it's
>>>certainly
>>> possible to add a newer version to your particular instance and persist
>>> the changes to the tool volume (see the cloud admin panel).
>>>
>>>
>>> On Nov 8, 2012, at 11:20 AM, Scooter Willis <[hidden email]>
>>>wrote:
>>>
>>>> Ok that fixed Bowtie.
>>>>
>>>> Any thoughts on expected state of installing Abyss from toolshed. It
>>>> indicates no requirements so my expectation is that installing from
>>>> toolshed should install the appropriate abyss package in a location
>>>>the
>>>> tool is expecting to find it.
>>>>
>>>> In looking at the toolshed looks like two wrappers have been
>>>>configured.
>>>> One by msjeon and the other edward-kirton. I did the install for
>>>>msjeon.
>>>> Will try the other.
>>>>
>>>> Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin
>>>> from prior testing and should be deleted. Current release version of
>>>> abyss
>>>> as of May 30 2012 is Abyss 1.3.4
>>>>
>>>>
>>>>
>>>>
>>>> On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:
>>>>
>>>>> Unfortunately the cloud instance upgrade path requires some manual
>>>>> intervention here due to tool migrations.  SSH in to your instance,
>>>>> edit
>>>>> /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
>>>>> references to BlastXML.  Save the file, restart galaxy, and you
>>>>>should
>>>>> be
>>>>> good to go.
>>>>>
>>>>> -Dannon
>>>>>
>>>>> On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Running a CloudMan created instance on Amazon.
>>>>>>
>>>>>> Updated using cloudman from
>>>>>>http://bitbucket.org/galaxy/galaxy-central
>>>>>> to version 8167:b0c916349070
>>>>>>
>>>>>> Tried to run bowtie and get the following error. Searched the
>>>>>>message
>>>>>> archives an someone else reported the same problem but thought it
>>>>>>was
>>>>>> related to a new version of cufflinks. Same problem also exists when
>>>>>> running abyss. Given the nature of the error message not even sure
>>>>>> what
>>>>>> the actual path should be. Also for the abyss install from toolshed
>>>>>> the
>>>>>> toolshed.g2.bx.psu.edu directory doesn't exist or the six
>>>>>>directories
>>>>>> underneath. However Abyss was/is installed at the following location
>>>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls
>>>>>>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq
>>>>>>>DistanceEst
>>>>>>> MergePaths   PathConsensus  SimpleGraph
>>>>>>> abyss2ace  abyss-fac       ABYSS-P         AdjList        KAligner
>>>>>>> Overlap      PathOverlap
>>>>>>> abyss2afg  abyss-fixmate   abyss-pe        Consensus
>>>>>>> MergeContigs
>>>>>>> ParseAligns  PopBubbles
>>>>>>
>>>>>> Would be great if someone could give some guidance on how to resolve
>>>>>> the problems. I understand if I need to do something under the
>>>>>>covers
>>>>>> but the expectation is that the one click install/updates should not
>>>>>> then require major command line work that isn't documented.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Scooter
>>>>>>
>>>>>>
>>>>>> Bowtie
>>>>>>
>>>>>> Stdout
>>>>>>
>>>>>>> Sequence file aligned.
>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>> directory:
>>>>>>>
>>>>>>>
>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>sh
>>>>>>> ar
>>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>> directory:
>>>>>>>
>>>>>>>
>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>sh
>>>>>>> ar
>>>>>>> ed/ensembl/builds.txt'
>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>> directory:
>>>>>>>
>>>>>>>
>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>sh
>>>>>>> ar
>>>>>>> ed/ncbi/builds.txt'
>>>>>>
>>>>>> Stderr
>>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>>> datatype
>>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has
>>>>>>> no
>>>>>>> attribute 'BlastXml'
>>>>>>
>>>>>>
>>>>>>
>>>>>> Using toolshed installed Abyss and get the same error in Stdout.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Stdout
>>>>>>
>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>> directory:
>>>>>>>
>>>>>>>
>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>sh
>>>>>>> ar
>>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>> directory:
>>>>>>>
>>>>>>>
>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>sh
>>>>>>> ar
>>>>>>> ed/ensembl/builds.txt'
>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>> directory:
>>>>>>>
>>>>>>>
>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>sh
>>>>>>> ar
>>>>>>> ed/ncbi/builds.txt'
>>>>>>
>>>>>> Stderr
>>>>>>> COMMAND FAILURE:
>>>>>>>
>>>>>>>
>>>>>>>/mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abys
>>>>>>>s_
>>>>>>> to
>>>>>>>
>>>>>>>
>>>>>>>ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_
>>>>>>>wr
>>>>>>> ap
>>>>>>> per.sh: line 9: ABYSS: command not found
>>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>>> datatype
>>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has
>>>>>>> no
>>>>>>> attribute 'BlastXml'
>>>>>>
>>>>>> Looking at the util directory based on the expected path of the read
>>>>>> build file
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/u
>>>>>>>ti
>>>>>>> l$
>>>>>>> ls
>>>>>>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
>>>>>>> odict.py           streamball.py
>>>>>>> aliaspickler.pyc  docutils_template.txt  inflection.py
>>>>>>>lrucache.py
>>>>>>> odict.pyc          streamball.pyc
>>>>>>> backports         expressions.py         inflection.pyc
>>>>>>>lrucache.pyc
>>>>>>> sanitize_html.py   template.py
>>>>>>> bunch.py          expressions.pyc        __init__.py     memdump.py
>>>>>>> sanitize_html.pyc  template.pyc
>>>>>>> bunch.pyc         hash_util.py           __init__.pyc
>>>>>>>none_like.py
>>>>>>> shed_util.py       topsort.py
>>>>>>> debugging.py      hash_util.pyc          json.py
>>>>>>> none_like.pyc
>>>>>>> shed_util.pyc      topsort.pyc
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ___________________________________________________________
>>>>>> Please keep all replies on the list by using "reply all"
>>>>>> in your mail client.  To manage your subscriptions to this
>>>>>> and other Galaxy lists, please use the interface at:
>>>>>>
>>>>>> http://lists.bx.psu.edu/
>>>>>
>>>>
>>>
>>
>


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Dannon Baker
Glad to help.  The first step would be to check and see if those files actually exist (and they should).

If those messages are occurring on stdout and not stderr though, they shouldn't be causing failed jobs and something else might be wrong.

-Dannon


On Nov 8, 2012, at 1:27 PM, Scooter Willis <[hidden email]> wrote:

> Thanks! That was the explanation I was looking for and the link for how to
>
> Do you have any suggestions for fixing the error that are still occurring
> after running Bowtie?
>
>
> Sequence file aligned.
> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/
> ucsc/publicbuilds.txt'
> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/
> ensembl/builds.txt'
> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/shared/
> ncbi/builds.txt'
>
>
>
> On 11/8/12 12:44 PM, "Dannon Baker" <[hidden email]> wrote:
>
>> You do not need to restart or add/remove worker nodes, the master's tool
>> and data volumes are shared via NFS.
>>
>> The easiest way to make this work for you in the cloud is probably going
>> to be installing the version of the Abyss binaries you'd like to use
>> manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency
>> management
>> (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and
>> then, after it has been installed, modify the abyss wrapper to include a
>> <requirement type="package">abyss</requirement> tag, and finally persist
>> changes to galaxyTools (which will also restart Galaxy, reloading the
>> tools).
>>
>> -Dannon
>>
>>
>> On Nov 8, 2012, at 12:09 PM, Scooter Willis <[hidden email]> wrote:
>>
>>> Dannon
>>>
>>> If I install abyss to the galaxy tools volume and use cloud admin to do
>>> a
>>> snapshot I assume I then need to remove and then add compute nodes? From
>>> the EC2 management console I see two 15GiB EBS volumes from the same
>>> snapshot and I have a master and one compute node running. The other
>>> approach is that the compute nodes have a NFS mount back into the
>>> galaxy_tools volume. Trying to make sure I understand the need to
>>> restart
>>> the compute nodes after an install of third party tools.
>>>
>>> From looking at the abyss wrapper it is expecting ABYSS to be found on
>>> the
>>> path via a .sh file that gets called. Where would I update the path to a
>>> particular installed tool such that it will be picked up for each
>>> compute
>>> node that gets started? I couldn't find any examples of doing a third
>>> party install with discussion about how that install would persist to
>>> future compute nodes. The missing element for me at the moment is where
>>> to
>>> make path related changes that will be picked up by compute nodes. I
>>> could
>>> hard code the path in the .sh file used by the wrapper. Just trying to
>>> figure out how to use the software as designed.
>>>
>>> Thanks
>>>
>>> Scooter
>>>
>>>
>>> On 11/8/12 11:49 AM, "Dannon Baker" <[hidden email]> wrote:
>>>
>>>> Neither abyss wrapper in the toolshed installs binaries (you'd see a
>>>> tool_dependencies.xml in the repository), that's left up to the end
>>>> user.
>>>> You might contact the authors of the wrappers via the toolshed to see
>>>> if
>>>> they were going to add dependency management and had a newer version,
>>>> but
>>>> otherwise you'll have to install binaries on your own.  The cloud
>>>> volumes
>>>> do have a version of the Abyss binaries preinstalled, but it's
>>>> certainly
>>>> possible to add a newer version to your particular instance and persist
>>>> the changes to the tool volume (see the cloud admin panel).
>>>>
>>>>
>>>> On Nov 8, 2012, at 11:20 AM, Scooter Willis <[hidden email]>
>>>> wrote:
>>>>
>>>>> Ok that fixed Bowtie.
>>>>>
>>>>> Any thoughts on expected state of installing Abyss from toolshed. It
>>>>> indicates no requirements so my expectation is that installing from
>>>>> toolshed should install the appropriate abyss package in a location
>>>>> the
>>>>> tool is expecting to find it.
>>>>>
>>>>> In looking at the toolshed looks like two wrappers have been
>>>>> configured.
>>>>> One by msjeon and the other edward-kirton. I did the install for
>>>>> msjeon.
>>>>> Will try the other.
>>>>>
>>>>> Is the abyss install located at /mnt/galaxyTools/tools/abyss/1.2.5/bin
>>>>> from prior testing and should be deleted. Current release version of
>>>>> abyss
>>>>> as of May 30 2012 is Abyss 1.3.4
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:
>>>>>
>>>>>> Unfortunately the cloud instance upgrade path requires some manual
>>>>>> intervention here due to tool migrations.  SSH in to your instance,
>>>>>> edit
>>>>>> /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
>>>>>> references to BlastXML.  Save the file, restart galaxy, and you
>>>>>> should
>>>>>> be
>>>>>> good to go.
>>>>>>
>>>>>> -Dannon
>>>>>>
>>>>>> On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> Running a CloudMan created instance on Amazon.
>>>>>>>
>>>>>>> Updated using cloudman from
>>>>>>> http://bitbucket.org/galaxy/galaxy-central
>>>>>>> to version 8167:b0c916349070
>>>>>>>
>>>>>>> Tried to run bowtie and get the following error. Searched the
>>>>>>> message
>>>>>>> archives an someone else reported the same problem but thought it
>>>>>>> was
>>>>>>> related to a new version of cufflinks. Same problem also exists when
>>>>>>> running abyss. Given the nature of the error message not even sure
>>>>>>> what
>>>>>>> the actual path should be. Also for the abyss install from toolshed
>>>>>>> the
>>>>>>> toolshed.g2.bx.psu.edu directory doesn't exist or the six
>>>>>>> directories
>>>>>>> underneath. However Abyss was/is installed at the following location
>>>>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$ ls
>>>>>>>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq
>>>>>>>> DistanceEst
>>>>>>>> MergePaths   PathConsensus  SimpleGraph
>>>>>>>> abyss2ace  abyss-fac       ABYSS-P         AdjList        KAligner
>>>>>>>> Overlap      PathOverlap
>>>>>>>> abyss2afg  abyss-fixmate   abyss-pe        Consensus
>>>>>>>> MergeContigs
>>>>>>>> ParseAligns  PopBubbles
>>>>>>>
>>>>>>> Would be great if someone could give some guidance on how to resolve
>>>>>>> the problems. I understand if I need to do something under the
>>>>>>> covers
>>>>>>> but the expectation is that the one click install/updates should not
>>>>>>> then require major command line work that isn't documented.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> Scooter
>>>>>>>
>>>>>>>
>>>>>>> Bowtie
>>>>>>>
>>>>>>> Stdout
>>>>>>>
>>>>>>>> Sequence file aligned.
>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>> directory:
>>>>>>>>
>>>>>>>>
>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>> sh
>>>>>>>> ar
>>>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>> directory:
>>>>>>>>
>>>>>>>>
>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>> sh
>>>>>>>> ar
>>>>>>>> ed/ensembl/builds.txt'
>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>> directory:
>>>>>>>>
>>>>>>>>
>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>> sh
>>>>>>>> ar
>>>>>>>> ed/ncbi/builds.txt'
>>>>>>>
>>>>>>> Stderr
>>>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>>>> datatype
>>>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has
>>>>>>>> no
>>>>>>>> attribute 'BlastXml'
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Using toolshed installed Abyss and get the same error in Stdout.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Stdout
>>>>>>>
>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>> directory:
>>>>>>>>
>>>>>>>>
>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>> sh
>>>>>>>> ar
>>>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>> directory:
>>>>>>>>
>>>>>>>>
>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>> sh
>>>>>>>> ar
>>>>>>>> ed/ensembl/builds.txt'
>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>> directory:
>>>>>>>>
>>>>>>>>
>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/
>>>>>>>> sh
>>>>>>>> ar
>>>>>>>> ed/ncbi/builds.txt'
>>>>>>>
>>>>>>> Stderr
>>>>>>>> COMMAND FAILURE:
>>>>>>>>
>>>>>>>>
>>>>>>>> /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/abys
>>>>>>>> s_
>>>>>>>> to
>>>>>>>>
>>>>>>>>
>>>>>>>> ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abyss_
>>>>>>>> wr
>>>>>>>> ap
>>>>>>>> per.sh: line 9: ABYSS: command not found
>>>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>>>> datatype
>>>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object has
>>>>>>>> no
>>>>>>>> attribute 'BlastXml'
>>>>>>>
>>>>>>> Looking at the util directory based on the expected path of the read
>>>>>>> build file
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/u
>>>>>>>> ti
>>>>>>>> l$
>>>>>>>> ls
>>>>>>>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
>>>>>>>> odict.py           streamball.py
>>>>>>>> aliaspickler.pyc  docutils_template.txt  inflection.py
>>>>>>>> lrucache.py
>>>>>>>> odict.pyc          streamball.pyc
>>>>>>>> backports         expressions.py         inflection.pyc
>>>>>>>> lrucache.pyc
>>>>>>>> sanitize_html.py   template.py
>>>>>>>> bunch.py          expressions.pyc        __init__.py     memdump.py
>>>>>>>> sanitize_html.pyc  template.pyc
>>>>>>>> bunch.pyc         hash_util.py           __init__.pyc
>>>>>>>> none_like.py
>>>>>>>> shed_util.py       topsort.py
>>>>>>>> debugging.py      hash_util.pyc          json.py
>>>>>>>> none_like.pyc
>>>>>>>> shed_util.pyc      topsort.pyc
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ___________________________________________________________
>>>>>>> Please keep all replies on the list by using "reply all"
>>>>>>> in your mail client.  To manage your subscriptions to this
>>>>>>> and other Galaxy lists, please use the interface at:
>>>>>>>
>>>>>>> http://lists.bx.psu.edu/
>>>>>>
>>>>>
>>>>
>>>
>>
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Scooter Willis-2
Errors are being output to stdout

The files don't exist and complicated by the /../../ where I don't know
which subdirectory to follow. Attached screen shot of job failing giving
the ERROR message in the web gui. I also get the same errors when
attempting to run Abys which doesn't run because it isn't installed.

galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$
find -name publicbuilds.txt
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls
aliaspickler.py   debugging.py           hash_util.py    __init__.py
lrucache.pyc   odict.pyc          streamball.py   topsort.pyc
aliaspickler.pyc  docutils_ext           hash_util.pyc   __init__.pyc
memdump.py     sanitize_html.py   streamball.pyc
backports         docutils_template.txt  heartbeat.py    json.py
none_like.py   sanitize_html.pyc  template.py
bunch.py          expressions.py         inflection.py   json.pyc
none_like.pyc  shed_util.py       template.pyc
bunch.pyc         expressions.pyc        inflection.pyc  lrucache.py
odict.py       shed_util.pyc      topsort.py






On 11/8/12 1:34 PM, "Dannon Baker" <[hidden email]> wrote:

>Glad to help.  The first step would be to check and see if those files
>actually exist (and they should).
>
>If those messages are occurring on stdout and not stderr though, they
>shouldn't be causing failed jobs and something else might be wrong.
>
>-Dannon
>
>
>On Nov 8, 2012, at 1:27 PM, Scooter Willis <[hidden email]> wrote:
>
>> Thanks! That was the explanation I was looking for and the link for how
>>to
>>
>> Do you have any suggestions for fixing the error that are still
>>occurring
>> after running Bowtie?
>>
>>
>> Sequence file aligned.
>> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
>>
>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
>>d/
>> ucsc/publicbuilds.txt'
>> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
>>
>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
>>d/
>> ensembl/builds.txt'
>> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
>>
>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
>>d/
>> ncbi/builds.txt'
>>
>>
>>
>> On 11/8/12 12:44 PM, "Dannon Baker" <[hidden email]> wrote:
>>
>>> You do not need to restart or add/remove worker nodes, the master's
>>>tool
>>> and data volumes are shared via NFS.
>>>
>>> The easiest way to make this work for you in the cloud is probably
>>>going
>>> to be installing the version of the Abyss binaries you'd like to use
>>> manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency
>>> management
>>> (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and
>>> then, after it has been installed, modify the abyss wrapper to include
>>>a
>>> <requirement type="package">abyss</requirement> tag, and finally
>>>persist
>>> changes to galaxyTools (which will also restart Galaxy, reloading the
>>> tools).
>>>
>>> -Dannon
>>>
>>>
>>> On Nov 8, 2012, at 12:09 PM, Scooter Willis <[hidden email]>
>>>wrote:
>>>
>>>> Dannon
>>>>
>>>> If I install abyss to the galaxy tools volume and use cloud admin to
>>>>do
>>>> a
>>>> snapshot I assume I then need to remove and then add compute nodes?
>>>>From
>>>> the EC2 management console I see two 15GiB EBS volumes from the same
>>>> snapshot and I have a master and one compute node running. The other
>>>> approach is that the compute nodes have a NFS mount back into the
>>>> galaxy_tools volume. Trying to make sure I understand the need to
>>>> restart
>>>> the compute nodes after an install of third party tools.
>>>>
>>>> From looking at the abyss wrapper it is expecting ABYSS to be found on
>>>> the
>>>> path via a .sh file that gets called. Where would I update the path
>>>>to a
>>>> particular installed tool such that it will be picked up for each
>>>> compute
>>>> node that gets started? I couldn't find any examples of doing a third
>>>> party install with discussion about how that install would persist to
>>>> future compute nodes. The missing element for me at the moment is
>>>>where
>>>> to
>>>> make path related changes that will be picked up by compute nodes. I
>>>> could
>>>> hard code the path in the .sh file used by the wrapper. Just trying to
>>>> figure out how to use the software as designed.
>>>>
>>>> Thanks
>>>>
>>>> Scooter
>>>>
>>>>
>>>> On 11/8/12 11:49 AM, "Dannon Baker" <[hidden email]> wrote:
>>>>
>>>>> Neither abyss wrapper in the toolshed installs binaries (you'd see a
>>>>> tool_dependencies.xml in the repository), that's left up to the end
>>>>> user.
>>>>> You might contact the authors of the wrappers via the toolshed to see
>>>>> if
>>>>> they were going to add dependency management and had a newer version,
>>>>> but
>>>>> otherwise you'll have to install binaries on your own.  The cloud
>>>>> volumes
>>>>> do have a version of the Abyss binaries preinstalled, but it's
>>>>> certainly
>>>>> possible to add a newer version to your particular instance and
>>>>>persist
>>>>> the changes to the tool volume (see the cloud admin panel).
>>>>>
>>>>>
>>>>> On Nov 8, 2012, at 11:20 AM, Scooter Willis <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Ok that fixed Bowtie.
>>>>>>
>>>>>> Any thoughts on expected state of installing Abyss from toolshed. It
>>>>>> indicates no requirements so my expectation is that installing from
>>>>>> toolshed should install the appropriate abyss package in a location
>>>>>> the
>>>>>> tool is expecting to find it.
>>>>>>
>>>>>> In looking at the toolshed looks like two wrappers have been
>>>>>> configured.
>>>>>> One by msjeon and the other edward-kirton. I did the install for
>>>>>> msjeon.
>>>>>> Will try the other.
>>>>>>
>>>>>> Is the abyss install located at
>>>>>>/mnt/galaxyTools/tools/abyss/1.2.5/bin
>>>>>> from prior testing and should be deleted. Current release version of
>>>>>> abyss
>>>>>> as of May 30 2012 is Abyss 1.3.4
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:
>>>>>>
>>>>>>> Unfortunately the cloud instance upgrade path requires some manual
>>>>>>> intervention here due to tool migrations.  SSH in to your instance,
>>>>>>> edit
>>>>>>> /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
>>>>>>> references to BlastXML.  Save the file, restart galaxy, and you
>>>>>>> should
>>>>>>> be
>>>>>>> good to go.
>>>>>>>
>>>>>>> -Dannon
>>>>>>>
>>>>>>> On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Running a CloudMan created instance on Amazon.
>>>>>>>>
>>>>>>>> Updated using cloudman from
>>>>>>>> http://bitbucket.org/galaxy/galaxy-central
>>>>>>>> to version 8167:b0c916349070
>>>>>>>>
>>>>>>>> Tried to run bowtie and get the following error. Searched the
>>>>>>>> message
>>>>>>>> archives an someone else reported the same problem but thought it
>>>>>>>> was
>>>>>>>> related to a new version of cufflinks. Same problem also exists
>>>>>>>>when
>>>>>>>> running abyss. Given the nature of the error message not even sure
>>>>>>>> what
>>>>>>>> the actual path should be. Also for the abyss install from
>>>>>>>>toolshed
>>>>>>>> the
>>>>>>>> toolshed.g2.bx.psu.edu directory doesn't exist or the six
>>>>>>>> directories
>>>>>>>> underneath. However Abyss was/is installed at the following
>>>>>>>>location
>>>>>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$
>>>>>>>>>ls
>>>>>>>>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq
>>>>>>>>> DistanceEst
>>>>>>>>> MergePaths   PathConsensus  SimpleGraph
>>>>>>>>> abyss2ace  abyss-fac       ABYSS-P         AdjList
>>>>>>>>>KAligner
>>>>>>>>> Overlap      PathOverlap
>>>>>>>>> abyss2afg  abyss-fixmate   abyss-pe        Consensus
>>>>>>>>> MergeContigs
>>>>>>>>> ParseAligns  PopBubbles
>>>>>>>>
>>>>>>>> Would be great if someone could give some guidance on how to
>>>>>>>>resolve
>>>>>>>> the problems. I understand if I need to do something under the
>>>>>>>> covers
>>>>>>>> but the expectation is that the one click install/updates should
>>>>>>>>not
>>>>>>>> then require major command line work that isn't documented.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> Scooter
>>>>>>>>
>>>>>>>>
>>>>>>>> Bowtie
>>>>>>>>
>>>>>>>> Stdout
>>>>>>>>
>>>>>>>>> Sequence file aligned.
>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>> directory:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>a/
>>>>>>>>> sh
>>>>>>>>> ar
>>>>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>> directory:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>a/
>>>>>>>>> sh
>>>>>>>>> ar
>>>>>>>>> ed/ensembl/builds.txt'
>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>> directory:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>a/
>>>>>>>>> sh
>>>>>>>>> ar
>>>>>>>>> ed/ncbi/builds.txt'
>>>>>>>>
>>>>>>>> Stderr
>>>>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>>>>> datatype
>>>>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object
>>>>>>>>>has
>>>>>>>>> no
>>>>>>>>> attribute 'BlastXml'
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Using toolshed installed Abyss and get the same error in Stdout.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Stdout
>>>>>>>>
>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>> directory:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>a/
>>>>>>>>> sh
>>>>>>>>> ar
>>>>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>> directory:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>a/
>>>>>>>>> sh
>>>>>>>>> ar
>>>>>>>>> ed/ensembl/builds.txt'
>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>> directory:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>a/
>>>>>>>>> sh
>>>>>>>>> ar
>>>>>>>>> ed/ncbi/builds.txt'
>>>>>>>>
>>>>>>>> Stderr
>>>>>>>>> COMMAND FAILURE:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>/mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/ab
>>>>>>>>>ys
>>>>>>>>> s_
>>>>>>>>> to
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abys
>>>>>>>>>s_
>>>>>>>>> wr
>>>>>>>>> ap
>>>>>>>>> per.sh: line 9: ABYSS: command not found
>>>>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>>>>> datatype
>>>>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object
>>>>>>>>>has
>>>>>>>>> no
>>>>>>>>> attribute 'BlastXml'
>>>>>>>>
>>>>>>>> Looking at the util directory based on the expected path of the
>>>>>>>>read
>>>>>>>> build file
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy
>>>>>>>>>/u
>>>>>>>>> ti
>>>>>>>>> l$
>>>>>>>>> ls
>>>>>>>>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
>>>>>>>>> odict.py           streamball.py
>>>>>>>>> aliaspickler.pyc  docutils_template.txt  inflection.py
>>>>>>>>> lrucache.py
>>>>>>>>> odict.pyc          streamball.pyc
>>>>>>>>> backports         expressions.py         inflection.pyc
>>>>>>>>> lrucache.pyc
>>>>>>>>> sanitize_html.py   template.py
>>>>>>>>> bunch.py          expressions.pyc        __init__.py
>>>>>>>>>memdump.py
>>>>>>>>> sanitize_html.pyc  template.pyc
>>>>>>>>> bunch.pyc         hash_util.py           __init__.pyc
>>>>>>>>> none_like.py
>>>>>>>>> shed_util.py       topsort.py
>>>>>>>>> debugging.py      hash_util.pyc          json.py
>>>>>>>>> none_like.pyc
>>>>>>>>> shed_util.pyc      topsort.pyc
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ___________________________________________________________
>>>>>>>> Please keep all replies on the list by using "reply all"
>>>>>>>> in your mail client.  To manage your subscriptions to this
>>>>>>>> and other Galaxy lists, please use the interface at:
>>>>>>>>
>>>>>>>> http://lists.bx.psu.edu/
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

default.png (41K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Dannon Baker
That path should resolve to /mnt/galaxyTools/galaxy-central/tool-data/shared/ucsc/ (or ensembl, ncbi, etc)

Can you tell me the output of 'hg tip' 'hg st' and 'hg diff' from the root /mnt/galaxyTools/galaxy-central/ ?

-Dannon

On Nov 8, 2012, at 1:41 PM, Scooter Willis <[hidden email]> wrote:

> Errors are being output to stdout
>
> The files don't exist and complicated by the /../../ where I don't know
> which subdirectory to follow. Attached screen shot of job failing giving
> the ERROR message in the web gui. I also get the same errors when
> attempting to run Abys which doesn't run because it isn't installed.
>
> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$
> find -name publicbuilds.txt
> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls
> aliaspickler.py   debugging.py           hash_util.py    __init__.py
> lrucache.pyc   odict.pyc          streamball.py   topsort.pyc
> aliaspickler.pyc  docutils_ext           hash_util.pyc   __init__.pyc
> memdump.py     sanitize_html.py   streamball.pyc
> backports         docutils_template.txt  heartbeat.py    json.py
> none_like.py   sanitize_html.pyc  template.py
> bunch.py          expressions.py         inflection.py   json.pyc
> none_like.pyc  shed_util.py       template.pyc
> bunch.pyc         expressions.pyc        inflection.pyc  lrucache.py
> odict.py       shed_util.pyc      topsort.py
>
>
>
>
>
>
> On 11/8/12 1:34 PM, "Dannon Baker" <[hidden email]> wrote:
>
>> Glad to help.  The first step would be to check and see if those files
>> actually exist (and they should).
>>
>> If those messages are occurring on stdout and not stderr though, they
>> shouldn't be causing failed jobs and something else might be wrong.
>>
>> -Dannon
>>
>>
>> On Nov 8, 2012, at 1:27 PM, Scooter Willis <[hidden email]> wrote:
>>
>>> Thanks! That was the explanation I was looking for and the link for how
>>> to
>>>
>>> Do you have any suggestions for fixing the error that are still
>>> occurring
>>> after running Bowtie?
>>>
>>>
>>> Sequence file aligned.
>>> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
>>>
>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
>>> d/
>>> ucsc/publicbuilds.txt'
>>> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
>>>
>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
>>> d/
>>> ensembl/builds.txt'
>>> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
>>>
>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
>>> d/
>>> ncbi/builds.txt'
>>>
>>>
>>>
>>> On 11/8/12 12:44 PM, "Dannon Baker" <[hidden email]> wrote:
>>>
>>>> You do not need to restart or add/remove worker nodes, the master's
>>>> tool
>>>> and data volumes are shared via NFS.
>>>>
>>>> The easiest way to make this work for you in the cloud is probably
>>>> going
>>>> to be installing the version of the Abyss binaries you'd like to use
>>>> manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency
>>>> management
>>>> (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and
>>>> then, after it has been installed, modify the abyss wrapper to include
>>>> a
>>>> <requirement type="package">abyss</requirement> tag, and finally
>>>> persist
>>>> changes to galaxyTools (which will also restart Galaxy, reloading the
>>>> tools).
>>>>
>>>> -Dannon
>>>>
>>>>
>>>> On Nov 8, 2012, at 12:09 PM, Scooter Willis <[hidden email]>
>>>> wrote:
>>>>
>>>>> Dannon
>>>>>
>>>>> If I install abyss to the galaxy tools volume and use cloud admin to
>>>>> do
>>>>> a
>>>>> snapshot I assume I then need to remove and then add compute nodes?
>>>>> From
>>>>> the EC2 management console I see two 15GiB EBS volumes from the same
>>>>> snapshot and I have a master and one compute node running. The other
>>>>> approach is that the compute nodes have a NFS mount back into the
>>>>> galaxy_tools volume. Trying to make sure I understand the need to
>>>>> restart
>>>>> the compute nodes after an install of third party tools.
>>>>>
>>>>> From looking at the abyss wrapper it is expecting ABYSS to be found on
>>>>> the
>>>>> path via a .sh file that gets called. Where would I update the path
>>>>> to a
>>>>> particular installed tool such that it will be picked up for each
>>>>> compute
>>>>> node that gets started? I couldn't find any examples of doing a third
>>>>> party install with discussion about how that install would persist to
>>>>> future compute nodes. The missing element for me at the moment is
>>>>> where
>>>>> to
>>>>> make path related changes that will be picked up by compute nodes. I
>>>>> could
>>>>> hard code the path in the .sh file used by the wrapper. Just trying to
>>>>> figure out how to use the software as designed.
>>>>>
>>>>> Thanks
>>>>>
>>>>> Scooter
>>>>>
>>>>>
>>>>> On 11/8/12 11:49 AM, "Dannon Baker" <[hidden email]> wrote:
>>>>>
>>>>>> Neither abyss wrapper in the toolshed installs binaries (you'd see a
>>>>>> tool_dependencies.xml in the repository), that's left up to the end
>>>>>> user.
>>>>>> You might contact the authors of the wrappers via the toolshed to see
>>>>>> if
>>>>>> they were going to add dependency management and had a newer version,
>>>>>> but
>>>>>> otherwise you'll have to install binaries on your own.  The cloud
>>>>>> volumes
>>>>>> do have a version of the Abyss binaries preinstalled, but it's
>>>>>> certainly
>>>>>> possible to add a newer version to your particular instance and
>>>>>> persist
>>>>>> the changes to the tool volume (see the cloud admin panel).
>>>>>>
>>>>>>
>>>>>> On Nov 8, 2012, at 11:20 AM, Scooter Willis <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> Ok that fixed Bowtie.
>>>>>>>
>>>>>>> Any thoughts on expected state of installing Abyss from toolshed. It
>>>>>>> indicates no requirements so my expectation is that installing from
>>>>>>> toolshed should install the appropriate abyss package in a location
>>>>>>> the
>>>>>>> tool is expecting to find it.
>>>>>>>
>>>>>>> In looking at the toolshed looks like two wrappers have been
>>>>>>> configured.
>>>>>>> One by msjeon and the other edward-kirton. I did the install for
>>>>>>> msjeon.
>>>>>>> Will try the other.
>>>>>>>
>>>>>>> Is the abyss install located at
>>>>>>> /mnt/galaxyTools/tools/abyss/1.2.5/bin
>>>>>>> from prior testing and should be deleted. Current release version of
>>>>>>> abyss
>>>>>>> as of May 30 2012 is Abyss 1.3.4
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:
>>>>>>>
>>>>>>>> Unfortunately the cloud instance upgrade path requires some manual
>>>>>>>> intervention here due to tool migrations.  SSH in to your instance,
>>>>>>>> edit
>>>>>>>> /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
>>>>>>>> references to BlastXML.  Save the file, restart galaxy, and you
>>>>>>>> should
>>>>>>>> be
>>>>>>>> good to go.
>>>>>>>>
>>>>>>>> -Dannon
>>>>>>>>
>>>>>>>> On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Running a CloudMan created instance on Amazon.
>>>>>>>>>
>>>>>>>>> Updated using cloudman from
>>>>>>>>> http://bitbucket.org/galaxy/galaxy-central
>>>>>>>>> to version 8167:b0c916349070
>>>>>>>>>
>>>>>>>>> Tried to run bowtie and get the following error. Searched the
>>>>>>>>> message
>>>>>>>>> archives an someone else reported the same problem but thought it
>>>>>>>>> was
>>>>>>>>> related to a new version of cufflinks. Same problem also exists
>>>>>>>>> when
>>>>>>>>> running abyss. Given the nature of the error message not even sure
>>>>>>>>> what
>>>>>>>>> the actual path should be. Also for the abyss install from
>>>>>>>>> toolshed
>>>>>>>>> the
>>>>>>>>> toolshed.g2.bx.psu.edu directory doesn't exist or the six
>>>>>>>>> directories
>>>>>>>>> underneath. However Abyss was/is installed at the following
>>>>>>>>> location
>>>>>>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$
>>>>>>>>>> ls
>>>>>>>>>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq
>>>>>>>>>> DistanceEst
>>>>>>>>>> MergePaths   PathConsensus  SimpleGraph
>>>>>>>>>> abyss2ace  abyss-fac       ABYSS-P         AdjList
>>>>>>>>>> KAligner
>>>>>>>>>> Overlap      PathOverlap
>>>>>>>>>> abyss2afg  abyss-fixmate   abyss-pe        Consensus
>>>>>>>>>> MergeContigs
>>>>>>>>>> ParseAligns  PopBubbles
>>>>>>>>>
>>>>>>>>> Would be great if someone could give some guidance on how to
>>>>>>>>> resolve
>>>>>>>>> the problems. I understand if I need to do something under the
>>>>>>>>> covers
>>>>>>>>> but the expectation is that the one click install/updates should
>>>>>>>>> not
>>>>>>>>> then require major command line work that isn't documented.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> Scooter
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Bowtie
>>>>>>>>>
>>>>>>>>> Stdout
>>>>>>>>>
>>>>>>>>>> Sequence file aligned.
>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>> directory:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>> a/
>>>>>>>>>> sh
>>>>>>>>>> ar
>>>>>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>> directory:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>> a/
>>>>>>>>>> sh
>>>>>>>>>> ar
>>>>>>>>>> ed/ensembl/builds.txt'
>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>> directory:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>> a/
>>>>>>>>>> sh
>>>>>>>>>> ar
>>>>>>>>>> ed/ncbi/builds.txt'
>>>>>>>>>
>>>>>>>>> Stderr
>>>>>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>>>>>> datatype
>>>>>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object
>>>>>>>>>> has
>>>>>>>>>> no
>>>>>>>>>> attribute 'BlastXml'
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Using toolshed installed Abyss and get the same error in Stdout.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Stdout
>>>>>>>>>
>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>> directory:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>> a/
>>>>>>>>>> sh
>>>>>>>>>> ar
>>>>>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>> directory:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>> a/
>>>>>>>>>> sh
>>>>>>>>>> ar
>>>>>>>>>> ed/ensembl/builds.txt'
>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>> directory:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>> a/
>>>>>>>>>> sh
>>>>>>>>>> ar
>>>>>>>>>> ed/ncbi/builds.txt'
>>>>>>>>>
>>>>>>>>> Stderr
>>>>>>>>>> COMMAND FAILURE:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/ab
>>>>>>>>>> ys
>>>>>>>>>> s_
>>>>>>>>>> to
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abys
>>>>>>>>>> s_
>>>>>>>>>> wr
>>>>>>>>>> ap
>>>>>>>>>> per.sh: line 9: ABYSS: command not found
>>>>>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>>>>>> datatype
>>>>>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object
>>>>>>>>>> has
>>>>>>>>>> no
>>>>>>>>>> attribute 'BlastXml'
>>>>>>>>>
>>>>>>>>> Looking at the util directory based on the expected path of the
>>>>>>>>> read
>>>>>>>>> build file
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy
>>>>>>>>>> /u
>>>>>>>>>> ti
>>>>>>>>>> l$
>>>>>>>>>> ls
>>>>>>>>>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
>>>>>>>>>> odict.py           streamball.py
>>>>>>>>>> aliaspickler.pyc  docutils_template.txt  inflection.py
>>>>>>>>>> lrucache.py
>>>>>>>>>> odict.pyc          streamball.pyc
>>>>>>>>>> backports         expressions.py         inflection.pyc
>>>>>>>>>> lrucache.pyc
>>>>>>>>>> sanitize_html.py   template.py
>>>>>>>>>> bunch.py          expressions.pyc        __init__.py
>>>>>>>>>> memdump.py
>>>>>>>>>> sanitize_html.pyc  template.pyc
>>>>>>>>>> bunch.pyc         hash_util.py           __init__.pyc
>>>>>>>>>> none_like.py
>>>>>>>>>> shed_util.py       topsort.py
>>>>>>>>>> debugging.py      hash_util.pyc          json.py
>>>>>>>>>> none_like.pyc
>>>>>>>>>> shed_util.pyc      topsort.pyc
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ___________________________________________________________
>>>>>>>>> Please keep all replies on the list by using "reply all"
>>>>>>>>> in your mail client.  To manage your subscriptions to this
>>>>>>>>> and other Galaxy lists, please use the interface at:
>>>>>>>>>
>>>>>>>>> http://lists.bx.psu.edu/
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
> <default.png>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Scooter Willis-2
Ok that directory exists but public builds.txt has a .sample at the end.

ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/tool-data/shared/ucsc$ ls
builds.txt builds.txt.sample publicbuilds.txt.sample ucsc_build_sites.txt.new
builds.txt.buildbot manual_builds.txt ucsc_build_sites.txt



ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg tip
Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
changeset: 8167:b0c916349070
tag: tip
user: clements
date: Wed Oct 31 23:08:09 2012 -0700
summary: Update docstrings so they no longer generate warnings in Sphix.

ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg st
Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
M run.sh
M tool-data/shared/ucsc/ucsc_build_sites.txt
? paster.log
? paster.log.16_06
? paster.log.16_46
? paster.log.17_29
? paster.log.21_01
? paster.log.22_24
? paster.log.22_38
? paster.log.22_42
? paster.pid
? static/images/cloud.gif
? static/images/cloud_text.png
? tool-data/HYPHY
? tool-data/shared/ucsc/ucsc_build_sites.txt.new
? tool_data_table_conf.xml.cloud
? tool_data_table_conf.xml_DEF
ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg diff
Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
diff -r b0c916349070 run.sh
--- a/run.sh Wed Oct 31 23:08:09 2012 -0700
+++ b/run.sh Thu Nov 08 19:01:34 2012 +0000
@@ -1,8 +1,9 @@
#!/bin/sh
+export DRMAA_LIBRARY_PATH=/opt/sge/lib/lx24-amd64/libdrmaa.so.1.0
cd `dirname $0`
-python ./scripts/check_python.py
+python -ES ./scripts/check_python.py
[ $? -ne 0 ] && exit 1
SAMPLES="
@@ -17,10 +18,7 @@
tool_sheds_conf.xml.sample
openid_conf.xml.sample
universe_wsgi.ini.sample
- tool-data/shared/ncbi/builds.txt.sample
- tool-data/shared/ensembl/builds.txt.sample
tool-data/shared/ucsc/builds.txt.sample
- tool-data/shared/ucsc/publicbuilds.txt.sample
tool-data/shared/igv/igv_build_sites.txt.sample
tool-data/shared/rviewer/rviewer_build_sites.txt.sample
tool-data/*.sample
@@ -36,20 +34,16 @@
fi
done
-if [ -n "$GALAXY_UNIVERSE_CONFIG_DIR" ]; then
- python ./scripts/build_universe_config.py "$GALAXY_UNIVERSE_CONFIG_DIR"
-fi
-
# explicitly attempt to fetch eggs before running
FETCH_EGGS=1
for arg in "$@"; do
[ "$arg" = "--stop-daemon" ] && FETCH_EGGS=0; break
done
if [ $FETCH_EGGS -eq 1 ]; then
- python ./scripts/check_eggs.py -q
+ python -ES ./scripts/check_eggs.py -q
if [ $? -ne 0 ]; then
echo "Some eggs are out of date, attempting to fetch..."
- python ./scripts/fetch_eggs.py
+ python -ES ./scripts/fetch_eggs.py
if [ $? -eq 0 ]; then
echo "Fetch successful."
else
@@ -68,8 +62,8 @@
fi
for server in $servers; do
echo "Handling $server with log file $server.log..."
- python ./scripts/paster.py serve universe_wsgi.ini --server-name=$server --pid-file=$server.pid --log-file=$server.log $@
+ python -ES ./scripts/paster.py serve universe_wsgi.ini --server-name=$server --pid-file=$server.pid --log-file=$server.log $@
done
else
- python ./scripts/paster.py serve universe_wsgi.ini $@
+ python -ES ./scripts/paster.py serve universe_wsgi.ini $@
fi
diff -r b0c916349070 tool-data/shared/ucsc/ucsc_build_sites.txt
--- a/tool-data/shared/ucsc/ucsc_build_sites.txt Wed Oct 31 23:08:09 2012 -0700
+++ b/tool-data/shared/ucsc/ucsc_build_sites.txt Thu Nov 08 19:01:34 2012 +0000
@@ -1,5 +1,5 @@
#Harvested from http://genome.ucsc.edu/cgi-bin/das/dsn
-main http://genome.ucsc.edu/cgi-bin/hgTracks? priPac1,danRer4,mm9,mm8,droAna1,mm5,caeRem2,mm7,mm6,panTro1,dm3,panTro2,anoCar1,ce4,galGal3,galGal2,ce1,rn3,rn2,droMoj1,droMoj2,rn4,droYak1,droYak2,dp3,dp2,dm1,canFam1,danRer5,canFam2,danRer3,danRer2,ornAna1,ci2,ci1,tetNig1,bosTau1,bosTau3,bosTau2,equCab1,oryLat1,droAna2,droEre1,ponAbe2,rheMac2,sacCer1,droPer1,droSim1,monDom1,cb1,dm2,droSec1,strPur1,droVir2,droVir1,strPur2,sc1,xenTro1,droGri1,xenTro2,cb3,gasAcu1,caePb1,anoGam1,fr2,fr1,hg15,hg16,hg17,hg18,felCat3,apiMel2,monDom4,apiMel1,ce2
+main http://genome.ucsc.edu/cgi-bin/hgTracks? priPac1,danRer4,mm9,mm8,droAna1,mm5,caeRem2,mm7,mm6,panTro1,dm3,panTro2,anoCar1,ce4,galGal3,galGal2,ce1,rn3,rn2,droMoj1,droMoj2,rn4,droYak1,droYak2,dp3,dp2,dm1,canFam1,danRer5,canFam2,danRer3,danRer2,ornAna1,ci2,ci1,tetNig1,bosTau1,bosTau3,bosTau2,equCab1,oryLat1,droAna2,droEre1,ponAbe2,rheMac2,sacCer1,droPer1,droSim1,monDom1,cb1,dm2,droSec1,strPur1,droVir2,droVir1,strPur2,sc1,xenTro1,droGri1,xenTro2,cb3,gasAcu1,caePb1,anoGam1,fr2,fr1,hg15,hg16,hg17,hg18,felCat3,apiMel2,monDom4,apiMel1,ce2,hg19
#Harvested from http://archaea.ucsc.edu/cgi-bin/das/dsn
archaea http://archaea.ucsc.edu/cgi-bin/hgTracks? alkaEhrl_MLHE_1,shewW318,idioLoih_L2TR,sulSol1,erwiCaro_ATROSEPTICA,symbTher_IAM14863,moorTher_ATCC39073,therFusc_YX,methHung1,bradJapo,therElon,shewPutrCN32,pediPent_ATCC25745,mariMari_MCS10,nanEqu1,baciSubt,chlaTrac,magnMagn_AMB_1,chroViol,ralsSola,acidCryp_JF_5,erytLito_HTCC2594,desuVulg_HILDENBOROUG,pyrAer1,sulfToko1,shewANA3,paraSp_UWE25,geobKaus_HTA426,rhizEtli_CFN_42,uncuMeth_RCI,candBloc_FLORIDANUS,deinRadi,yersPest_CO92,saccEryt_NRRL_2338,rhodRHA1,candCars_RUDDII,burkMall_ATCC23344,eschColi_O157H7,burk383,psycIngr_37,rhodSpha_2_4_1,wolbEndo_OF_DROSOPHIL,burkViet_G4,propAcne_KPA171202,enteFaec_V583,campJeju_81_176,acidJS42,heliPylo_26695,pseuHalo_TAC125,chroSale_DSM3043,methVann1,archFulg1,neisMeni_Z2491_1,fusoNucl,vermEise_EF01_2,anabVari_ATCC29413,tropWhip_TW08_27,heliHepa,acinSp_ADP1,anapMarg_ST_MARIES,natrPhar1,haheChej_KCTC_2396,therPetr_RKU_1,neisGono_FA1090_1,colwPsyc_34H,desuPsyc_LSV54,hyphNept_ATCC15444,vibrChol1,deinGeot_DSM11300,strePyog_M1_GAS,franCcI3,salmTyph,metaSedu,lactSali_UCC118,trepPall,neisMeni_MC58_1,syntWolf_GOETTINGEN,flavJohn_UW101,methBoon1,haemSomn_129PT,shewLoihPV4,igniHosp1,haemInfl_KW20,haloHalo_SL1,ferrAcid1,sphiAlas_RB2256,candPela_UBIQUE_HTCC1,caldSacc_DSM8903,aerPer1,lactPlan,carbHydr_Z_2901,therTher_HB8,vibrVuln_YJ016_1,rhodPalu_CGA009,acidCell_11B,siliPome_DSS_3,therVolc1,haloWals1,rubrXyla_DSM9941,shewAmaz,nocaJS61,vibrVuln_CMCP6_1,sinoMeli,ureaUrea,baciHalo,bartHens_HOUSTON_1,nitrWino_NB_255,hypeButy1,methBurt2,polaJS66,mesoLoti,methMari_C7,caulCres,neisMeni_FAM18_1,acidBact_ELLIN345,caldMaqu1,salmEnte_PARATYPI_ATC,glucOxyd_621H,cytoHutc_ATCC33406,nitrEuro,therMari,coxiBurn,woliSucc,heliPylo_HPAG1,mesoFlor_L1,pyrHor1,methAeol1,procMari_CCMP1375,pyroArse1,oenoOeni_PSU_1,alcaBork_SK2,wiggBrev,actiPleu_L20,lactLact,methJann1,paraDeni_PD1222,borrBurg,pyroIsla1,orieTsut_BORYONG,shewMR4,methKand1,methCaps_BATH,onioYell_PHYTOPLASMA,bordBron,cenaSymb1,burkCeno_HI2424,franTula_TULARENSIS,pyrFur2,mariAqua_VT8,heliPylo_J99,psycArct_273_4,vibrChol_MO10_1,vibrPara1,rickBell_RML369_C,metAce1,buchSp,ehrlRumi_WELGEVONDEN,methLabrZ_1,chlaPneu_CWL029,thioCrun_XCL_2,pyroCali1,chloTepi_TLS,stapAure_MU50,novoArom_DSM12444,magnMC1,zymoMobi_ZM4,salmTyph_TY2,chloChlo_CAD3,azoaSp_EBN1,therTher_HB27,bifiLong,picrTorr1,listInno,bdelBact,gramFors_KT0803,sulfAcid1,geobTher_NG80_2,peloCarb,ralsEutr_JMP134,mannSucc_MBEL55E,syneSp_WH8102,methTherPT1,clavMich_NCPPB_382,therAcid1,syntAcid_SB,porpGing_W83,therNeut0,leifXyli_XYLI_CTCB0,shewFrig,photProf_SS9,thioDeni_ATCC25259,methMaze1,desuRedu_MI_1,burkThai_E264,campFetu_82_40,blocFlor,jannCCS1,nitrMult_ATCC25196,streCoel,soliUsit_ELLIN6076,pastMult,saliRube_DSM13855,methTher1,nostSp,shigFlex_2A,saccDegr_2_40,oceaIhey,dehaEthe_195,rhodRubr_ATCC11170,arthFB24,shewMR7,pireSp,anaeDeha_2CP_C,haloVolc1,dichNodo_VCS1703A,tricEryt_IMS101,mycoGeni,thioDeni_ATCC33889,methSmit1,geobUran_RF4,shewDeni,halMar1,desuHafn_Y51,methStad1,granBeth_CGDNIH1,therPend1,legiPneu_PHILADELPHIA,vibrChol_O395_1,nitrOcea_ATCC19707,campJeju_RM1221,methPetr_PM1,heliAcin_SHEEBA,eschColi_APEC_O1,peloTher_SI,haloHalo1,syntFuma_MPOB,xyleFast,gloeViol,leucMese_ATCC8293,bactThet_VPI_5482,xantCamp,sodaGlos_MORSITANS,geobSulf,roseDeni_OCH_114,coryEffi_YS_314,brucMeli,mycoTube_H37RV,vibrFisc_ES114_1,pyrAby1,burkXeno_LB400,polyQLWP,stapMari1,peloLute_DSM273,burkCeno_AU_1054,shewBalt,nocaFarc_IFM10152,ente638,mculMari1,saliTrop_CNB_440,neorSenn_MIYAYAMA,aquiAeol,dechArom_RCB,myxoXant_DK_1622,burkPseu_1106A,burkCepa_AMMD,methMari_C5_1,azorCaul2,methFlag_KT,leptInte,eschColi_K12,synePCC6,baumCica_HOMALODISCA,methBark1,pseuAeru,geobMeta_GS15,eschColi_CFT073,photLumi,metMar1,hermArse,campJeju,therKoda1,aeroHydr_ATCC7966,baciAnth_AMES,shewOnei,therTeng,lawsIntr_PHE_MN1_00
#Harvested from http://genome-test.cse.ucsc.edu/cgi-bin/das/dsn




On 11/8/12 1:55 PM, "Dannon Baker" <[hidden email]> wrote:

That path should resolve to /mnt/galaxyTools/galaxy-central/tool-data/shared/ucsc/ (or ensembl, ncbi, etc)

Can you tell me the output of 'hg tip' 'hg st' and 'hg diff' from the root /mnt/galaxyTools/galaxy-central/ ?

-Dannon

On Nov 8, 2012, at 1:41 PM, Scooter Willis <[hidden email]> wrote:

Errors are being output to stdout
The files don't exist and complicated by the /../../ where I don't know
which subdirectory to follow. Attached screen shot of job failing giving
the ERROR message in the web gui. I also get the same errors when
attempting to run Abys which doesn't run because it isn't installed.
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$
find -name publicbuilds.txt
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls
aliaspickler.py   debugging.py           hash_util.py    __init__.py
lrucache.pyc   odict.pyc          streamball.py   topsort.pyc
aliaspickler.pyc  docutils_ext           hash_util.pyc   __init__.pyc
memdump.py     sanitize_html.py   streamball.pyc
backports         docutils_template.txt  heartbeat.py    json.py
none_like.py   sanitize_html.pyc  template.py
bunch.py          expressions.py         inflection.py   json.pyc
none_like.pyc  shed_util.py       template.pyc
bunch.pyc         expressions.pyc        inflection.pyc  lrucache.py
odict.py       shed_util.pyc      topsort.py
On 11/8/12 1:34 PM, "Dannon Baker" <[hidden email]> wrote:
Glad to help.  The first step would be to check and see if those files
actually exist (and they should).
If those messages are occurring on stdout and not stderr though, they
shouldn't be causing failed jobs and something else might be wrong.
-Dannon
On Nov 8, 2012, at 1:27 PM, Scooter Willis <[hidden email]> wrote:
Thanks! That was the explanation I was looking for and the link for how
to
Do you have any suggestions for fixing the error that are still
occurring
after running Bowtie?
Sequence file aligned.
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
d/
ucsc/publicbuilds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
d/
ensembl/builds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
d/
ncbi/builds.txt'
On 11/8/12 12:44 PM, "Dannon Baker" <[hidden email]> wrote:
You do not need to restart or add/remove worker nodes, the master's
tool
and data volumes are shared via NFS.
The easiest way to make this work for you in the cloud is probably
going
to be installing the version of the Abyss binaries you'd like to use
manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency
management
then, after it has been installed, modify the abyss wrapper to include
a
<requirement type="package">abyss</requirement> tag, and finally
persist
changes to galaxyTools (which will also restart Galaxy, reloading the
tools).
-Dannon
On Nov 8, 2012, at 12:09 PM, Scooter Willis <[hidden email]>
wrote:
Dannon
If I install abyss to the galaxy tools volume and use cloud admin to
do
a
snapshot I assume I then need to remove and then add compute nodes?
From
the EC2 management console I see two 15GiB EBS volumes from the same
snapshot and I have a master and one compute node running. The other
approach is that the compute nodes have a NFS mount back into the
galaxy_tools volume. Trying to make sure I understand the need to
restart
the compute nodes after an install of third party tools.
From looking at the abyss wrapper it is expecting ABYSS to be found on
the
path via a .sh file that gets called. Where would I update the path
to a
particular installed tool such that it will be picked up for each
compute
node that gets started? I couldn't find any examples of doing a third
party install with discussion about how that install would persist to
future compute nodes. The missing element for me at the moment is
where
to
make path related changes that will be picked up by compute nodes. I
could
hard code the path in the .sh file used by the wrapper. Just trying to
figure out how to use the software as designed.
Thanks
Scooter
On 11/8/12 11:49 AM, "Dannon Baker" <[hidden email]> wrote:
Neither abyss wrapper in the toolshed installs binaries (you'd see a
tool_dependencies.xml in the repository), that's left up to the end
user.
You might contact the authors of the wrappers via the toolshed to see
if
they were going to add dependency management and had a newer version,
but
otherwise you'll have to install binaries on your own.  The cloud
volumes
do have a version of the Abyss binaries preinstalled, but it's
certainly
possible to add a newer version to your particular instance and
persist
the changes to the tool volume (see the cloud admin panel).
On Nov 8, 2012, at 11:20 AM, Scooter Willis <[hidden email]>
wrote:
Ok that fixed Bowtie.
Any thoughts on expected state of installing Abyss from toolshed. It
indicates no requirements so my expectation is that installing from
toolshed should install the appropriate abyss package in a location
the
tool is expecting to find it.
In looking at the toolshed looks like two wrappers have been
configured.
One by msjeon and the other edward-kirton. I did the install for
msjeon.
Will try the other.
Is the abyss install located at
/mnt/galaxyTools/tools/abyss/1.2.5/bin
from prior testing and should be deleted. Current release version of
abyss
as of May 30 2012 is Abyss 1.3.4
On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:
Unfortunately the cloud instance upgrade path requires some manual
intervention here due to tool migrations.  SSH in to your instance,
edit
/mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
references to BlastXML.  Save the file, restart galaxy, and you
should
be
good to go.
-Dannon
On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]>
wrote:
Running a CloudMan created instance on Amazon.
Updated using cloudman from
to version 8167:b0c916349070
Tried to run bowtie and get the following error. Searched the
message
archives an someone else reported the same problem but thought it
was
related to a new version of cufflinks. Same problem also exists
when
running abyss. Given the nature of the error message not even sure
what
the actual path should be. Also for the abyss install from
toolshed
the
toolshed.g2.bx.psu.edu directory doesn't exist or the six
directories
underneath. However Abyss was/is installed at the following
location
galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$
ls
ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq
DistanceEst
MergePaths   PathConsensus  SimpleGraph
abyss2ace  abyss-fac       ABYSS-P         AdjList
KAligner
Overlap      PathOverlap
abyss2afg  abyss-fixmate   abyss-pe        Consensus
MergeContigs
ParseAligns  PopBubbles
Would be great if someone could give some guidance on how to
resolve
the problems. I understand if I need to do something under the
covers
but the expectation is that the one click install/updates should
not
then require major command line work that isn't documented.
Thanks
Scooter
Bowtie
Stdout
Sequence file aligned.
ERROR: Unable to read builds file: [Errno 2] No such file or
directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
a/
sh
ar
ed/ucsc/publicbuilds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or
directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
a/
sh
ar
ed/ensembl/builds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or
directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
a/
sh
ar
ed/ncbi/builds.txt'
Stderr
WARNING:galaxy.datatypes.registry:Error loading datatype with
extension 'blastxml': 'module' object has no attribute 'BlastXml'
WARNING:galaxy.datatypes.registry:Error appending sniffer for
datatype
'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object
has
no
attribute 'BlastXml'
Using toolshed installed Abyss and get the same error in Stdout.
Stdout
ERROR: Unable to read builds file: [Errno 2] No such file or
directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
a/
sh
ar
ed/ucsc/publicbuilds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or
directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
a/
sh
ar
ed/ensembl/builds.txt'
ERROR: Unable to read builds file: [Errno 2] No such file or
directory:
'/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
a/
sh
ar
ed/ncbi/builds.txt'
Stderr
COMMAND FAILURE:
/mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/ab
ys
s_
to
ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abys
s_
wr
ap
per.sh: line 9: ABYSS: command not found
WARNING:galaxy.datatypes.registry:Error loading datatype with
extension 'blastxml': 'module' object has no attribute 'BlastXml'
WARNING:galaxy.datatypes.registry:Error appending sniffer for
datatype
'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object
has
no
attribute 'BlastXml'
Looking at the util directory based on the expected path of the
read
build file
galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy
/u
ti
l$
ls
aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
odict.py           streamball.py
aliaspickler.pyc  docutils_template.txt  inflection.py
lrucache.py
odict.pyc          streamball.pyc
backports         expressions.py         inflection.pyc
lrucache.pyc
sanitize_html.py   template.py
bunch.py          expressions.pyc        __init__.py
memdump.py
sanitize_html.pyc  template.pyc
bunch.pyc         hash_util.py           __init__.pyc
none_like.py
shed_util.py       topsort.py
debugging.py      hash_util.pyc          json.py
none_like.pyc
shed_util.pyc      topsort.pyc
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
<default.png>



___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/
Reply | Threaded
Open this post in threaded view
|

Re: Problems with latest Galaxy 8167:b0c916349070

Dannon Baker
Ah ha!  This is the problem -- your run.sh is incorrect due to what I'm assuming was a failed merge.  Here's a correct copy, note the inclusion of the extra .sample files.

Modify or replace your run.sh with this (which is from the most recent cloud images), restart galaxy, and you're good to go.





On Nov 8, 2012, at 2:03 PM, Scooter Willis <[hidden email]> wrote:

> Ok that directory exists but public builds.txt has a .sample at the end.
>
> ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/tool-data/shared/ucsc$ ls
> builds.txt builds.txt.sample publicbuilds.txt.sample ucsc_build_sites.txt.new
> builds.txt.buildbot manual_builds.txt ucsc_build_sites.txt
>
>
>
> ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg tip
> Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
> Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
> changeset: 8167:b0c916349070
> tag: tip
> user: clements
> date: Wed Oct 31 23:08:09 2012 -0700
> summary: Update docstrings so they no longer generate warnings in Sphix.
>
> ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg st
> Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
> Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
> M run.sh
> M tool-data/shared/ucsc/ucsc_build_sites.txt
> ? paster.log
> ? paster.log.16_06
> ? paster.log.16_46
> ? paster.log.17_29
> ? paster.log.21_01
> ? paster.log.22_24
> ? paster.log.22_38
> ? paster.log.22_42
> ? paster.pid
> ? static/images/cloud.gif
> ? static/images/cloud_text.png
> ? tool-data/HYPHY
> ? tool-data/shared/ucsc/ucsc_build_sites.txt.new
> ? tool_data_table_conf.xml.cloud
> ? tool_data_table_conf.xml_DEF
> ubuntu@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central$ hg diff
> Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
> Not trusting file /mnt/galaxyTools/galaxy-central/.hg/hgrc from untrusted user galaxy, group galaxy
> diff -r b0c916349070 run.sh
> --- a/run.sh Wed Oct 31 23:08:09 2012 -0700
> +++ b/run.sh Thu Nov 08 19:01:34 2012 +0000
> @@ -1,8 +1,9 @@
> #!/bin/sh
> +export DRMAA_LIBRARY_PATH=/opt/sge/lib/lx24-amd64/libdrmaa.so.1.0
> cd `dirname $0`
> -python ./scripts/check_python.py
> +python -ES ./scripts/check_python.py
> [ $? -ne 0 ] && exit 1
> SAMPLES="
> @@ -17,10 +18,7 @@
> tool_sheds_conf.xml.sample
> openid_conf.xml.sample
> universe_wsgi.ini.sample
> - tool-data/shared/ncbi/builds.txt.sample
> - tool-data/shared/ensembl/builds.txt.sample
> tool-data/shared/ucsc/builds.txt.sample
> - tool-data/shared/ucsc/publicbuilds.txt.sample
> tool-data/shared/igv/igv_build_sites.txt.sample
> tool-data/shared/rviewer/rviewer_build_sites.txt.sample
> tool-data/*.sample
> @@ -36,20 +34,16 @@
> fi
> done
> -if [ -n "$GALAXY_UNIVERSE_CONFIG_DIR" ]; then
> - python ./scripts/build_universe_config.py "$GALAXY_UNIVERSE_CONFIG_DIR"
> -fi
> -
> # explicitly attempt to fetch eggs before running
> FETCH_EGGS=1
> for arg in "$@"; do
> [ "$arg" = "--stop-daemon" ] && FETCH_EGGS=0; break
> done
> if [ $FETCH_EGGS -eq 1 ]; then
> - python ./scripts/check_eggs.py -q
> + python -ES ./scripts/check_eggs.py -q
> if [ $? -ne 0 ]; then
> echo "Some eggs are out of date, attempting to fetch..."
> - python ./scripts/fetch_eggs.py
> + python -ES ./scripts/fetch_eggs.py
> if [ $? -eq 0 ]; then
> echo "Fetch successful."
> else
> @@ -68,8 +62,8 @@
> fi
> for server in $servers; do
> echo "Handling $server with log file $server.log..."
> - python ./scripts/paster.py serve universe_wsgi.ini --server-name=$server --pid-file=$server.pid --log-file=$server.log $@
> + python -ES ./scripts/paster.py serve universe_wsgi.ini --server-name=$server --pid-file=$server.pid --log-file=$server.log $@
> done
> else
> - python ./scripts/paster.py serve universe_wsgi.ini $@
> + python -ES ./scripts/paster.py serve universe_wsgi.ini $@
> fi
> diff -r b0c916349070 tool-data/shared/ucsc/ucsc_build_sites.txt
> --- a/tool-data/shared/ucsc/ucsc_build_sites.txt Wed Oct 31 23:08:09 2012 -0700
> +++ b/tool-data/shared/ucsc/ucsc_build_sites.txt Thu Nov 08 19:01:34 2012 +0000
> @@ -1,5 +1,5 @@
> #Harvested from http://genome.ucsc.edu/cgi-bin/das/dsn
> -main http://genome.ucsc.edu/cgi-bin/hgTracks? priPac1,danRer4,mm9,mm8,droAna1,mm5,caeRem2,mm7,mm6,panTro1,dm3,panTro2,anoCar1,ce4,galGal3,galGal2,ce1,rn3,rn2,droMoj1,droMoj2,rn4,droYak1,droYak2,dp3,dp2,dm1,canFam1,danRer5,canFam2,danRer3,danRer2,ornAna1,ci2,ci1,tetNig1,bosTau1,bosTau3,bosTau2,equCab1,oryLat1,droAna2,droEre1,ponAbe2,rheMac2,sacCer1,droPer1,droSim1,monDom1,cb1,dm2,droSec1,strPur1,droVir2,droVir1,strPur2,sc1,xenTro1,droGri1,xenTro2,cb3,gasAcu1,caePb1,anoGam1,fr2,fr1,hg15,hg16,hg17,hg18,felCat3,apiMel2,monDom4,apiMel1,ce2
> +main http://genome.ucsc.edu/cgi-bin/hgTracks? priPac1,danRer4,mm9,mm8,droAna1,mm5,caeRem2,mm7,mm6,panTro1,dm3,panTro2,anoCar1,ce4,galGal3,galGal2,ce1,rn3,rn2,droMoj1,droMoj2,rn4,droYak1,droYak2,dp3,dp2,dm1,canFam1,danRer5,canFam2,danRer3,danRer2,ornAna1,ci2,ci1,tetNig1,bosTau1,bosTau3,bosTau2,equCab1,oryLat1,droAna2,droEre1,ponAbe2,rheMac2,sacCer1,droPer1,droSim1,monDom1,cb1,dm2,droSec1,strPur1,droVir2,droVir1,strPur2,sc1,xenTro1,droGri1,xenTro2,cb3,gasAcu1,caePb1,anoGam1,fr2,fr1,hg15,hg16,hg17,hg18,felCat3,apiMel2,monDom4,apiMel1,ce2,hg19
> #Harvested from http://archaea.ucsc.edu/cgi-bin/das/dsn
> archaea http://archaea.ucsc.edu/cgi-bin/hgTracks? alkaEhrl_MLHE_1,shewW318,idioLoih_L2TR,sulSol1,erwiCaro_ATROSEPTICA,symbTher_IAM14863,moorTher_ATCC39073,therFusc_YX,methHung1,bradJapo,therElon,shewPutrCN32,pediPent_ATCC25745,mariMari_MCS10,nanEqu1,baciSubt,chlaTrac,magnMagn_AMB_1,chroViol,ralsSola,acidCryp_JF_5,erytLito_HTCC2594,desuVulg_HILDENBOROUG,pyrAer1,sulfToko1,shewANA3,paraSp_UWE25,geobKaus_HTA426,rhizEtli_CFN_42,uncuMeth_RCI,candBloc_FLORIDANUS,deinRadi,yersPest_CO92,saccEryt_NRRL_2338,rhodRHA1,candCars_RUDDII,burkMall_ATCC23344,eschColi_O157H7,burk383,psycIngr_37,rhodSpha_2_4_1,wolbEndo_OF_DROSOPHIL,burkViet_G4,propAcne_KPA171202,enteFaec_V583,campJeju_81_176,acidJS42,heliPylo_26695,pseuHalo_TAC125,chroSale_DSM3043,methVann1,archFulg1,neisMeni_Z2491_1,fusoNucl,vermEise_EF01_2,anabVari_ATCC29413,tropWhip_TW08_27,heliHepa,acinSp_ADP1,anapMarg_ST_MARIES,natrPhar1,haheChej_KCTC_2396,therPetr_RKU_1,neisGono_FA1090_1,colwPsyc_34H,desuPsyc_LSV54,hyphNept_ATCC15444,vibrChol1,deinGeot_DSM11300,strePyog_M1_GAS,franCcI3,salmTyph,metaSedu,lactSali_UCC118,trepPall,neisMeni_MC58_1,syntWolf_GOETTINGEN,flavJohn_UW101,methBoon1,haemSomn_129PT,shewLoihPV4,igniHosp1,haemInfl_KW20,haloHalo_SL1,ferrAcid1,sphiAlas_RB2256,candPela_UBIQUE_HTCC1,caldSacc_DSM8903,aerPer1,lactPlan,carbHydr_Z_2901,therTher_HB8,vibrVuln_YJ016_1,rhodPalu_CGA009,acidCell_11B,siliPome_DSS_3,therVolc1,haloWals1,rubrXyla_DSM9941,shewAmaz,nocaJS61,vibrVuln_CMCP6_1,sinoMeli,ureaUrea,baciHalo,bartHens_HOUSTON_1,nitrWino_NB_255,hypeButy1,methBurt2,polaJS66,mesoLoti,methMari_C7,caulCres,neisMeni_FAM18_1,acidBact_ELLIN345,caldMaqu1,salmEnte_PARATYPI_ATC,glucOxyd_621H,cytoHutc_ATCC33406,nitrEuro,therMari,coxiBurn,woliSucc,heliPylo_HPAG1,mesoFlor_L1,pyrHor1,methAeol1,procMari_CCMP1375,pyroArse1,oenoOeni_PSU_1,alcaBork_SK2,wiggBrev,actiPleu_L20,lactLact,methJann1,paraDeni_PD1222,borrBurg,pyroIsla1,orieTsut_BORYONG,shewMR4,methKand1,methCaps_BATH,onioYell_PHYTOPLASMA,bordBron,cenaSymb1,burkCeno_HI2424,franTula_TULARENSIS,pyrFur2,mariAqua_VT8,heliPylo_J99,psycArct_273_4,vibrChol_MO10_1,vibrPara1,rickBell_RML369_C,metAce1,buchSp,ehrlRumi_WELGEVONDEN,methLabrZ_1,chlaPneu_CWL029,thioCrun_XCL_2,pyroCali1,chloTepi_TLS,stapAure_MU50,novoArom_DSM12444,magnMC1,zymoMobi_ZM4,salmTyph_TY2,chloChlo_CAD3,azoaSp_EBN1,therTher_HB27,bifiLong,picrTorr1,listInno,bdelBact,gramFors_KT0803,sulfAcid1,geobTher_NG80_2,peloCarb,ralsEutr_JMP134,mannSucc_MBEL55E,syneSp_WH8102,methTherPT1,clavMich_NCPPB_382,therAcid1,syntAcid_SB,porpGing_W83,therNeut0,leifXyli_XYLI_CTCB0,shewFrig,photProf_SS9,thioDeni_ATCC25259,methMaze1,desuRedu_MI_1,burkThai_E264,campFetu_82_40,blocFlor,jannCCS1,nitrMult_ATCC25196,streCoel,soliUsit_ELLIN6076,pastMult,saliRube_DSM13855,methTher1,nostSp,shigFlex_2A,saccDegr_2_40,oceaIhey,dehaEthe_195,rhodRubr_ATCC11170,arthFB24,shewMR7,pireSp,anaeDeha_2CP_C,haloVolc1,dichNodo_VCS1703A,tricEryt_IMS101,mycoGeni,thioDeni_ATCC33889,methSmit1,geobUran_RF4,shewDeni,halMar1,desuHafn_Y51,methStad1,granBeth_CGDNIH1,therPend1,legiPneu_PHILADELPHIA,vibrChol_O395_1,nitrOcea_ATCC19707,campJeju_RM1221,methPetr_PM1,heliAcin_SHEEBA,eschColi_APEC_O1,peloTher_SI,haloHalo1,syntFuma_MPOB,xyleFast,gloeViol,leucMese_ATCC8293,bactThet_VPI_5482,xantCamp,sodaGlos_MORSITANS,geobSulf,roseDeni_OCH_114,coryEffi_YS_314,brucMeli,mycoTube_H37RV,vibrFisc_ES114_1,pyrAby1,burkXeno_LB400,polyQLWP,stapMari1,peloLute_DSM273,burkCeno_AU_1054,shewBalt,nocaFarc_IFM10152,ente638,mculMari1,saliTrop_CNB_440,neorSenn_MIYAYAMA,aquiAeol,dechArom_RCB,myxoXant_DK_1622,burkPseu_1106A,burkCepa_AMMD,methMari_C5_1,azorCaul2,methFlag_KT,leptInte,eschColi_K12,synePCC6,baumCica_HOMALODISCA,methBark1,pseuAeru,geobMeta_GS15,eschColi_CFT073,photLumi,metMar1,hermArse,campJeju,therKoda1,aeroHydr_ATCC7966,baciAnth_AMES,shewOnei,therTeng,lawsIntr_PHE_MN1_00
> #Harvested from http://genome-test.cse.ucsc.edu/cgi-bin/das/dsn
>
>
>
>
> On 11/8/12 1:55 PM, "Dannon Baker" <[hidden email]> wrote:
>
>> That path should resolve to /mnt/galaxyTools/galaxy-central/tool-data/shared/ucsc/ (or ensembl, ncbi, etc)
>>
>> Can you tell me the output of 'hg tip' 'hg st' and 'hg diff' from the root /mnt/galaxyTools/galaxy-central/ ?
>>
>> -Dannon
>>
>> On Nov 8, 2012, at 1:41 PM, Scooter Willis <[hidden email]> wrote:
>>
>>> Errors are being output to stdout
>>> The files don't exist and complicated by the /../../ where I don't know
>>> which subdirectory to follow. Attached screen shot of job failing giving
>>> the ERROR message in the web gui. I also get the same errors when
>>> attempting to run Abys which doesn't run because it isn't installed.
>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$
>>> find -name publicbuilds.txt
>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy/util$ ls
>>> aliaspickler.py   debugging.py           hash_util.py    __init__.py
>>> lrucache.pyc   odict.pyc          streamball.py   topsort.pyc
>>> aliaspickler.pyc  docutils_ext           hash_util.pyc   __init__.pyc
>>> memdump.py     sanitize_html.py   streamball.pyc
>>> backports         docutils_template.txt  heartbeat.py    json.py
>>> none_like.py   sanitize_html.pyc  template.py
>>> bunch.py          expressions.py         inflection.py   json.pyc
>>> none_like.pyc  shed_util.py       template.pyc
>>> bunch.pyc         expressions.pyc        inflection.pyc  lrucache.py
>>> odict.py       shed_util.pyc      topsort.py
>>> On 11/8/12 1:34 PM, "Dannon Baker" <[hidden email]> wrote:
>>>> Glad to help.  The first step would be to check and see if those files
>>>> actually exist (and they should).
>>>> If those messages are occurring on stdout and not stderr though, they
>>>> shouldn't be causing failed jobs and something else might be wrong.
>>>> -Dannon
>>>> On Nov 8, 2012, at 1:27 PM, Scooter Willis <[hidden email]> wrote:
>>>>> Thanks! That was the explanation I was looking for and the link for how
>>>>> to
>>>>> Do you have any suggestions for fixing the error that are still
>>>>> occurring
>>>>> after running Bowtie?
>>>>> Sequence file aligned.
>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
>>>>> d/
>>>>> ucsc/publicbuilds.txt'
>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
>>>>> d/
>>>>> ensembl/builds.txt'
>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or directory:
>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-data/share
>>>>> d/
>>>>> ncbi/builds.txt'
>>>>> On 11/8/12 12:44 PM, "Dannon Baker" <[hidden email]> wrote:
>>>>>> You do not need to restart or add/remove worker nodes, the master's
>>>>>> tool
>>>>>> and data volumes are shared via NFS.
>>>>>> The easiest way to make this work for you in the cloud is probably
>>>>>> going
>>>>>> to be installing the version of the Abyss binaries you'd like to use
>>>>>> manually in /mnt/galaxyTools/abyss/<version>/ using galaxy's dependency
>>>>>> management
>>>>>> (http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies) and
>>>>>> then, after it has been installed, modify the abyss wrapper to include
>>>>>> a
>>>>>> <requirement type="package">abyss</requirement> tag, and finally
>>>>>> persist
>>>>>> changes to galaxyTools (which will also restart Galaxy, reloading the
>>>>>> tools).
>>>>>> -Dannon
>>>>>> On Nov 8, 2012, at 12:09 PM, Scooter Willis <[hidden email]>
>>>>>> wrote:
>>>>>>> Dannon
>>>>>>> If I install abyss to the galaxy tools volume and use cloud admin to
>>>>>>> do
>>>>>>> a
>>>>>>> snapshot I assume I then need to remove and then add compute nodes?
>>>>>>> From
>>>>>>> the EC2 management console I see two 15GiB EBS volumes from the same
>>>>>>> snapshot and I have a master and one compute node running. The other
>>>>>>> approach is that the compute nodes have a NFS mount back into the
>>>>>>> galaxy_tools volume. Trying to make sure I understand the need to
>>>>>>> restart
>>>>>>> the compute nodes after an install of third party tools.
>>>>>>> From looking at the abyss wrapper it is expecting ABYSS to be found on
>>>>>>> the
>>>>>>> path via a .sh file that gets called. Where would I update the path
>>>>>>> to a
>>>>>>> particular installed tool such that it will be picked up for each
>>>>>>> compute
>>>>>>> node that gets started? I couldn't find any examples of doing a third
>>>>>>> party install with discussion about how that install would persist to
>>>>>>> future compute nodes. The missing element for me at the moment is
>>>>>>> where
>>>>>>> to
>>>>>>> make path related changes that will be picked up by compute nodes. I
>>>>>>> could
>>>>>>> hard code the path in the .sh file used by the wrapper. Just trying to
>>>>>>> figure out how to use the software as designed.
>>>>>>> Thanks
>>>>>>> Scooter
>>>>>>> On 11/8/12 11:49 AM, "Dannon Baker" <[hidden email]> wrote:
>>>>>>>> Neither abyss wrapper in the toolshed installs binaries (you'd see a
>>>>>>>> tool_dependencies.xml in the repository), that's left up to the end
>>>>>>>> user.
>>>>>>>> You might contact the authors of the wrappers via the toolshed to see
>>>>>>>> if
>>>>>>>> they were going to add dependency management and had a newer version,
>>>>>>>> but
>>>>>>>> otherwise you'll have to install binaries on your own.  The cloud
>>>>>>>> volumes
>>>>>>>> do have a version of the Abyss binaries preinstalled, but it's
>>>>>>>> certainly
>>>>>>>> possible to add a newer version to your particular instance and
>>>>>>>> persist
>>>>>>>> the changes to the tool volume (see the cloud admin panel).
>>>>>>>> On Nov 8, 2012, at 11:20 AM, Scooter Willis <[hidden email]>
>>>>>>>> wrote:
>>>>>>>>> Ok that fixed Bowtie.
>>>>>>>>> Any thoughts on expected state of installing Abyss from toolshed. It
>>>>>>>>> indicates no requirements so my expectation is that installing from
>>>>>>>>> toolshed should install the appropriate abyss package in a location
>>>>>>>>> the
>>>>>>>>> tool is expecting to find it.
>>>>>>>>> In looking at the toolshed looks like two wrappers have been
>>>>>>>>> configured.
>>>>>>>>> One by msjeon and the other edward-kirton. I did the install for
>>>>>>>>> msjeon.
>>>>>>>>> Will try the other.
>>>>>>>>> Is the abyss install located at
>>>>>>>>> /mnt/galaxyTools/tools/abyss/1.2.5/bin
>>>>>>>>> from prior testing and should be deleted. Current release version of
>>>>>>>>> abyss
>>>>>>>>> as of May 30 2012 is Abyss 1.3.4
>>>>>>>>> On 11/8/12 11:00 AM, "Dannon Baker" <[hidden email]> wrote:
>>>>>>>>>> Unfortunately the cloud instance upgrade path requires some manual
>>>>>>>>>> intervention here due to tool migrations.  SSH in to your instance,
>>>>>>>>>> edit
>>>>>>>>>> /mnt/galaxyTools/galaxy-central/datatypes_conf.xml removing any
>>>>>>>>>> references to BlastXML.  Save the file, restart galaxy, and you
>>>>>>>>>> should
>>>>>>>>>> be
>>>>>>>>>> good to go.
>>>>>>>>>> -Dannon
>>>>>>>>>> On Nov 8, 2012, at 10:40 AM, Scooter Willis <[hidden email]>
>>>>>>>>>> wrote:
>>>>>>>>>>> Running a CloudMan created instance on Amazon.
>>>>>>>>>>> Updated using cloudman from
>>>>>>>>>>> http://bitbucket.org/galaxy/galaxy-central
>>>>>>>>>>> to version 8167:b0c916349070
>>>>>>>>>>> Tried to run bowtie and get the following error. Searched the
>>>>>>>>>>> message
>>>>>>>>>>> archives an someone else reported the same problem but thought it
>>>>>>>>>>> was
>>>>>>>>>>> related to a new version of cufflinks. Same problem also exists
>>>>>>>>>>> when
>>>>>>>>>>> running abyss. Given the nature of the error message not even sure
>>>>>>>>>>> what
>>>>>>>>>>> the actual path should be. Also for the abyss install from
>>>>>>>>>>> toolshed
>>>>>>>>>>> the
>>>>>>>>>>> toolshed.g2.bx.psu.edu directory doesn't exist or the six
>>>>>>>>>>> directories
>>>>>>>>>>> underneath. However Abyss was/is installed at the following
>>>>>>>>>>> location
>>>>>>>>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/tools/abyss/1.2.5/bin$
>>>>>>>>>>>> ls
>>>>>>>>>>>> ABYSS      abyss-adjtodot  abyss-joindist  abyss-tofastq
>>>>>>>>>>>> DistanceEst
>>>>>>>>>>>> MergePaths   PathConsensus  SimpleGraph
>>>>>>>>>>>> abyss2ace  abyss-fac       ABYSS-P         AdjList
>>>>>>>>>>>> KAligner
>>>>>>>>>>>> Overlap      PathOverlap
>>>>>>>>>>>> abyss2afg  abyss-fixmate   abyss-pe        Consensus
>>>>>>>>>>>> MergeContigs
>>>>>>>>>>>> ParseAligns  PopBubbles
>>>>>>>>>>> Would be great if someone could give some guidance on how to
>>>>>>>>>>> resolve
>>>>>>>>>>> the problems. I understand if I need to do something under the
>>>>>>>>>>> covers
>>>>>>>>>>> but the expectation is that the one click install/updates should
>>>>>>>>>>> not
>>>>>>>>>>> then require major command line work that isn't documented.
>>>>>>>>>>> Thanks
>>>>>>>>>>> Scooter
>>>>>>>>>>> Bowtie
>>>>>>>>>>> Stdout
>>>>>>>>>>>> Sequence file aligned.
>>>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>>>> directory:
>>>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>>>> a/
>>>>>>>>>>>> sh
>>>>>>>>>>>> ar
>>>>>>>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>>>> directory:
>>>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>>>> a/
>>>>>>>>>>>> sh
>>>>>>>>>>>> ar
>>>>>>>>>>>> ed/ensembl/builds.txt'
>>>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>>>> directory:
>>>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>>>> a/
>>>>>>>>>>>> sh
>>>>>>>>>>>> ar
>>>>>>>>>>>> ed/ncbi/builds.txt'
>>>>>>>>>>> Stderr
>>>>>>>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>>>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>>>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>>>>>>>> datatype
>>>>>>>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object
>>>>>>>>>>>> has
>>>>>>>>>>>> no
>>>>>>>>>>>> attribute 'BlastXml'
>>>>>>>>>>> Using toolshed installed Abyss and get the same error in Stdout.
>>>>>>>>>>> Stdout
>>>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>>>> directory:
>>>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>>>> a/
>>>>>>>>>>>> sh
>>>>>>>>>>>> ar
>>>>>>>>>>>> ed/ucsc/publicbuilds.txt'
>>>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>>>> directory:
>>>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>>>> a/
>>>>>>>>>>>> sh
>>>>>>>>>>>> ar
>>>>>>>>>>>> ed/ensembl/builds.txt'
>>>>>>>>>>>> ERROR: Unable to read builds file: [Errno 2] No such file or
>>>>>>>>>>>> directory:
>>>>>>>>>>>> '/mnt/galaxyTools/galaxy-central/lib/galaxy/util/../../../tool-dat
>>>>>>>>>>>> a/
>>>>>>>>>>>> sh
>>>>>>>>>>>> ar
>>>>>>>>>>>> ed/ncbi/builds.txt'
>>>>>>>>>>> Stderr
>>>>>>>>>>>> COMMAND FAILURE:
>>>>>>>>>>>> /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/msjeon/ab
>>>>>>>>>>>> ys
>>>>>>>>>>>> s_
>>>>>>>>>>>> to
>>>>>>>>>>>> ol/ce99b6666d1f/abyss_tool/abyss_toolsuite-92636934a189/abyss/abys
>>>>>>>>>>>> s_
>>>>>>>>>>>> wr
>>>>>>>>>>>> ap
>>>>>>>>>>>> per.sh: line 9: ABYSS: command not found
>>>>>>>>>>>> WARNING:galaxy.datatypes.registry:Error loading datatype with
>>>>>>>>>>>> extension 'blastxml': 'module' object has no attribute 'BlastXml'
>>>>>>>>>>>> WARNING:galaxy.datatypes.registry:Error appending sniffer for
>>>>>>>>>>>> datatype
>>>>>>>>>>>> 'galaxy.datatypes.xml:BlastXml' to sniff_order: 'module' object
>>>>>>>>>>>> has
>>>>>>>>>>>> no
>>>>>>>>>>>> attribute 'BlastXml'
>>>>>>>>>>> Looking at the util directory based on the expected path of the
>>>>>>>>>>> read
>>>>>>>>>>> build file
>>>>>>>>>>>> galaxy@ip-10-110-143-10:/mnt/galaxyTools/galaxy-central/lib/galaxy
>>>>>>>>>>>> /u
>>>>>>>>>>>> ti
>>>>>>>>>>>> l$
>>>>>>>>>>>> ls
>>>>>>>>>>>> aliaspickler.py   docutils_ext           heartbeat.py    json.pyc
>>>>>>>>>>>> odict.py           streamball.py
>>>>>>>>>>>> aliaspickler.pyc  docutils_template.txt  inflection.py
>>>>>>>>>>>> lrucache.py
>>>>>>>>>>>> odict.pyc          streamball.pyc
>>>>>>>>>>>> backports         expressions.py         inflection.pyc
>>>>>>>>>>>> lrucache.pyc
>>>>>>>>>>>> sanitize_html.py   template.py
>>>>>>>>>>>> bunch.py          expressions.pyc        __init__.py
>>>>>>>>>>>> memdump.py
>>>>>>>>>>>> sanitize_html.pyc  template.pyc
>>>>>>>>>>>> bunch.pyc         hash_util.py           __init__.pyc
>>>>>>>>>>>> none_like.py
>>>>>>>>>>>> shed_util.py       topsort.py
>>>>>>>>>>>> debugging.py      hash_util.pyc          json.py
>>>>>>>>>>>> none_like.pyc
>>>>>>>>>>>> shed_util.pyc      topsort.pyc
>>>>>>>>>>> ___________________________________________________________
>>>>>>>>>>> Please keep all replies on the list by using "reply all"
>>>>>>>>>>> in your mail client.  To manage your subscriptions to this
>>>>>>>>>>> and other Galaxy lists, please use the interface at:
>>>>>>>>>>> http://lists.bx.psu.edu/
>>> <default.png>
>>
>>

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

run.sh (2K) Download Attachment