replace template keys in ncbi-blastp failing with exit value 9

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

replace template keys in ncbi-blastp failing with exit value 9

ML mail
Dear Ergatis users,

I have installed the latest version of Ergatis with the latest version of TIGR
Workflow without SGE (grid disabled in ergatis.ini) on Debian/GNU Linux 6.0 and
wanted to run the pipeline creation tutorial just to test including a simple
formatdb and ncbi-blastp. The formatdb part works fine and succeeds,
unfortunately not the ncbi-blastp part: somehow it stops in iterator i1 at the
"replace template keys" part under "analysis group elements" it gives there a
return value of 9 with the error message "Error executing script exit value: 9".

Below I have included the stderr output file of this "replace template keys"
part:

Died with 'Bad key $;FILTER_HSPS_FOR_STATS$; in template file' at
/opt/ergatis/bin/replace_template_keys.pl line 282

And here the related output of "view stdout/stderr" on the pipeline:

FATAL 16:07:56:504 [Thread: ID:74 - replace template keys] ExceptionHandler
handleExceptionQuietMode:137 Runtime error executing command
/opt/ergatis/bin/replace_template_keys
--component_conf=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/ncbi-blastp.default.final.config
 --distribopts=nodistrib=1
--iterator_list=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter
 --keys=ITERATOR_NAME=i1
--output_dir=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1
--output_xml=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter.xml.gz
z --template_xml=/opt/ergatis/docs/ncbi-blastp.i1.xml 1>
/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stdout
 2>
/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stderr

Error executing script exit value: 9

java.lang.RuntimeException: Error executing script exit value: 9
    at
org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:336)

    at
org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:160)

    at
org.tigr.workflow.common.JavaSystemCommandProcessor.process(JavaSystemCommandProcessor.java:192)

    at
org.tigr.workflow.common.CommandProcessor.executeCommand(CommandProcessor.java:167)

    at org.tigr.workflow.common.CommandProcessor.run(CommandProcessor.java:271)
    at
org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:243)

    at
org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:222)

    at
org.tigr.antware.shared.concurrent.ThreadPoolWorker.access$000(ThreadPoolWorker.java:26)

    at
org.tigr.antware.shared.concurrent.ThreadPoolWorker$1.run(ThreadPoolWorker.java:102)

    at java.lang.Thread.run(Thread.java:636)

Does anyone know what could be going wrong here? I am looking the whole
afternoon and couldn't find a solution yet.

Any feedback would be greatly appreciated.

Many thanks in advance!
Pete



     

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: replace template keys in ncbi-blastp failing with exit value 9

Aaron Gussman
Hi Pete,
  Typically when there's an error like that, it's because there's an
ergatis variable in the component.i1.xml that's not accounted for in
the component.config or any of the other configs.  In this case it
appears to be "$;FILTER_HSPS_FOR_STATS$;".  Most likely that variable
appears in ncbi-blastp.i1.xml but not
ncbi-blastp.default.final.config.

  I checked our installed version of ncbi-blastp.i1.xml, but it
doesn't have a $;FILTER_HSPS_FOR_STATS$;.  Possibly this was added in
a newer version?

=Aaron

On Mon, Feb 14, 2011 at 11:31 AM, ML mail <[hidden email]> wrote:

> Dear Ergatis users,
>
> I have installed the latest version of Ergatis with the latest version of TIGR
> Workflow without SGE (grid disabled in ergatis.ini) on Debian/GNU Linux 6.0 and
> wanted to run the pipeline creation tutorial just to test including a simple
> formatdb and ncbi-blastp. The formatdb part works fine and succeeds,
> unfortunately not the ncbi-blastp part: somehow it stops in iterator i1 at the
> "replace template keys" part under "analysis group elements" it gives there a
> return value of 9 with the error message "Error executing script exit value: 9".
>
> Below I have included the stderr output file of this "replace template keys"
> part:
>
> Died with 'Bad key $;FILTER_HSPS_FOR_STATS$; in template file' at
> /opt/ergatis/bin/replace_template_keys.pl line 282
>
> And here the related output of "view stdout/stderr" on the pipeline:
>
> FATAL 16:07:56:504 [Thread: ID:74 - replace template keys] ExceptionHandler
> handleExceptionQuietMode:137 Runtime error executing command
> /opt/ergatis/bin/replace_template_keys
> --component_conf=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/ncbi-blastp.default.final.config
>  --distribopts=nodistrib=1
> --iterator_list=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter
>  --keys=ITERATOR_NAME=i1
> --output_dir=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1
> --output_xml=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter.xml.gz
> z --template_xml=/opt/ergatis/docs/ncbi-blastp.i1.xml 1>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stdout
>  2>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stderr
>
> Error executing script exit value: 9
>
> java.lang.RuntimeException: Error executing script exit value: 9
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:336)
>
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:160)
>
>    at
> org.tigr.workflow.common.JavaSystemCommandProcessor.process(JavaSystemCommandProcessor.java:192)
>
>    at
> org.tigr.workflow.common.CommandProcessor.executeCommand(CommandProcessor.java:167)
>
>    at org.tigr.workflow.common.CommandProcessor.run(CommandProcessor.java:271)
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:243)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:222)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.access$000(ThreadPoolWorker.java:26)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker$1.run(ThreadPoolWorker.java:102)
>
>    at java.lang.Thread.run(Thread.java:636)
>
> Does anyone know what could be going wrong here? I am looking the whole
> afternoon and couldn't find a solution yet.
>
> Any feedback would be greatly appreciated.
>
> Many thanks in advance!
> Pete
>
>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: replace template keys in ncbi-blastp failing with exit value 9

Joshua Orvis
Pete -

Aaron's right here.  Did you pull files from SVN for your install or download a release tarball?  It looks like you can fix your problem by adding the following line to the ncbi-blastp.config file in your installation and then building a new pipeline:

$;FILTER_HSPS_FOR_STATS$;=1

Joshua



On Mon, Feb 14, 2011 at 10:59 AM, Aaron Gussman <[hidden email]> wrote:
Hi Pete,
 Typically when there's an error like that, it's because there's an
ergatis variable in the component.i1.xml that's not accounted for in
the component.config or any of the other configs.  In this case it
appears to be "$;FILTER_HSPS_FOR_STATS$;".  Most likely that variable
appears in ncbi-blastp.i1.xml but not
ncbi-blastp.default.final.config.

 I checked our installed version of ncbi-blastp.i1.xml, but it
doesn't have a $;FILTER_HSPS_FOR_STATS$;.  Possibly this was added in
a newer version?

=Aaron

On Mon, Feb 14, 2011 at 11:31 AM, ML mail <[hidden email]> wrote:
> Dear Ergatis users,
>
> I have installed the latest version of Ergatis with the latest version of TIGR
> Workflow without SGE (grid disabled in ergatis.ini) on Debian/GNU Linux 6.0 and
> wanted to run the pipeline creation tutorial just to test including a simple
> formatdb and ncbi-blastp. The formatdb part works fine and succeeds,
> unfortunately not the ncbi-blastp part: somehow it stops in iterator i1 at the
> "replace template keys" part under "analysis group elements" it gives there a
> return value of 9 with the error message "Error executing script exit value: 9".
>
> Below I have included the stderr output file of this "replace template keys"
> part:
>
> Died with 'Bad key $;FILTER_HSPS_FOR_STATS$; in template file' at
> /opt/ergatis/bin/replace_template_keys.pl line 282
>
> And here the related output of "view stdout/stderr" on the pipeline:
>
> FATAL 16:07:56:504 [Thread: ID:74 - replace template keys] ExceptionHandler
> handleExceptionQuietMode:137 Runtime error executing command
> /opt/ergatis/bin/replace_template_keys
> --component_conf=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/ncbi-blastp.default.final.config
>  --distribopts=nodistrib=1
> --iterator_list=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter
>  --keys=ITERATOR_NAME=i1
> --output_dir=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1
> --output_xml=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter.xml.gz
> z --template_xml=/opt/ergatis/docs/ncbi-blastp.i1.xml 1>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stdout
>  2>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stderr
>
> Error executing script exit value: 9
>
> java.lang.RuntimeException: Error executing script exit value: 9
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:336)
>
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:160)
>
>    at
> org.tigr.workflow.common.JavaSystemCommandProcessor.process(JavaSystemCommandProcessor.java:192)
>
>    at
> org.tigr.workflow.common.CommandProcessor.executeCommand(CommandProcessor.java:167)
>
>    at org.tigr.workflow.common.CommandProcessor.run(CommandProcessor.java:271)
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:243)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:222)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.access$000(ThreadPoolWorker.java:26)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker$1.run(ThreadPoolWorker.java:102)
>
>    at java.lang.Thread.run(Thread.java:636)
>
> Does anyone know what could be going wrong here? I am looking the whole
> afternoon and couldn't find a solution yet.
>
> Any feedback would be greatly appreciated.
>
> Many thanks in advance!
> Pete
>
>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: replace template keys in ncbi-blastp failing with exit value 9

ML mail
Hi Aaron, Hi Joshua,

First of all many thanks for the very fast answer! Sorry for not being more precise, I should have mentioned the exact version number of ergatis I am using. So to resume, I am using the tarball from sourceforge and not the SVN version, here are exact version numbers:

ergatis: v2r13b2
BSML: v2r15b1

As suggested by you guys, I have now added the $;FILTER_HSPS_FOR_STATS$;=1 parameter under [parameters] stanza in the NCBI blastp component config file /opt/ergatis/docs/ncbi-blastp.conf and rebuilt the pipeline. The good news is that replace template keys doesn't complain anymore about that specific variable but now about another variable as you can see below:

Died with 'Bad key $;COMPRESS_BSML_OUTPUT$; in template file' at /opt/ergatis/bin/replace_template_keys.pl line 282

Can it be that I did something wrong at the installation or do you suggest me to use the SVN dev version instead of the tarball? I guess I shouldn't be manually adding (all) the missing variable in the component config file.

Looking forward to your feedback.

Pete



From: Joshua Orvis <[hidden email]>
To: [hidden email]
Sent: Mon, February 14, 2011 5:02:28 PM
Subject: Re: [Ergatis-users] replace template keys in ncbi-blastp failing with exit value 9

Pete -

Aaron's right here.  Did you pull files from SVN for your install or download a release tarball?  It looks like you can fix your problem by adding the following line to the ncbi-blastp.config file in your installation and then building a new pipeline:

$;FILTER_HSPS_FOR_STATS$;=1

Joshua



On Mon, Feb 14, 2011 at 10:59 AM, Aaron Gussman <[hidden email]> wrote:
Hi Pete,
 Typically when there's an error like that, it's because there's an
ergatis variable in the component.i1.xml that's not accounted for in
the component.config or any of the other configs.  In this case it
appears to be "$;FILTER_HSPS_FOR_STATS$;".  Most likely that variable
appears in ncbi-blastp.i1.xml but not
ncbi-blastp.default.final.config.

 I checked our installed version of ncbi-blastp.i1.xml, but it
doesn't have a $;FILTER_HSPS_FOR_STATS$;.  Possibly this was added in
a newer version?

=Aaron

On Mon, Feb 14, 2011 at 11:31 AM, ML mail <[hidden email]> wrote:
> Dear Ergatis users,
>
> I have installed the latest version of Ergatis with the latest version of TIGR
> Workflow without SGE (grid disabled in ergatis.ini) on Debian/GNU Linux 6.0 and
> wanted to run the pipeline creation tutorial just to test including a simple
> formatdb and ncbi-blastp. The formatdb part works fine and succeeds,
> unfortunately not the ncbi-blastp part: somehow it stops in iterator i1 at the
> "replace template keys" part under "analysis group elements" it gives there a
> return value of 9 with the error message "Error executing script exit value: 9".
>
> Below I have included the stderr output file of this "replace template keys"
> part:
>
> Died with 'Bad key $;FILTER_HSPS_FOR_STATS$; in template file' at
> /opt/ergatis/bin/replace_template_keys.pl line 282
>
> And here the related output of "view stdout/stderr" on the pipeline:
>
> FATAL 16:07:56:504 [Thread: ID:74 - replace template keys] ExceptionHandler
> handleExceptionQuietMode:137 Runtime error executing command
> /opt/ergatis/bin/replace_template_keys
> --component_conf=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/ncbi-blastp.default.final.config
>  --distribopts=nodistrib=1
> --iterator_list=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter
>  --keys=ITERATOR_NAME=i1
> --output_dir=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1
> --output_xml=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter.xml.gz
> z --template_xml=/opt/ergatis/docs/ncbi-blastp.i1.xml 1>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stdout
>  2>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stderr
>
> Error executing script exit value: 9
>
> java.lang.RuntimeException: Error executing script exit value: 9
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:336)
>
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:160)
>
>    at
> org.tigr.workflow.common.JavaSystemCommandProcessor.process(JavaSystemCommandProcessor.java:192)
>
>    at
> org.tigr.workflow.common.CommandProcessor.executeCommand(CommandProcessor.java:167)
>
>    at org.tigr.workflow.common.CommandProcessor.run(CommandProcessor.java:271)
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:243)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:222)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.access$000(ThreadPoolWorker.java:26)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker$1.run(ThreadPoolWorker.java:102)
>
>    at java.lang.Thread.run(Thread.java:636)
>
> Does anyone know what could be going wrong here? I am looking the whole
> afternoon and couldn't find a solution yet.
>
> Any feedback would be greatly appreciated.
>
> Many thanks in advance!
> Pete
>
>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: replace template keys in ncbi-blastp failing with exit value 9

Joshua Orvis
Pete -

Sorry about the problems you're encountering here.  I'll look into patching the last release, where it looks like that component definition is broken.  I'm attaching the current code from the SVN head that you can try.  Just put these files in place over your existing ncbi-blastp component files and try a new pipeline.

Joshua




2011/2/14 ML mail <[hidden email]>
Hi Aaron, Hi Joshua,

First of all many thanks for the very fast answer! Sorry for not being more precise, I should have mentioned the exact version number of ergatis I am using. So to resume, I am using the tarball from sourceforge and not the SVN version, here are exact version numbers:

ergatis: v2r13b2
BSML: v2r15b1

As suggested by you guys, I have now added the $;FILTER_HSPS_FOR_STATS$;=1 parameter under [parameters] stanza in the NCBI blastp component config file /opt/ergatis/docs/ncbi-blastp.conf and rebuilt the pipeline. The good news is that replace template keys doesn't complain anymore about that specific variable but now about another variable as you can see below:

Died with 'Bad key $;COMPRESS_BSML_OUTPUT$; in template file' at /opt/ergatis/bin/replace_template_keys.pl line 282

Can it be that I did something wrong at the installation or do you suggest me to use the SVN dev version instead of the tarball? I guess I shouldn't be manually adding (all) the missing variable in the component config file.

Looking forward to your feedback.

Pete



From: Joshua Orvis <[hidden email]>
To: [hidden email]
Sent: Mon, February 14, 2011 5:02:28 PM
Subject: Re: [Ergatis-users] replace template keys in ncbi-blastp failing with exit value 9

Pete -

Aaron's right here.  Did you pull files from SVN for your install or download a release tarball?  It looks like you can fix your problem by adding the following line to the ncbi-blastp.config file in your installation and then building a new pipeline:

$;FILTER_HSPS_FOR_STATS$;=1

Joshua



On Mon, Feb 14, 2011 at 10:59 AM, Aaron Gussman <[hidden email]> wrote:
Hi Pete,
 Typically when there's an error like that, it's because there's an
ergatis variable in the component.i1.xml that's not accounted for in
the component.config or any of the other configs.  In this case it
appears to be "$;FILTER_HSPS_FOR_STATS$;".  Most likely that variable
appears in ncbi-blastp.i1.xml but not
ncbi-blastp.default.final.config.

 I checked our installed version of ncbi-blastp.i1.xml, but it
doesn't have a $;FILTER_HSPS_FOR_STATS$;.  Possibly this was added in
a newer version?

=Aaron

On Mon, Feb 14, 2011 at 11:31 AM, ML mail <[hidden email]> wrote:
> Dear Ergatis users,
>
> I have installed the latest version of Ergatis with the latest version of TIGR
> Workflow without SGE (grid disabled in ergatis.ini) on Debian/GNU Linux 6.0 and
> wanted to run the pipeline creation tutorial just to test including a simple
> formatdb and ncbi-blastp. The formatdb part works fine and succeeds,
> unfortunately not the ncbi-blastp part: somehow it stops in iterator i1 at the
> "replace template keys" part under "analysis group elements" it gives there a
> return value of 9 with the error message "Error executing script exit value: 9".
>
> Below I have included the stderr output file of this "replace template keys"
> part:
>
> Died with 'Bad key $;FILTER_HSPS_FOR_STATS$; in template file' at
> /opt/ergatis/bin/replace_template_keys.pl line 282
>
> And here the related output of "view stdout/stderr" on the pipeline:
>
> FATAL 16:07:56:504 [Thread: ID:74 - replace template keys] ExceptionHandler
> handleExceptionQuietMode:137 Runtime error executing command
> /opt/ergatis/bin/replace_template_keys
> --component_conf=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/ncbi-blastp.default.final.config
>  --distribopts=nodistrib=1
> --iterator_list=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter
>  --keys=ITERATOR_NAME=i1
> --output_dir=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1
> --output_xml=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter.xml.gz
> z --template_xml=/opt/ergatis/docs/ncbi-blastp.i1.xml 1>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stdout
>  2>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stderr
>
> Error executing script exit value: 9
>
> java.lang.RuntimeException: Error executing script exit value: 9
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:336)
>
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:160)
>
>    at
> org.tigr.workflow.common.JavaSystemCommandProcessor.process(JavaSystemCommandProcessor.java:192)
>
>    at
> org.tigr.workflow.common.CommandProcessor.executeCommand(CommandProcessor.java:167)
>
>    at org.tigr.workflow.common.CommandProcessor.run(CommandProcessor.java:271)
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:243)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:222)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.access$000(ThreadPoolWorker.java:26)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker$1.run(ThreadPoolWorker.java:102)
>
>    at java.lang.Thread.run(Thread.java:636)
>
> Does anyone know what could be going wrong here? I am looking the whole
> afternoon and couldn't find a solution yet.
>
> Any feedback would be greatly appreciated.
>
> Many thanks in advance!
> Pete
>
>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

ncbi-blastp.config (3K) Download Attachment
ncbi-blastp.i1.xml (7K) Download Attachment
ncbi-blastp.xml (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

replace template keys in ncbi-blastp failing with exit value 9

ML mail
Dear Joshua,

Thanks for the workaround, I tried copying the 3 new ncbp-blastp component files you have provided me yesterday over the current ones of my local installation. So now the good news is that the "replace template keys" step of the ncbi-blastp component gets done correctly but the workflow gets stuck at the "convert raw to btab" step just after running the ncbi-blastp step.

I can see the blast2btab processes (shell script + PERL script) as you can see below (command: ps -ef|grep btab):

www-data 10665 10541  0 13:20 ?        00:00:00 /bin/sh /opt/ergatis/bin/blast2btab --input=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.raw --output=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.btab --split=0
www-data 10668 10665  0 13:20 ?        00:00:00 /usr/bin/perl /opt/ergatis/bin/blast2btab.pl --input=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.raw --output=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.btab --split=0

But nothing happens, an strace of the blast2btab.pl script (PID: 10668) revels that the process is waiting in a read function, as you can see below:

% sudo strace -p 10668
Process 10668 attached - interrupt to quit
read(0,

On the other hand, I managed to run a small test pipeline using formatdb and ncbi-tblastn, so it might be just a problem related to the ncbi-blastp component.

My test pipeline which fails with ncbi-blastp was using the sample proteins.faa file delivered with the package and installed in /opt/ergatis/samples/proteins.faa and because there isn't any glutamine_synthetases.faa file as suggested in the documentation for the pipeline creation. I have used the annotated proteins fasta file of the genome of Deferribacter desulfuricans available here ftp://ftp.ncbi.nih.gov/genbank/genomes/Bacteria/Deferribacter_desulfuricans_SSM1_uid37285/AP011529.faa

Would it be wiser to re-install ergatis using the latest svn HEAD (trunk) release, what do you think? Also do let me know if you need any more details or log outputs.

Regard,
Pete




From: Joshua Orvis <[hidden email]>
To: [hidden email]
Sent: Mon, February 14, 2011 6:12:38 PM
Subject: Re: [Ergatis-users] replace template keys in ncbi-blastp failing with exit value 9

Pete -

Sorry about the problems you're encountering here.  I'll look into patching the last release, where it looks like that component definition is broken.  I'm attaching the current code from the SVN head that you can try.  Just put these files in place over your existing ncbi-blastp component files and try a new pipeline.

Joshua




2011/2/14 ML mail <[hidden email]>
Hi Aaron, Hi Joshua,

First of all many thanks for the very fast answer! Sorry for not being more precise, I should have mentioned the exact version number of ergatis I am using. So to resume, I am using the tarball from sourceforge and not the SVN version, here are exact version numbers:

ergatis: v2r13b2
BSML: v2r15b1

As suggested by you guys, I have now added the $;FILTER_HSPS_FOR_STATS$;=1 parameter under [parameters] stanza in the NCBI blastp component config file /opt/ergatis/docs/ncbi-blastp.conf and rebuilt the pipeline. The good news is that replace template keys doesn't complain anymore about that specific variable but now about another variable as you can see below:

Died with 'Bad key $;COMPRESS_BSML_OUTPUT$; in template file' at /opt/ergatis/bin/replace_template_keys.pl line 282

Can it be that I did something wrong at the installation or do you suggest me to use the SVN dev version instead of the tarball? I guess I shouldn't be manually adding (all) the missing variable in the component config file.

Looking forward to your feedback.

Pete



From: Joshua Orvis <[hidden email]>
To: [hidden email]
Sent: Mon, February 14, 2011 5:02:28 PM
Subject: Re: [Ergatis-users] replace template keys in ncbi-blastp failing with exit value 9

Pete -

Aaron's right here.  Did you pull files from SVN for your install or download a release tarball?  It looks like you can fix your problem by adding the following line to the ncbi-blastp.config file in your installation and then building a new pipeline:

$;FILTER_HSPS_FOR_STATS$;=1

Joshua



On Mon, Feb 14, 2011 at 10:59 AM, Aaron Gussman <[hidden email]> wrote:
Hi Pete,
 Typically when there's an error like that, it's because there's an
ergatis variable in the component.i1.xml that's not accounted for in
the component.config or any of the other configs.  In this case it
appears to be "$;FILTER_HSPS_FOR_STATS$;".  Most likely that variable
appears in ncbi-blastp.i1.xml but not
ncbi-blastp.default.final.config.

 I checked our installed version of ncbi-blastp.i1.xml, but it
doesn't have a $;FILTER_HSPS_FOR_STATS$;.  Possibly this was added in
a newer version?

=Aaron

On Mon, Feb 14, 2011 at 11:31 AM, ML mail <[hidden email]> wrote:
> Dear Ergatis users,
>
> I have installed the latest version of Ergatis with the latest version of TIGR
> Workflow without SGE (grid disabled in ergatis.ini) on Debian/GNU Linux 6.0 and
> wanted to run the pipeline creation tutorial just to test including a simple
> formatdb and ncbi-blastp. The formatdb part works fine and succeeds,
> unfortunately not the ncbi-blastp part: somehow it stops in iterator i1 at the
> "replace template keys" part under "analysis group elements" it gives there a
> return value of 9 with the error message "Error executing script exit value: 9".
>
> Below I have included the stderr output file of this "replace template keys"
> part:
>
> Died with 'Bad key $;FILTER_HSPS_FOR_STATS$; in template file' at
> /opt/ergatis/bin/replace_template_keys.pl line 282
>
> And here the related output of "view stdout/stderr" on the pipeline:
>
> FATAL 16:07:56:504 [Thread: ID:74 - replace template keys] ExceptionHandler
> handleExceptionQuietMode:137 Runtime error executing command
> /opt/ergatis/bin/replace_template_keys
> --component_conf=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/ncbi-blastp.default.final.config
>  --distribopts=nodistrib=1
> --iterator_list=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter
>  --keys=ITERATOR_NAME=i1
> --output_dir=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1
> --output_xml=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter.xml.gz
> z --template_xml=/opt/ergatis/docs/ncbi-blastp.i1.xml 1>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stdout
>  2>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stderr
>
> Error executing script exit value: 9
>
> java.lang.RuntimeException: Error executing script exit value: 9
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:336)
>
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:160)
>
>    at
> org.tigr.workflow.common.JavaSystemCommandProcessor.process(JavaSystemCommandProcessor.java:192)
>
>    at
> org.tigr.workflow.common.CommandProcessor.executeCommand(CommandProcessor.java:167)
>
>    at org.tigr.workflow.common.CommandProcessor.run(CommandProcessor.java:271)
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:243)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:222)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.access$000(ThreadPoolWorker.java:26)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker$1.run(ThreadPoolWorker.java:102)
>
>    at java.lang.Thread.run(Thread.java:636)
>
> Does anyone know what could be going wrong here? I am looking the whole
> afternoon and couldn't find a solution yet.
>
> Any feedback would be greatly appreciated.
>
> Many thanks in advance!
> Pete
>
>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users





------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: replace template keys in ncbi-blastp failing with exit value 9

Joshua Orvis
There could definitely be issues with the mixing of file versions here.  Feel free to check out and install the latest in SVN with this command:

svn export https://ergatis.svn.sourceforge.net/svnroot/ergatis/release/trunk ergatis-trunk

Joshua



2011/2/15 ML mail <[hidden email]>
Dear Joshua,

Thanks for the workaround, I tried copying the 3 new ncbp-blastp component files you have provided me yesterday over the current ones of my local installation. So now the good news is that the "replace template keys" step of the ncbi-blastp component gets done correctly but the workflow gets stuck at the "convert raw to btab" step just after running the ncbi-blastp step.

I can see the blast2btab processes (shell script + PERL script) as you can see below (command: ps -ef|grep btab):

www-data 10665 10541  0 13:20 ?        00:00:00 /bin/sh /opt/ergatis/bin/blast2btab --input=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.raw --output=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.btab --split=0
www-data 10668 10665  0 13:20 ?        00:00:00 /usr/bin/perl /opt/ergatis/bin/blast2btab.pl --input=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.raw --output=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.btab --split=0

But nothing happens, an strace of the blast2btab.pl script (PID: 10668) revels that the process is waiting in a read function, as you can see below:

% sudo strace -p 10668
Process 10668 attached - interrupt to quit
read(0,

On the other hand, I managed to run a small test pipeline using formatdb and ncbi-tblastn, so it might be just a problem related to the ncbi-blastp component.

My test pipeline which fails with ncbi-blastp was using the sample proteins.faa file delivered with the package and installed in /opt/ergatis/samples/proteins.faa and because there isn't any glutamine_synthetases.faa file as suggested in the documentation for the pipeline creation. I have used the annotated proteins fasta file of the genome of Deferribacter desulfuricans available here ftp://ftp.ncbi.nih.gov/genbank/genomes/Bacteria/Deferribacter_desulfuricans_SSM1_uid37285/AP011529.faa

Would it be wiser to re-install ergatis using the latest svn HEAD (trunk) release, what do you think? Also do let me know if you need any more details or log outputs.

Regard,
Pete




From: Joshua Orvis <[hidden email]>
To: [hidden email]
Sent: Mon, February 14, 2011 6:12:38 PM

Subject: Re: [Ergatis-users] replace template keys in ncbi-blastp failing with exit value 9

Pete -

Sorry about the problems you're encountering here.  I'll look into patching the last release, where it looks like that component definition is broken.  I'm attaching the current code from the SVN head that you can try.  Just put these files in place over your existing ncbi-blastp component files and try a new pipeline.

Joshua




2011/2/14 ML mail <[hidden email]>
Hi Aaron, Hi Joshua,

First of all many thanks for the very fast answer! Sorry for not being more precise, I should have mentioned the exact version number of ergatis I am using. So to resume, I am using the tarball from sourceforge and not the SVN version, here are exact version numbers:

ergatis: v2r13b2
BSML: v2r15b1

As suggested by you guys, I have now added the $;FILTER_HSPS_FOR_STATS$;=1 parameter under [parameters] stanza in the NCBI blastp component config file /opt/ergatis/docs/ncbi-blastp.conf and rebuilt the pipeline. The good news is that replace template keys doesn't complain anymore about that specific variable but now about another variable as you can see below:

Died with 'Bad key $;COMPRESS_BSML_OUTPUT$; in template file' at /opt/ergatis/bin/replace_template_keys.pl line 282

Can it be that I did something wrong at the installation or do you suggest me to use the SVN dev version instead of the tarball? I guess I shouldn't be manually adding (all) the missing variable in the component config file.

Looking forward to your feedback.

Pete



From: Joshua Orvis <[hidden email]>
To: [hidden email]
Sent: Mon, February 14, 2011 5:02:28 PM
Subject: Re: [Ergatis-users] replace template keys in ncbi-blastp failing with exit value 9

Pete -

Aaron's right here.  Did you pull files from SVN for your install or download a release tarball?  It looks like you can fix your problem by adding the following line to the ncbi-blastp.config file in your installation and then building a new pipeline:

$;FILTER_HSPS_FOR_STATS$;=1

Joshua



On Mon, Feb 14, 2011 at 10:59 AM, Aaron Gussman <[hidden email]> wrote:
Hi Pete,
 Typically when there's an error like that, it's because there's an
ergatis variable in the component.i1.xml that's not accounted for in
the component.config or any of the other configs.  In this case it
appears to be "$;FILTER_HSPS_FOR_STATS$;".  Most likely that variable
appears in ncbi-blastp.i1.xml but not
ncbi-blastp.default.final.config.

 I checked our installed version of ncbi-blastp.i1.xml, but it
doesn't have a $;FILTER_HSPS_FOR_STATS$;.  Possibly this was added in
a newer version?

=Aaron

On Mon, Feb 14, 2011 at 11:31 AM, ML mail <[hidden email]> wrote:
> Dear Ergatis users,
>
> I have installed the latest version of Ergatis with the latest version of TIGR
> Workflow without SGE (grid disabled in ergatis.ini) on Debian/GNU Linux 6.0 and
> wanted to run the pipeline creation tutorial just to test including a simple
> formatdb and ncbi-blastp. The formatdb part works fine and succeeds,
> unfortunately not the ncbi-blastp part: somehow it stops in iterator i1 at the
> "replace template keys" part under "analysis group elements" it gives there a
> return value of 9 with the error message "Error executing script exit value: 9".
>
> Below I have included the stderr output file of this "replace template keys"
> part:
>
> Died with 'Bad key $;FILTER_HSPS_FOR_STATS$; in template file' at
> /opt/ergatis/bin/replace_template_keys.pl line 282
>
> And here the related output of "view stdout/stderr" on the pipeline:
>
> FATAL 16:07:56:504 [Thread: ID:74 - replace template keys] ExceptionHandler
> handleExceptionQuietMode:137 Runtime error executing command
> /opt/ergatis/bin/replace_template_keys
> --component_conf=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/ncbi-blastp.default.final.config
>  --distribopts=nodistrib=1
> --iterator_list=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter
>  --keys=ITERATOR_NAME=i1
> --output_dir=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1
> --output_xml=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter.xml.gz
> z --template_xml=/opt/ergatis/docs/ncbi-blastp.i1.xml 1>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stdout
>  2>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stderr
>
> Error executing script exit value: 9
>
> java.lang.RuntimeException: Error executing script exit value: 9
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:336)
>
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:160)
>
>    at
> org.tigr.workflow.common.JavaSystemCommandProcessor.process(JavaSystemCommandProcessor.java:192)
>
>    at
> org.tigr.workflow.common.CommandProcessor.executeCommand(CommandProcessor.java:167)
>
>    at org.tigr.workflow.common.CommandProcessor.run(CommandProcessor.java:271)
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:243)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:222)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.access$000(ThreadPoolWorker.java:26)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker$1.run(ThreadPoolWorker.java:102)
>
>    at java.lang.Thread.run(Thread.java:636)
>
> Does anyone know what could be going wrong here? I am looking the whole
> afternoon and couldn't find a solution yet.
>
> Any feedback would be greatly appreciated.
>
> Many thanks in advance!
> Pete
>
>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users





------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: replace template keys in ncbi-blastp failing with exit value 9

ML mail
I just had a go at the SVN version and I must say that my previous test pipeline (formatdb/ncbi-blastp) worked perfectly. So it looks like the issues I had in v2r13b2 are corrected in the trunk dev version of ergatis.

Now I just have two small comments:

- when running perl Makefile.PL INSTALL_BASE=/opt/ergatis it fails with the following error message:

failed to create WORKFLOW_DOCS_DIR: No such file or directory at Makefile.PL line 121.

In fact it is right: it cannot create /opt/ergatis/docs because /opt/ergatis doesn't exist at that stage, creating manualy /opt/ergatis fixes this problem but you might want to make sure that in the Makefile.PL $install_base gets created beforehand.

- still in the Makefile.PL, it seems that "pipeline_templates:" doesn't actually copy the global pipeline templates, in fact it only creates their directory first and then displays the files with echo as exec command in the find command (line 166 of Makefile.PL). Is there any reasons for that? Maybe these global pipeline templates are not considered "mature" yet to be installed automatically?

Pete



From: Joshua Orvis <[hidden email]>
To: [hidden email]
Sent: Tue, February 15, 2011 4:40:15 PM
Subject: Re: [Ergatis-users] replace template keys in ncbi-blastp failing with exit value 9

There could definitely be issues with the mixing of file versions here.  Feel free to check out and install the latest in SVN with this command:

svn export https://ergatis.svn.sourceforge.net/svnroot/ergatis/release/trunk ergatis-trunk

Joshua



2011/2/15 ML mail <[hidden email]>
Dear Joshua,

Thanks for the workaround, I tried copying the 3 new ncbp-blastp component files you have provided me yesterday over the current ones of my local installation. So now the good news is that the "replace template keys" step of the ncbi-blastp component gets done correctly but the workflow gets stuck at the "convert raw to btab" step just after running the ncbi-blastp step.

I can see the blast2btab processes (shell script + PERL script) as you can see below (command: ps -ef|grep btab):

www-data 10665 10541  0 13:20 ?        00:00:00 /bin/sh /opt/ergatis/bin/blast2btab --input=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.raw --output=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.btab --split=0
www-data 10668 10665  0 13:20 ?        00:00:00 /usr/bin/perl /opt/ergatis/bin/blast2btab.pl --input=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.raw --output=/usr/local/projects/strep/output_repository/ncbi-blastp/8_default/i1/g1/proteins.ncbi-blastp.btab --split=0

But nothing happens, an strace of the blast2btab.pl script (PID: 10668) revels that the process is waiting in a read function, as you can see below:

% sudo strace -p 10668
Process 10668 attached - interrupt to quit
read(0,

On the other hand, I managed to run a small test pipeline using formatdb and ncbi-tblastn, so it might be just a problem related to the ncbi-blastp component.

My test pipeline which fails with ncbi-blastp was using the sample proteins.faa file delivered with the package and installed in /opt/ergatis/samples/proteins.faa and because there isn't any glutamine_synthetases.faa file as suggested in the documentation for the pipeline creation. I have used the annotated proteins fasta file of the genome of Deferribacter desulfuricans available here ftp://ftp.ncbi.nih.gov/genbank/genomes/Bacteria/Deferribacter_desulfuricans_SSM1_uid37285/AP011529.faa

Would it be wiser to re-install ergatis using the latest svn HEAD (trunk) release, what do you think? Also do let me know if you need any more details or log outputs.

Regard,
Pete




From: Joshua Orvis <[hidden email]>
To: [hidden email]
Sent: Mon, February 14, 2011 6:12:38 PM

Subject: Re: [Ergatis-users] replace template keys in ncbi-blastp failing with exit value 9

Pete -

Sorry about the problems you're encountering here.  I'll look into patching the last release, where it looks like that component definition is broken.  I'm attaching the current code from the SVN head that you can try.  Just put these files in place over your existing ncbi-blastp component files and try a new pipeline.

Joshua




2011/2/14 ML mail <[hidden email]>
Hi Aaron, Hi Joshua,

First of all many thanks for the very fast answer! Sorry for not being more precise, I should have mentioned the exact version number of ergatis I am using. So to resume, I am using the tarball from sourceforge and not the SVN version, here are exact version numbers:

ergatis: v2r13b2
BSML: v2r15b1

As suggested by you guys, I have now added the $;FILTER_HSPS_FOR_STATS$;=1 parameter under [parameters] stanza in the NCBI blastp component config file /opt/ergatis/docs/ncbi-blastp.conf and rebuilt the pipeline. The good news is that replace template keys doesn't complain anymore about that specific variable but now about another variable as you can see below:

Died with 'Bad key $;COMPRESS_BSML_OUTPUT$; in template file' at /opt/ergatis/bin/replace_template_keys.pl line 282

Can it be that I did something wrong at the installation or do you suggest me to use the SVN dev version instead of the tarball? I guess I shouldn't be manually adding (all) the missing variable in the component config file.

Looking forward to your feedback.

Pete



From: Joshua Orvis <[hidden email]>
To: [hidden email]
Sent: Mon, February 14, 2011 5:02:28 PM
Subject: Re: [Ergatis-users] replace template keys in ncbi-blastp failing with exit value 9

Pete -

Aaron's right here.  Did you pull files from SVN for your install or download a release tarball?  It looks like you can fix your problem by adding the following line to the ncbi-blastp.config file in your installation and then building a new pipeline:

$;FILTER_HSPS_FOR_STATS$;=1

Joshua



On Mon, Feb 14, 2011 at 10:59 AM, Aaron Gussman <[hidden email]> wrote:
Hi Pete,
 Typically when there's an error like that, it's because there's an
ergatis variable in the component.i1.xml that's not accounted for in
the component.config or any of the other configs.  In this case it
appears to be "$;FILTER_HSPS_FOR_STATS$;".  Most likely that variable
appears in ncbi-blastp.i1.xml but not
ncbi-blastp.default.final.config.

 I checked our installed version of ncbi-blastp.i1.xml, but it
doesn't have a $;FILTER_HSPS_FOR_STATS$;.  Possibly this was added in
a newer version?

=Aaron

On Mon, Feb 14, 2011 at 11:31 AM, ML mail <[hidden email]> wrote:
> Dear Ergatis users,
>
> I have installed the latest version of Ergatis with the latest version of TIGR
> Workflow without SGE (grid disabled in ergatis.ini) on Debian/GNU Linux 6.0 and
> wanted to run the pipeline creation tutorial just to test including a simple
> formatdb and ncbi-blastp. The formatdb part works fine and succeeds,
> unfortunately not the ncbi-blastp part: somehow it stops in iterator i1 at the
> "replace template keys" part under "analysis group elements" it gives there a
> return value of 9 with the error message "Error executing script exit value: 9".
>
> Below I have included the stderr output file of this "replace template keys"
> part:
>
> Died with 'Bad key $;FILTER_HSPS_FOR_STATS$; in template file' at
> /opt/ergatis/bin/replace_template_keys.pl line 282
>
> And here the related output of "view stdout/stderr" on the pipeline:
>
> FATAL 16:07:56:504 [Thread: ID:74 - replace template keys] ExceptionHandler
> handleExceptionQuietMode:137 Runtime error executing command
> /opt/ergatis/bin/replace_template_keys
> --component_conf=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/ncbi-blastp.default.final.config
>  --distribopts=nodistrib=1
> --iterator_list=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter
>  --keys=ITERATOR_NAME=i1
> --output_dir=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1
> --output_xml=/usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/g1.iter.xml.gz
> z --template_xml=/opt/ergatis/docs/ncbi-blastp.i1.xml 1>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stdout
>  2>
> /usr/local/projects/strep/workflow/runtime/ncbi-blastp/2_default/i1/g1/replace_template_keys.stderr
>
> Error executing script exit value: 9
>
> java.lang.RuntimeException: Error executing script exit value: 9
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:336)
>
>    at
> org.tigr.antware.shared.util.RunSystemCommand.runCommand(RunSystemCommand.java:160)
>
>    at
> org.tigr.workflow.common.JavaSystemCommandProcessor.process(JavaSystemCommandProcessor.java:192)
>
>    at
> org.tigr.workflow.common.CommandProcessor.executeCommand(CommandProcessor.java:167)
>
>    at org.tigr.workflow.common.CommandProcessor.run(CommandProcessor.java:271)
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:243)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.runTarget(ThreadPoolWorker.java:222)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker.access$000(ThreadPoolWorker.java:26)
>
>    at
> org.tigr.antware.shared.concurrent.ThreadPoolWorker$1.run(ThreadPoolWorker.java:102)
>
>    at java.lang.Thread.run(Thread.java:636)
>
> Does anyone know what could be going wrong here? I am looking the whole
> afternoon and couldn't find a solution yet.
>
> Any feedback would be greatly appreciated.
>
> Many thanks in advance!
> Pete
>
>
>
>
>
> ------------------------------------------------------------------------------
> The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
> Pinpoint memory and threading errors before they happen.
> Find and fix more than 250 security defects in the development cycle.
> Locate bottlenecks in serial and parallel code that limit performance.
> http://p.sf.net/sfu/intel-dev2devfeb
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users





------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE:
Pinpoint memory and threading errors before they happen.
Find and fix more than 250 security defects in the development cycle.
Locate bottlenecks in serial and parallel code that limit performance.
http://p.sf.net/sfu/intel-dev2devfeb
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users