Pipeline stuck

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Pipeline stuck

ML mail
Dear Ergatis users,

I was out of luck running the prokaryotic genome annotation pipeline annotation
over this week-end because the virtual machine I was running it on (VirtualBox)
crashed.


Now I have the problem that Ergatis thinks the pipeline is still running and if
I try to get the details of the pipeline by clicking on the pipeline ID the
browser loads and loads for 5 minutes but all I get outputted back the the
browser is a comma, yes a "," comma and nothing else. The URL it is trying to
contact is the following:

http://localhost:8080/ergatis/cgi/view_pipeline.cgi?instance=/usr/local/projects/new/workflow/runtime/pipeline/6/pipeline.xml


On the web server side, here is what I get in the access_log file:

Timeout waiting for output from CGI script
/var/www/ergatis/cgi/view_pipeline.cgi, referer:
http://localhost:8080/ergatis/cgi/pipeline_list.cgi?repository_root=/usr/local/projects/new


Any idea what this could be? And how I can restart maybe my pipeline manually?
Or do I need to restart the whole pipeline again? I would really like to be able
to restart it from where it crashed if possible.

http://ergatis:8080/ergatis/cgi/view_pipeline.cgi?instance=/usr/local/projects/dsvnew/workflow/runtime/pipeline/6/pipeline.xml


Many thanks for your feedback.

ML


------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Pipeline stuck

ML mail
I managed to find out that the problem with the pipline viewer not loading was
that the pipeline.xml.log file was 14 GB big. So I simply deleted this file and
rerun the pipeline.

Is it normal that this file gets so huge?

Regards,
ML




----- Original Message ----
From: ML mail <[hidden email]>
To: [hidden email]
Sent: Mon, May 16, 2011 10:52:07 AM
Subject: [Ergatis-users] Pipeline stuck

Dear Ergatis users,

I was out of luck running the prokaryotic genome annotation pipeline annotation
over this week-end because the virtual machine I was running it on (VirtualBox)
crashed.


Now I have the problem that Ergatis thinks the pipeline is still running and if
I try to get the details of the pipeline by clicking on the pipeline ID the
browser loads and loads for 5 minutes but all I get outputted back the the
browser is a comma, yes a "," comma and nothing else. The URL it is trying to
contact is the following:

http://localhost:8080/ergatis/cgi/view_pipeline.cgi?instance=/usr/local/projects/new/workflow/runtime/pipeline/6/pipeline.xml



On the web server side, here is what I get in the access_log file:

Timeout waiting for output from CGI script
/var/www/ergatis/cgi/view_pipeline.cgi, referer:
http://localhost:8080/ergatis/cgi/pipeline_list.cgi?repository_root=/usr/local/projects/new



Any idea what this could be? And how I can restart maybe my pipeline manually?
Or do I need to restart the whole pipeline again? I would really like to be able

to restart it from where it crashed if possible.

http://ergatis:8080/ergatis/cgi/view_pipeline.cgi?instance=/usr/local/projects/dsvnew/workflow/runtime/pipeline/6/pipeline.xml



Many thanks for your feedback.

ML


------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Pipeline stuck

Joshua Orvis
This log file size is a product of how high you have your Workflow Engine logging parameter set to.  Go into your Workflow installation directory and find a file called log4j.properties and edit it.  Find the line like this one:

log4j.rootLogger=WARN#org.tigr.antware.shared.util.Finest, FILE, CON

Notice where it says "WARN" there?  You can set that to different levels, and if yours says "INFO", "DEBUG" or a few others your log output will be pretty high.  If you change it to WARN you should find log files more manageable.

Joshua



On Mon, May 16, 2011 at 6:49 AM, ML mail <[hidden email]> wrote:
I managed to find out that the problem with the pipline viewer not loading was
that the pipeline.xml.log file was 14 GB big. So I simply deleted this file and
rerun the pipeline.

Is it normal that this file gets so huge?

Regards,
ML




----- Original Message ----
From: ML mail <[hidden email]>
To: [hidden email]
Sent: Mon, May 16, 2011 10:52:07 AM
Subject: [Ergatis-users] Pipeline stuck

Dear Ergatis users,

I was out of luck running the prokaryotic genome annotation pipeline annotation
over this week-end because the virtual machine I was running it on (VirtualBox)
crashed.


Now I have the problem that Ergatis thinks the pipeline is still running and if
I try to get the details of the pipeline by clicking on the pipeline ID the
browser loads and loads for 5 minutes but all I get outputted back the the
browser is a comma, yes a "," comma and nothing else. The URL it is trying to
contact is the following:

http://localhost:8080/ergatis/cgi/view_pipeline.cgi?instance=/usr/local/projects/new/workflow/runtime/pipeline/6/pipeline.xml



On the web server side, here is what I get in the access_log file:

Timeout waiting for output from CGI script
/var/www/ergatis/cgi/view_pipeline.cgi, referer:
http://localhost:8080/ergatis/cgi/pipeline_list.cgi?repository_root=/usr/local/projects/new



Any idea what this could be? And how I can restart maybe my pipeline manually?
Or do I need to restart the whole pipeline again? I would really like to be able

to restart it from where it crashed if possible.

http://ergatis:8080/ergatis/cgi/view_pipeline.cgi?instance=/usr/local/projects/dsvnew/workflow/runtime/pipeline/6/pipeline.xml



Many thanks for your feedback.

ML


------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Pipeline stuck

ML mail
Dear Joshua,

Thanks for that tip, I didn't think to check the the TIGR WorkFlow config... So as suggested I now changed DEBUG with WARN...

Regards,
ML



From: Joshua Orvis <[hidden email]>
To: [hidden email]
Sent: Mon, May 16, 2011 1:14:37 PM
Subject: Re: [Ergatis-users] Pipeline stuck

This log file size is a product of how high you have your Workflow Engine logging parameter set to.  Go into your Workflow installation directory and find a file called log4j.properties and edit it.  Find the line like this one:

log4j.rootLogger=WARN#org.tigr.antware.shared.util.Finest, FILE, CON

Notice where it says "WARN" there?  You can set that to different levels, and if yours says "INFO", "DEBUG" or a few others your log output will be pretty high.  If you change it to WARN you should find log files more manageable.

Joshua



On Mon, May 16, 2011 at 6:49 AM, ML mail <[hidden email]> wrote:
I managed to find out that the problem with the pipline viewer not loading was
that the pipeline.xml.log file was 14 GB big. So I simply deleted this file and
rerun the pipeline.

Is it normal that this file gets so huge?

Regards,
ML




----- Original Message ----
From: ML mail <[hidden email]>
To: [hidden email]
Sent: Mon, May 16, 2011 10:52:07 AM
Subject: [Ergatis-users] Pipeline stuck

Dear Ergatis users,

I was out of luck running the prokaryotic genome annotation pipeline annotation
over this week-end because the virtual machine I was running it on (VirtualBox)
crashed.


Now I have the problem that Ergatis thinks the pipeline is still running and if
I try to get the details of the pipeline by clicking on the pipeline ID the
browser loads and loads for 5 minutes but all I get outputted back the the
browser is a comma, yes a "," comma and nothing else. The URL it is trying to
contact is the following:

http://localhost:8080/ergatis/cgi/view_pipeline.cgi?instance=/usr/local/projects/new/workflow/runtime/pipeline/6/pipeline.xml



On the web server side, here is what I get in the access_log file:

Timeout waiting for output from CGI script
/var/www/ergatis/cgi/view_pipeline.cgi, referer:
http://localhost:8080/ergatis/cgi/pipeline_list.cgi?repository_root=/usr/local/projects/new



Any idea what this could be? And how I can restart maybe my pipeline manually?
Or do I need to restart the whole pipeline again? I would really like to be able

to restart it from where it crashed if possible.

http://ergatis:8080/ergatis/cgi/view_pipeline.cgi?instance=/usr/local/projects/dsvnew/workflow/runtime/pipeline/6/pipeline.xml



Many thanks for your feedback.

ML


------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Overlap_analysis.default (prokaryotic annotation pipeline) trying to submit to cluster even though grid enabled = 0

Jason Kwan
Hi Everyone,

I'm currently trying to get the prokaryotic annotation (without loading)
to work in ergatis.  The system is set up on a single machine (Mac) and
I have set grid enabled = 0 in ergatis.ini.  By and large I've been able
to troubleshoot the issues I've come up against, but this one is kind of
weird and I can't really find the reason for it.  When I do a test run
of the prokaryotic pipeline it gets to the overlap_analysis.default
component:

When this command is run:
/opt/ergatis/bin/overlap_analysis
--input_list="/usr/local/projects/lin_actino/output_repository/promote_gene_prediction/115_promote_prediction/promote_gene_prediction.bsml.list"
--input_file=""
--rna_bsml=/usr/local/projects/lin_actino/output_repository/tRNAscan-SE/115_find_tRNA/tRNAscan-SE.bsml.list,/usr/local/projects/lin_actino/output_repository/RNAmmer/115_default/RNAmmer.bsml.list
--evidence_bsml=/usr/local/projects/lin_actino/output_repository/hmmpfam/115_pre_overlap_analysis/hmmpfam.bsml.list,/usr/local/projects/lin_actino/output_repository/ber/115_pre_overlap_analysis/ber.bsml.list
--output_directory=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default
--cutoff=60
--flagged_overlaps_file=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default/flagged_overlaps.txt
--log=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default/overlap_analysis.log


I get this error message:
return value: 1
message: Submission Error with submitting job to clusterCould not parse
jobID from submission output:
/usr/workflow_manager/bin/sge_submitter.sh: line 49: qsub: command not
found /usr/workflow_manager/bin/sge_submitter.sh: line 49: qsub: command
not found

So it looks like it is trying to submit a grid job, even though grid
enabled is set to 0.  I tried deleting all optional grid variables in
ergatis.ini (which I had left as default before) and this did not work.  
I noticed that in the config settings it had this variable:

$;NODISTRIB$; = 1

So if this is set to 1, then it shouldn't be distributed?

Thanks for your help

Jason




------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Overlap_analysis.default (prokaryotic annotation pipeline) trying to submit to cluster even though grid enabled = 0

Chris Hemmerich

Jason,

My guess is that in docs/overlap_analaysis.xml, this command has type
'RunDistributedCommand' rather than 'RunUnixCommand'. You can change the
former to the latter to make it run on your local machine.

  Chris


On Tue, 17 May 2011, Jason Kwan wrote:

> Hi Everyone,
>
> I'm currently trying to get the prokaryotic annotation (without loading)
> to work in ergatis.  The system is set up on a single machine (Mac) and
> I have set grid enabled = 0 in ergatis.ini.  By and large I've been able
> to troubleshoot the issues I've come up against, but this one is kind of
> weird and I can't really find the reason for it.  When I do a test run
> of the prokaryotic pipeline it gets to the overlap_analysis.default
> component:
>
> When this command is run:
> /opt/ergatis/bin/overlap_analysis
> --input_list="/usr/local/projects/lin_actino/output_repository/promote_gene_prediction/115_promote_prediction/promote_gene_prediction.bsml.list"
> --input_file=""
> --rna_bsml=/usr/local/projects/lin_actino/output_repository/tRNAscan-SE/115_find_tRNA/tRNAscan-SE.bsml.list,/usr/local/projects/lin_actino/output_repository/RNAmmer/115_default/RNAmmer.bsml.list
> --evidence_bsml=/usr/local/projects/lin_actino/output_repository/hmmpfam/115_pre_overlap_analysis/hmmpfam.bsml.list,/usr/local/projects/lin_actino/output_repository/ber/115_pre_overlap_analysis/ber.bsml.list
> --output_directory=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default
> --cutoff=60
> --flagged_overlaps_file=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default/flagged_overlaps.txt
> --log=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default/overlap_analysis.log
>
>
> I get this error message:
> return value: 1
> message: Submission Error with submitting job to clusterCould not parse
> jobID from submission output:
> /usr/workflow_manager/bin/sge_submitter.sh: line 49: qsub: command not
> found /usr/workflow_manager/bin/sge_submitter.sh: line 49: qsub: command
> not found
>
> So it looks like it is trying to submit a grid job, even though grid
> enabled is set to 0.  I tried deleting all optional grid variables in
> ergatis.ini (which I had left as default before) and this did not work.
> I noticed that in the config settings it had this variable:
>
> $;NODISTRIB$; = 1
>
> So if this is set to 1, then it shouldn't be distributed?
>
> Thanks for your help
>
> Jason
>
>
>
>
> ------------------------------------------------------------------------------
> Achieve unprecedented app performance and reliability
> What every C/C++ and Fortran developer should know.
> Learn how Intel has extended the reach of its next-generation tools
> to help boost performance applications - inlcuding clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Overlap_analysis.default (prokaryotic annotation pipeline) trying to submit to cluster even though grid enabled = 0

Joshua Orvis
In reply to this post by Jason Kwan
Jason -

I just took a look at this and it looks like this component was hard-coded to have a distributed step.  Look in the overlap_analysis.xml file and find the commandSet of type "remote-serial" and change it to just be "serial".  You'll need to do this in your existing pipeline if you want to fix it and re-run, then in the component XML file so that future pipelines will also be OK.  There was a bug filed against this recently, so there's no need to create another.

Joshua




On Tue, May 17, 2011 at 9:10 AM, Jason Kwan <[hidden email]> wrote:
Hi Everyone,

I'm currently trying to get the prokaryotic annotation (without loading)
to work in ergatis.  The system is set up on a single machine (Mac) and
I have set grid enabled = 0 in ergatis.ini.  By and large I've been able
to troubleshoot the issues I've come up against, but this one is kind of
weird and I can't really find the reason for it.  When I do a test run
of the prokaryotic pipeline it gets to the overlap_analysis.default
component:

When this command is run:
/opt/ergatis/bin/overlap_analysis
--input_list="/usr/local/projects/lin_actino/output_repository/promote_gene_prediction/115_promote_prediction/promote_gene_prediction.bsml.list"
--input_file=""
--rna_bsml=/usr/local/projects/lin_actino/output_repository/tRNAscan-SE/115_find_tRNA/tRNAscan-SE.bsml.list,/usr/local/projects/lin_actino/output_repository/RNAmmer/115_default/RNAmmer.bsml.list
--evidence_bsml=/usr/local/projects/lin_actino/output_repository/hmmpfam/115_pre_overlap_analysis/hmmpfam.bsml.list,/usr/local/projects/lin_actino/output_repository/ber/115_pre_overlap_analysis/ber.bsml.list
--output_directory=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default
--cutoff=60
--flagged_overlaps_file=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default/flagged_overlaps.txt
--log=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default/overlap_analysis.log


I get this error message:
return value: 1
message: Submission Error with submitting job to clusterCould not parse
jobID from submission output:
/usr/workflow_manager/bin/sge_submitter.sh: line 49: qsub: command not
found /usr/workflow_manager/bin/sge_submitter.sh: line 49: qsub: command
not found

So it looks like it is trying to submit a grid job, even though grid
enabled is set to 0.  I tried deleting all optional grid variables in
ergatis.ini (which I had left as default before) and this did not work.
I noticed that in the config settings it had this variable:

$;NODISTRIB$; = 1

So if this is set to 1, then it shouldn't be distributed?

Thanks for your help

Jason




------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Overlap_analysis.default (prokaryotic annotation pipeline) trying to submit to cluster even though grid enabled = 0

Jason Kwan
Thanks to Joshua, Chris and Kevin for their help, which provided the solution!  I found one instance of RunDistributedCommand in overlap_analysis.xml, there were no instances of remote-serial. 

Thanks again

Jason

On 5/17/11 8:21 AM, Joshua Orvis wrote:
Jason -

I just took a look at this and it looks like this component was hard-coded to have a distributed step.  Look in the overlap_analysis.xml file and find the commandSet of type "remote-serial" and change it to just be "serial".  You'll need to do this in your existing pipeline if you want to fix it and re-run, then in the component XML file so that future pipelines will also be OK.  There was a bug filed against this recently, so there's no need to create another.

Joshua




On Tue, May 17, 2011 at 9:10 AM, Jason Kwan <[hidden email]> wrote:
Hi Everyone,

I'm currently trying to get the prokaryotic annotation (without loading)
to work in ergatis.  The system is set up on a single machine (Mac) and
I have set grid enabled = 0 in ergatis.ini.  By and large I've been able
to troubleshoot the issues I've come up against, but this one is kind of
weird and I can't really find the reason for it.  When I do a test run
of the prokaryotic pipeline it gets to the overlap_analysis.default
component:

When this command is run:
/opt/ergatis/bin/overlap_analysis
--input_list="/usr/local/projects/lin_actino/output_repository/promote_gene_prediction/115_promote_prediction/promote_gene_prediction.bsml.list"
--input_file=""
--rna_bsml=/usr/local/projects/lin_actino/output_repository/tRNAscan-SE/115_find_tRNA/tRNAscan-SE.bsml.list,/usr/local/projects/lin_actino/output_repository/RNAmmer/115_default/RNAmmer.bsml.list
--evidence_bsml=/usr/local/projects/lin_actino/output_repository/hmmpfam/115_pre_overlap_analysis/hmmpfam.bsml.list,/usr/local/projects/lin_actino/output_repository/ber/115_pre_overlap_analysis/ber.bsml.list
--output_directory=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default
--cutoff=60
--flagged_overlaps_file=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default/flagged_overlaps.txt
--log=/usr/local/projects/lin_actino/output_repository/overlap_analysis/115_default/overlap_analysis.log


I get this error message:
return value: 1
message: Submission Error with submitting job to clusterCould not parse
jobID from submission output:
/usr/workflow_manager/bin/sge_submitter.sh: line 49: qsub: command not
found /usr/workflow_manager/bin/sge_submitter.sh: line 49: qsub: command
not found

So it looks like it is trying to submit a grid job, even though grid
enabled is set to 0.  I tried deleting all optional grid variables in
ergatis.ini (which I had left as default before) and this did not work.
I noticed that in the config settings it had this variable:

$;NODISTRIB$; = 1

So if this is set to 1, then it shouldn't be distributed?

Thanks for your help

Jason




------------------------------------------------------------------------------
Achieve unprecedented app performance and reliability
What every C/C++ and Fortran developer should know.
Learn how Intel has extended the reach of its next-generation tools
to help boost performance applications - inlcuding clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

ps_scan (prokaryotic pipeline) not being passed psa2msa argument

Jason Kwan
Hi Everyone,

I'm doing test runs of the prokaryotic annotation pipeline currently.  When it gets to the ps_scan component I get the error message:

return value: 1
message: Error executing script exit value: 1Option psa2msa requires an argument

...because the command does not include anything for the psa2msa argument:

/opt/ps_scan/ps_scan.pl -s -d '/opt/db/prosite.dat' /usr/local/projects/lin_actino/output_repository/translate_sequence/123_final_polypeptides/i1/g5/NODE_5_length_7187_cov_357.polypeptide.284042.1.fsa --pfscan /opt/ps_scan/pfscan --psa2msa

However, the ps_scan paths are all present and correct in the config section of this component:

[component ps_scan]

  • $;PS_SCAN_EXEC$; = /opt/ps_scan/ps_scan.pl
  • $;PROSITE_DAT_PATH$; = /opt/db/prosite.dat
  • $;PFSCAN_EXEC$; = /opt/ps_scan/pfscan
  • $;PSA2MSA$; = /opt/ps_scan/psa2msa
Can anyone help me find exactly where the $;PSA2MSA$; variable is not being passed on?

Thanks again for your help

Jason Kwan


------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: ps_scan (prokaryotic pipeline) not being passed psa2msa argument

Joshua Orvis
Jason -

It looks like there was a bug in the software.config file that has already been fixed, but you got it before the fix was made.  In your software.config file you should have this section for ps_scan (it gets imported into your component config at runtime):

[component ps_scan]
$;PS_SCAN_EXEC$;=/usr/local/bin/ps_scan.pl
$;PROSITE_DAT_PATH$;=/usr/local/projects/db/prosite/prosite.dat
$;PFSCAN_EXEC$;=/usr/local/bin/pfscan
$;PSA2MSA_EXEC$;=/usr/local/bin/psa2msa

In your version it looks like the last variable was $;PSA2MSA$; rather than $;PSA2MSA_EXEC$;  This made the value blank, since in the component's ps_scan.i1.xml file it has a line like this:

<arg>$;OTHER_OPTS$; -d '$;PROSITE_DAT$;' $;I_FILE_PATH$; --pfscan $;PFSCAN_EXEC$; --psa2msa $;PSA2MSA_EXEC$;</arg>

Notice at the end there that the variable it was looking for was $;PSA2MSA_EXEC$;.  Make sure these all agree, and you should be set.

Joshua



2011/5/18 Jason Kwan <[hidden email]>
Hi Everyone,

I'm doing test runs of the prokaryotic annotation pipeline currently.  When it gets to the ps_scan component I get the error message:

return value: 1
message: Error executing script exit value: 1Option psa2msa requires an argument

...because the command does not include anything for the psa2msa argument:

/opt/ps_scan/ps_scan.pl -s -d '/opt/db/prosite.dat' /usr/local/projects/lin_actino/output_repository/translate_sequence/123_final_polypeptides/i1/g5/NODE_5_length_7187_cov_357.polypeptide.284042.1.fsa --pfscan /opt/ps_scan/pfscan --psa2msa

However, the ps_scan paths are all present and correct in the config section of this component:

[component ps_scan]

  • $;PS_SCAN_EXEC$; = /opt/ps_scan/ps_scan.pl
  • $;PROSITE_DAT_PATH$; = /opt/db/prosite.dat
  • $;PFSCAN_EXEC$; = /opt/ps_scan/pfscan
  • $;PSA2MSA$; = /opt/ps_scan/psa2msa
Can anyone help me find exactly where the $;PSA2MSA$; variable is not being passed on?

Thanks again for your help

Jason Kwan


------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

problems with parse_evidence components in prokaryotic annotation pipeline

Jason Kwan
Hi Everyone,

Hopefully this is almost it as far as problems with this pipeline go.  I have problems with the parse_evidence.hmmpfam_pre and parse_evidence.ber_pre components.

For parse_evidence.hmmpfam_pre, it wants to know the tigr_roles_db_dir.  I thought this might be a directory containing roles_data.db that I got from the manatee distribution, but I don't think this is it.  It is looking for tigr_roles_db_dir/hmm/PFAM_role_id.txt, which I don't have.  I followed Aaron's instructions from his message on 5/13/11, Aaron: was there something you also had to do with this module to illiminate use of TIGR roles? 

For parse_evidence.ber_pre, initially there was an error saying that the username and password arguments were required.  Thinking this would be for a database that it was going to create, I just set username=test and password=test.  I now get this error:

return value: 2
message: Error executing script exit value: 2Could not create parser object (PFunc::EvidenceParser::BERUniprotParser). [DBI connect('host=jabba;packetSize=8092','test',...) failed: Unknown MySQL server host 'jabba' (1) at /opt/ergatis/lib/perl5//UnirefClusters/Database.pm line 241 ] at /opt/ergatis/bin/parse_evidence.pl line 197.

So is this trying to connect to an existing mysql database that I should have set up on localhost?

Incidently, I found an error in the template for this pipeline.  In my version the section for priam_ec_assignment.default reads:

                <commandSet>
                    <state>incomplete</state>
                    <name>priam_ec_assignment.default</name>
                </commandSet>

...and I changed the first line to <commandSet type="serial"> to get the component to show up.

Thanks to everyone for the help so far.  Hopefully these should be that last things for this pipeline as I'm quite near the end. 

Jason Kwan

On 5/18/11 7:42 AM, Joshua Orvis wrote:
Jason -

It looks like there was a bug in the software.config file that has already been fixed, but you got it before the fix was made.  In your software.config file you should have this section for ps_scan (it gets imported into your component config at runtime):

[component ps_scan]
$;PS_SCAN_EXEC$;=/usr/local/bin/ps_scan.pl
$;PROSITE_DAT_PATH$;=/usr/local/projects/db/prosite/prosite.dat
$;PFSCAN_EXEC$;=/usr/local/bin/pfscan
$;PSA2MSA_EXEC$;=/usr/local/bin/psa2msa

In your version it looks like the last variable was $;PSA2MSA$; rather than $;PSA2MSA_EXEC$;  This made the value blank, since in the component's ps_scan.i1.xml file it has a line like this:

<arg>$;OTHER_OPTS$; -d '$;PROSITE_DAT$;' $;I_FILE_PATH$; --pfscan $;PFSCAN_EXEC$; --psa2msa $;PSA2MSA_EXEC$;</arg>

Notice at the end there that the variable it was looking for was $;PSA2MSA_EXEC$;.  Make sure these all agree, and you should be set.

Joshua



2011/5/18 Jason Kwan <[hidden email]>
Hi Everyone,

I'm doing test runs of the prokaryotic annotation pipeline currently.  When it gets to the ps_scan component I get the error message:

return value: 1
message: Error executing script exit value: 1Option psa2msa requires an argument

...because the command does not include anything for the psa2msa argument:

/opt/ps_scan/ps_scan.pl -s -d '/opt/db/prosite.dat' /usr/local/projects/lin_actino/output_repository/translate_sequence/123_final_polypeptides/i1/g5/NODE_5_length_7187_cov_357.polypeptide.284042.1.fsa --pfscan /opt/ps_scan/pfscan --psa2msa

However, the ps_scan paths are all present and correct in the config section of this component:

[component ps_scan]

  • $;PS_SCAN_EXEC$; = /opt/ps_scan/ps_scan.pl
  • $;PROSITE_DAT_PATH$; = /opt/db/prosite.dat
  • $;PFSCAN_EXEC$; = /opt/ps_scan/pfscan
  • $;PSA2MSA$; = /opt/ps_scan/psa2msa
Can anyone help me find exactly where the $;PSA2MSA$; variable is not being passed on?

Thanks again for your help

Jason Kwan


------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: problems with parse_evidence components in prokaryotic annotation pipeline

Chris Hemmerich

Jason,

>From looking through our install notes for this pipeline, try grabbing the
following library files from trunk in the Ergatis subversion repository:

/lib/

   NestedDB.pm
   UnirefAnnotation/DBUtil.pm
   UnirefAnnotation/Database.pm
   UnirefAnnotation/Lookup.pm
   UnirefAnnotation/NestedDB.pm
   PFunc/EvidenceParser/BERUniprotParser.pm

Cheers,
  Chris

On Wed, 18 May 2011, Jason Kwan wrote:

> Hi Everyone,
>
> Hopefully this is almost it as far as problems with this pipeline go.  I have
> problems with the parse_evidence.hmmpfam_pre and parse_evidence.ber_pre
> components.
>
> For parse_evidence.hmmpfam_pre, it wants to know the tigr_roles_db_dir.  I
> thought this might be a directory containing roles_data.db that I got from
> the manatee distribution, but I don't think this is it.  It is looking for
> tigr_roles_db_dir/hmm/PFAM_role_id.txt, which I don't have.  I followed
> Aaron's instructions from his message on 5/13/11, Aaron: was there something
> you also had to do with this module to illiminate use of TIGR roles?
>
> For parse_evidence.ber_pre, initially there was an error saying that the
> username and password arguments were required.  Thinking this would be for a
> database that it was going to create, I just set username=test and
> password=test.  I now get this error:
>
> return value: 2
> message: Error executing script exit value: 2Could not create parser object
> (PFunc::EvidenceParser::BERUniprotParser). [DBI
> connect('host=jabba;packetSize=8092','test',...) failed: Unknown MySQL server
> host 'jabba' (1) at /opt/ergatis/lib/perl5//UnirefClusters/Database.pm line
> 241 ] at /opt/ergatis/bin/parse_evidence.pl line 197.
>
> So is this trying to connect to an existing mysql database that I should have
> set up on localhost?
>
> Incidently, I found an error in the template for this pipeline.  In my
> version the section for priam_ec_assignment.default reads:
>
> <commandSet>
> <state>incomplete</state>
> <name>priam_ec_assignment.default</name>
> </commandSet>
>
> ...and I changed the first line to <commandSet type="serial"> to get the
> component to show up.
>
> Thanks to everyone for the help so far.  Hopefully these should be that last
> things for this pipeline as I'm quite near the end.
>
> Jason Kwan
>
> On 5/18/11 7:42 AM, Joshua Orvis wrote:
>>  Jason -
>>
>>  It looks like there was a bug in the software.config file that has already
>>  been fixed, but you got it before the fix was made.  In your
>>  software.config file you should have this section for ps_scan (it gets
>>  imported into your component config at runtime):
>>
>>  [component ps_scan]
>>  $;PS_SCAN_EXEC$;=/usr/local/bin/ps_scan.pl <http://ps_scan.pl>
>>  $;PROSITE_DAT_PATH$;=/usr/local/projects/db/prosite/prosite.dat
>>  $;PFSCAN_EXEC$;=/usr/local/bin/pfscan
>>  $;PSA2MSA_EXEC$;=/usr/local/bin/psa2msa
>>
>>  In your version it looks like the last variable was $;PSA2MSA$; rather
>>  than $;PSA2MSA_EXEC$;  This made the value blank, since in the component's
>>  ps_scan.i1.xml file it has a line like this:
>>
>>  <arg>$;OTHER_OPTS$; -d '$;PROSITE_DAT$;' $;I_FILE_PATH$; --pfscan
>>  $;PFSCAN_EXEC$; --psa2msa $;PSA2MSA_EXEC$;</arg>
>>
>>  Notice at the end there that the variable it was looking for was
>>  $;PSA2MSA_EXEC$;.  Make sure these all agree, and you should be set.
>>
>>  Joshua
>>
>>
>>
>>  2011/5/18 Jason Kwan <[hidden email] <mailto:[hidden email]>>
>>
>>      Hi Everyone,
>>
>>      I'm doing test runs of the prokaryotic annotation pipeline
>>      currently.  When it gets to the ps_scan component I get the error
>>      message:
>>
>>      return value: 1
>>      message: Error executing script exit value: 1Option psa2msa
>>      requires an argument
>>
>>      ...because the command does not include anything for the psa2msa
>>      argument:
>>
>>      /opt/ps_scan/ps_scan.pl <http://ps_scan.pl> -s -d
>>      '/opt/db/prosite.dat'
>>      /usr/local/projects/lin_actino/output_repository/translate_sequence/123_final_polypeptides/i1/g5/NODE_5_length_7187_cov_357.polypeptide.284042.1.fsa
>>      --pfscan /opt/ps_scan/pfscan --psa2msa
>>
>>      However, the ps_scan paths are all present and correct in the
>>      config section of this component:
>>
>>
>>            [component ps_scan]
>>
>>          * $;PS_SCAN_EXEC$; = /opt/ps_scan/ps_scan.pl <http://ps_scan.pl>
>>          * $;PROSITE_DAT_PATH$; = /opt/db/prosite.dat
>>          * $;PFSCAN_EXEC$; = /opt/ps_scan/pfscan
>>          * $;PSA2MSA$; = /opt/ps_scan/psa2msa
>>
>>      Can anyone help me find exactly where the $;PSA2MSA$; variable is
>>      not being passed on?
>>
>>      Thanks again for your help
>>
>>      Jason Kwan
>>
>>
>>      ------------------------------------------------------------------------------
>>      What Every C/C++ and Fortran developer Should Know!
>>      Read this article and learn how Intel has extended the reach of its
>>      next-generation tools to help Windows* and Linux* C/C++ and Fortran
>>      developers boost performance applications - including clusters.
>>      http://p.sf.net/sfu/intel-dev2devmay
>>      _______________________________________________
>>      Ergatis-users mailing list
>>      [hidden email]
>>      <mailto:[hidden email]>
>>      https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>>
>
>
------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: problems with parse_evidence components in prokaryotic annotation pipeline

Jason Kwan
Hi Chris,

So I checked out those files with svn and copied them to the
/opt/ergatis/lib/perl5 directory (keeping the directory structure
intact).  I got a similar message about the host "jabba", so I changed
this to localhost and put in my MySql username and password in the
config for the parse_evidence.ber_pre component.  I now get this error:

return value: 255
message: Error executing script exit value: 255Subroutine new redefined
at /opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm line 8.
Subroutine get_annotation redefined at
/opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm line 16. Subroutine
is_trusted redefined at
/opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm line 24. Subroutine
_init redefined at /opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm
line 31. DBD::mysql::db do failed: Unknown database 'uniprot_annotation'
at /opt/ergatis/lib/perl5//UnirefAnnotation/DBUtil.pm line 280. Could
not create parser object (PFunc::EvidenceParser::BERUniprotParser).
[Could not create object type: UnirefAnnotation::Database
[DBD::mysql::db do failed: Unknown database 'uniprot_annotation' at
/opt/ergatis/lib/perl5//UnirefAnnotation/DBUtil.pm line 280. ] ] at
/opt/ergatis/bin/parse_evidence.pl line 197.

Do I have to first create the uniprot_annotation database in mysql?

...meanwhile I still get the same error for the parse_evidence.hmmpfam_pre:

return value: 2
message: Error executing script exit value: 2MLDBM error: Second level
tie failed, "No such file or directory" at
/opt/ergatis/lib/perl5//TIGR/Roles/HMM/PfamToRoleLookup.pm line 59 Could
not create parser object (PFunc::EvidenceParser::HMMParser). [Could not
open /opt/db/TIGRroles/hmm/PFAM_role_id.txt (No such file or directory)
at /opt/ergatis/lib/perl5//TIGR/Roles/HMM/PfamToRoleLookup.pm line 78. ]
at /opt/ergatis/bin/parse_evidence.pl line 197.

Is there any way to either get the TIGR roles lookup files it is looking
for or to set it up so it doesn't use them?

Thanks again for your help.

Jason


On 5/19/11 7:26 AM, Chris Hemmerich wrote:

> Jason,
>
>  From looking through our install notes for this pipeline, try grabbing the
> following library files from trunk in the Ergatis subversion repository:
>
> /lib/
>
>     NestedDB.pm
>     UnirefAnnotation/DBUtil.pm
>     UnirefAnnotation/Database.pm
>     UnirefAnnotation/Lookup.pm
>     UnirefAnnotation/NestedDB.pm
>     PFunc/EvidenceParser/BERUniprotParser.pm
>
> Cheers,
>    Chris
>
> On Wed, 18 May 2011, Jason Kwan wrote:
>
>> Hi Everyone,
>>
>> Hopefully this is almost it as far as problems with this pipeline go.  I have
>> problems with the parse_evidence.hmmpfam_pre and parse_evidence.ber_pre
>> components.
>>
>> For parse_evidence.hmmpfam_pre, it wants to know the tigr_roles_db_dir.  I
>> thought this might be a directory containing roles_data.db that I got from
>> the manatee distribution, but I don't think this is it.  It is looking for
>> tigr_roles_db_dir/hmm/PFAM_role_id.txt, which I don't have.  I followed
>> Aaron's instructions from his message on 5/13/11, Aaron: was there something
>> you also had to do with this module to illiminate use of TIGR roles?
>>
>> For parse_evidence.ber_pre, initially there was an error saying that the
>> username and password arguments were required.  Thinking this would be for a
>> database that it was going to create, I just set username=test and
>> password=test.  I now get this error:
>>
>> return value: 2
>> message: Error executing script exit value: 2Could not create parser object
>> (PFunc::EvidenceParser::BERUniprotParser). [DBI
>> connect('host=jabba;packetSize=8092','test',...) failed: Unknown MySQL server
>> host 'jabba' (1) at /opt/ergatis/lib/perl5//UnirefClusters/Database.pm line
>> 241 ] at /opt/ergatis/bin/parse_evidence.pl line 197.
>>
>> So is this trying to connect to an existing mysql database that I should have
>> set up on localhost?
>>
>> Incidently, I found an error in the template for this pipeline.  In my
>> version the section for priam_ec_assignment.default reads:
>>
>> <commandSet>
>> <state>incomplete</state>
>> <name>priam_ec_assignment.default</name>
>> </commandSet>
>>
>> ...and I changed the first line to<commandSet type="serial">  to get the
>> component to show up.
>>
>> Thanks to everyone for the help so far.  Hopefully these should be that last
>> things for this pipeline as I'm quite near the end.
>>
>> Jason Kwan
>>
>> On 5/18/11 7:42 AM, Joshua Orvis wrote:
>>>   Jason -
>>>
>>>   It looks like there was a bug in the software.config file that has already
>>>   been fixed, but you got it before the fix was made.  In your
>>>   software.config file you should have this section for ps_scan (it gets
>>>   imported into your component config at runtime):
>>>
>>>   [component ps_scan]
>>>   $;PS_SCAN_EXEC$;=/usr/local/bin/ps_scan.pl<http://ps_scan.pl>
>>>   $;PROSITE_DAT_PATH$;=/usr/local/projects/db/prosite/prosite.dat
>>>   $;PFSCAN_EXEC$;=/usr/local/bin/pfscan
>>>   $;PSA2MSA_EXEC$;=/usr/local/bin/psa2msa
>>>
>>>   In your version it looks like the last variable was $;PSA2MSA$; rather
>>>   than $;PSA2MSA_EXEC$;  This made the value blank, since in the component's
>>>   ps_scan.i1.xml file it has a line like this:
>>>
>>>   <arg>$;OTHER_OPTS$; -d '$;PROSITE_DAT$;' $;I_FILE_PATH$; --pfscan
>>>   $;PFSCAN_EXEC$; --psa2msa $;PSA2MSA_EXEC$;</arg>
>>>
>>>   Notice at the end there that the variable it was looking for was
>>>   $;PSA2MSA_EXEC$;.  Make sure these all agree, and you should be set.
>>>
>>>   Joshua
>>>
>>>
>>>
>>>   2011/5/18 Jason Kwan<[hidden email]<mailto:[hidden email]>>
>>>
>>>       Hi Everyone,
>>>
>>>       I'm doing test runs of the prokaryotic annotation pipeline
>>>       currently.  When it gets to the ps_scan component I get the error
>>>       message:
>>>
>>>       return value: 1
>>>       message: Error executing script exit value: 1Option psa2msa
>>>       requires an argument
>>>
>>>       ...because the command does not include anything for the psa2msa
>>>       argument:
>>>
>>>       /opt/ps_scan/ps_scan.pl<http://ps_scan.pl>  -s -d
>>>       '/opt/db/prosite.dat'
>>>       /usr/local/projects/lin_actino/output_repository/translate_sequence/123_final_polypeptides/i1/g5/NODE_5_length_7187_cov_357.polypeptide.284042.1.fsa
>>>       --pfscan /opt/ps_scan/pfscan --psa2msa
>>>
>>>       However, the ps_scan paths are all present and correct in the
>>>       config section of this component:
>>>
>>>
>>>             [component ps_scan]
>>>
>>>           * $;PS_SCAN_EXEC$; = /opt/ps_scan/ps_scan.pl<http://ps_scan.pl>
>>>           * $;PROSITE_DAT_PATH$; = /opt/db/prosite.dat
>>>           * $;PFSCAN_EXEC$; = /opt/ps_scan/pfscan
>>>           * $;PSA2MSA$; = /opt/ps_scan/psa2msa
>>>
>>>       Can anyone help me find exactly where the $;PSA2MSA$; variable is
>>>       not being passed on?
>>>
>>>       Thanks again for your help
>>>
>>>       Jason Kwan
>>>
>>>
>>>       ------------------------------------------------------------------------------
>>>       What Every C/C++ and Fortran developer Should Know!
>>>       Read this article and learn how Intel has extended the reach of its
>>>       next-generation tools to help Windows* and Linux* C/C++ and Fortran
>>>       developers boost performance applications - including clusters.
>>>       http://p.sf.net/sfu/intel-dev2devmay
>>>       _______________________________________________
>>>       Ergatis-users mailing list
>>>       [hidden email]
>>>       <mailto:[hidden email]>
>>>       https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
>>>
> >


------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: problems with parse_evidence components in prokaryotic annotation pipeline

Kevin Galens
Hi Jason,

Which version of ergatis are you using? I'm currently putting together a
more detailed README and set of datasets required for the newest version
of the prok pipeline (from ergatis-v2r14b4 available on the sourceforge
website.)

I'll send these your way when they are ready.

Kevin

On 5/19/11 11:36 AM, "Jason Kwan" <[hidden email]> wrote:

>Hi Chris,
>
>So I checked out those files with svn and copied them to the
>/opt/ergatis/lib/perl5 directory (keeping the directory structure
>intact).  I got a similar message about the host "jabba", so I changed
>this to localhost and put in my MySql username and password in the
>config for the parse_evidence.ber_pre component.  I now get this error:
>
>return value: 255
>message: Error executing script exit value: 255Subroutine new redefined
>at /opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm line 8.
>Subroutine get_annotation redefined at
>/opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm line 16. Subroutine
>is_trusted redefined at
>/opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm line 24. Subroutine
>_init redefined at /opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm
>line 31. DBD::mysql::db do failed: Unknown database 'uniprot_annotation'
>at /opt/ergatis/lib/perl5//UnirefAnnotation/DBUtil.pm line 280. Could
>not create parser object (PFunc::EvidenceParser::BERUniprotParser).
>[Could not create object type: UnirefAnnotation::Database
>[DBD::mysql::db do failed: Unknown database 'uniprot_annotation' at
>/opt/ergatis/lib/perl5//UnirefAnnotation/DBUtil.pm line 280. ] ] at
>/opt/ergatis/bin/parse_evidence.pl line 197.
>
>Do I have to first create the uniprot_annotation database in mysql?
>
>...meanwhile I still get the same error for the
>parse_evidence.hmmpfam_pre:
>
>return value: 2
>message: Error executing script exit value: 2MLDBM error: Second level
>tie failed, "No such file or directory" at
>/opt/ergatis/lib/perl5//TIGR/Roles/HMM/PfamToRoleLookup.pm line 59 Could
>not create parser object (PFunc::EvidenceParser::HMMParser). [Could not
>open /opt/db/TIGRroles/hmm/PFAM_role_id.txt (No such file or directory)
>at /opt/ergatis/lib/perl5//TIGR/Roles/HMM/PfamToRoleLookup.pm line 78. ]
>at /opt/ergatis/bin/parse_evidence.pl line 197.
>
>Is there any way to either get the TIGR roles lookup files it is looking
>for or to set it up so it doesn't use them?
>
>Thanks again for your help.
>
>Jason
>
>
>On 5/19/11 7:26 AM, Chris Hemmerich wrote:
>> Jason,
>>
>>  From looking through our install notes for this pipeline, try grabbing
>>the
>> following library files from trunk in the Ergatis subversion repository:
>>
>> /lib/
>>
>>     NestedDB.pm
>>     UnirefAnnotation/DBUtil.pm
>>     UnirefAnnotation/Database.pm
>>     UnirefAnnotation/Lookup.pm
>>     UnirefAnnotation/NestedDB.pm
>>     PFunc/EvidenceParser/BERUniprotParser.pm
>>
>> Cheers,
>>    Chris
>>
>> On Wed, 18 May 2011, Jason Kwan wrote:
>>
>>> Hi Everyone,
>>>
>>> Hopefully this is almost it as far as problems with this pipeline go.
>>>I have
>>> problems with the parse_evidence.hmmpfam_pre and parse_evidence.ber_pre
>>> components.
>>>
>>> For parse_evidence.hmmpfam_pre, it wants to know the
>>>tigr_roles_db_dir.  I
>>> thought this might be a directory containing roles_data.db that I got
>>>from
>>> the manatee distribution, but I don't think this is it.  It is looking
>>>for
>>> tigr_roles_db_dir/hmm/PFAM_role_id.txt, which I don't have.  I followed
>>> Aaron's instructions from his message on 5/13/11, Aaron: was there
>>>something
>>> you also had to do with this module to illiminate use of TIGR roles?
>>>
>>> For parse_evidence.ber_pre, initially there was an error saying that
>>>the
>>> username and password arguments were required.  Thinking this would be
>>>for a
>>> database that it was going to create, I just set username=test and
>>> password=test.  I now get this error:
>>>
>>> return value: 2
>>> message: Error executing script exit value: 2Could not create parser
>>>object
>>> (PFunc::EvidenceParser::BERUniprotParser). [DBI
>>> connect('host=jabba;packetSize=8092','test',...) failed: Unknown MySQL
>>>server
>>> host 'jabba' (1) at /opt/ergatis/lib/perl5//UnirefClusters/Database.pm
>>>line
>>> 241 ] at /opt/ergatis/bin/parse_evidence.pl line 197.
>>>
>>> So is this trying to connect to an existing mysql database that I
>>>should have
>>> set up on localhost?
>>>
>>> Incidently, I found an error in the template for this pipeline.  In my
>>> version the section for priam_ec_assignment.default reads:
>>>
>>> <commandSet>
>>> <state>incomplete</state>
>>> <name>priam_ec_assignment.default</name>
>>> </commandSet>
>>>
>>> ...and I changed the first line to<commandSet type="serial">  to get
>>>the
>>> component to show up.
>>>
>>> Thanks to everyone for the help so far.  Hopefully these should be
>>>that last
>>> things for this pipeline as I'm quite near the end.
>>>
>>> Jason Kwan
>>>
>>> On 5/18/11 7:42 AM, Joshua Orvis wrote:
>>>>   Jason -
>>>>
>>>>   It looks like there was a bug in the software.config file that has
>>>>already
>>>>   been fixed, but you got it before the fix was made.  In your
>>>>   software.config file you should have this section for ps_scan (it
>>>>gets
>>>>   imported into your component config at runtime):
>>>>
>>>>   [component ps_scan]
>>>>   $;PS_SCAN_EXEC$;=/usr/local/bin/ps_scan.pl<http://ps_scan.pl>
>>>>   $;PROSITE_DAT_PATH$;=/usr/local/projects/db/prosite/prosite.dat
>>>>   $;PFSCAN_EXEC$;=/usr/local/bin/pfscan
>>>>   $;PSA2MSA_EXEC$;=/usr/local/bin/psa2msa
>>>>
>>>>   In your version it looks like the last variable was $;PSA2MSA$;
>>>>rather
>>>>   than $;PSA2MSA_EXEC$;  This made the value blank, since in the
>>>>component's
>>>>   ps_scan.i1.xml file it has a line like this:
>>>>
>>>>   <arg>$;OTHER_OPTS$; -d '$;PROSITE_DAT$;' $;I_FILE_PATH$; --pfscan
>>>>   $;PFSCAN_EXEC$; --psa2msa $;PSA2MSA_EXEC$;</arg>
>>>>
>>>>   Notice at the end there that the variable it was looking for was
>>>>   $;PSA2MSA_EXEC$;.  Make sure these all agree, and you should be set.
>>>>
>>>>   Joshua
>>>>
>>>>
>>>>
>>>>   2011/5/18 Jason Kwan<[hidden email]<mailto:[hidden email]>>
>>>>
>>>>       Hi Everyone,
>>>>
>>>>       I'm doing test runs of the prokaryotic annotation pipeline
>>>>       currently.  When it gets to the ps_scan component I get the
>>>>error
>>>>       message:
>>>>
>>>>       return value: 1
>>>>       message: Error executing script exit value: 1Option psa2msa
>>>>       requires an argument
>>>>
>>>>       ...because the command does not include anything for the psa2msa
>>>>       argument:
>>>>
>>>>       /opt/ps_scan/ps_scan.pl<http://ps_scan.pl>  -s -d
>>>>       '/opt/db/prosite.dat'
>>>>      
>>>>/usr/local/projects/lin_actino/output_repository/translate_sequence/123
>>>>_final_polypeptides/i1/g5/NODE_5_length_7187_cov_357.polypeptide.284042
>>>>.1.fsa
>>>>       --pfscan /opt/ps_scan/pfscan --psa2msa
>>>>
>>>>       However, the ps_scan paths are all present and correct in the
>>>>       config section of this component:
>>>>
>>>>
>>>>             [component ps_scan]
>>>>
>>>>           * $;PS_SCAN_EXEC$; =
>>>>/opt/ps_scan/ps_scan.pl<http://ps_scan.pl>
>>>>           * $;PROSITE_DAT_PATH$; = /opt/db/prosite.dat
>>>>           * $;PFSCAN_EXEC$; = /opt/ps_scan/pfscan
>>>>           * $;PSA2MSA$; = /opt/ps_scan/psa2msa
>>>>
>>>>       Can anyone help me find exactly where the $;PSA2MSA$; variable
>>>>is
>>>>       not being passed on?
>>>>
>>>>       Thanks again for your help
>>>>
>>>>       Jason Kwan
>>>>
>>>>
>>>>      
>>>>-----------------------------------------------------------------------
>>>>-------
>>>>       What Every C/C++ and Fortran developer Should Know!
>>>>       Read this article and learn how Intel has extended the reach of
>>>>its
>>>>       next-generation tools to help Windows* and Linux* C/C++ and
>>>>Fortran
>>>>       developers boost performance applications - including clusters.
>>>>       http://p.sf.net/sfu/intel-dev2devmay
>>>>       _______________________________________________
>>>>       Ergatis-users mailing list
>>>>       [hidden email]
>>>>       <mailto:[hidden email]>
>>>>       https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>>
>>>>
>> >
>
>
>--------------------------------------------------------------------------
>----
>What Every C/C++ and Fortran developer Should Know!
>Read this article and learn how Intel has extended the reach of its
>next-generation tools to help Windows* and Linux* C/C++ and Fortran
>developers boost performance applications - including clusters.
>http://p.sf.net/sfu/intel-dev2devmay
>_______________________________________________
>Ergatis-users mailing list
>[hidden email]
>https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: problems with parse_evidence components in prokaryotic annotation pipeline

Jason Kwan
Hi Kevin,

It is v2r13b2.  By the way, I have been in discussion with Todd Creasy about the lookup files for Manatee - apparently there is binary incompatability between Mac and linux in this case so we may run into that here too.  I think Jaysheel Bhavsar is making some (manatee) lookup files that are mac compatible.  For this pipeline, if you send me a method for making the files, I don't mind constructing them myself as I realize Mac OS X is a little-used platform for ergatis.  I would of course send the binaries to you for general distribution.

Thanks for your help

Jason.

On 5/19/11 11:10 AM, Kevin Galens wrote:
Hi Jason,

Which version of ergatis are you using? I'm currently putting together a
more detailed README and set of datasets required for the newest version
of the prok pipeline (from ergatis-v2r14b4 available on the sourceforge
website.)

I'll send these your way when they are ready.

Kevin

On 5/19/11 11:36 AM, "Jason Kwan" [hidden email] wrote:

Hi Chris,

So I checked out those files with svn and copied them to the
/opt/ergatis/lib/perl5 directory (keeping the directory structure
intact).  I got a similar message about the host "jabba", so I changed
this to localhost and put in my MySql username and password in the
config for the parse_evidence.ber_pre component.  I now get this error:

return value: 255
message: Error executing script exit value: 255Subroutine new redefined
at /opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm line 8.
Subroutine get_annotation redefined at
/opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm line 16. Subroutine
is_trusted redefined at
/opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm line 24. Subroutine
_init redefined at /opt/ergatis/lib/perl5//UnirefAnnotation/NestedDB.pm
line 31. DBD::mysql::db do failed: Unknown database 'uniprot_annotation'
at /opt/ergatis/lib/perl5//UnirefAnnotation/DBUtil.pm line 280. Could
not create parser object (PFunc::EvidenceParser::BERUniprotParser).
[Could not create object type: UnirefAnnotation::Database
[DBD::mysql::db do failed: Unknown database 'uniprot_annotation' at
/opt/ergatis/lib/perl5//UnirefAnnotation/DBUtil.pm line 280. ] ] at
/opt/ergatis/bin/parse_evidence.pl line 197.

Do I have to first create the uniprot_annotation database in mysql?

...meanwhile I still get the same error for the
parse_evidence.hmmpfam_pre:

return value: 2
message: Error executing script exit value: 2MLDBM error: Second level
tie failed, "No such file or directory" at
/opt/ergatis/lib/perl5//TIGR/Roles/HMM/PfamToRoleLookup.pm line 59 Could
not create parser object (PFunc::EvidenceParser::HMMParser). [Could not
open /opt/db/TIGRroles/hmm/PFAM_role_id.txt (No such file or directory)
at /opt/ergatis/lib/perl5//TIGR/Roles/HMM/PfamToRoleLookup.pm line 78. ]
at /opt/ergatis/bin/parse_evidence.pl line 197.

Is there any way to either get the TIGR roles lookup files it is looking
for or to set it up so it doesn't use them?

Thanks again for your help.

Jason


On 5/19/11 7:26 AM, Chris Hemmerich wrote:
Jason,

 From looking through our install notes for this pipeline, try grabbing
the
following library files from trunk in the Ergatis subversion repository:

/lib/

    NestedDB.pm
    UnirefAnnotation/DBUtil.pm
    UnirefAnnotation/Database.pm
    UnirefAnnotation/Lookup.pm
    UnirefAnnotation/NestedDB.pm
    PFunc/EvidenceParser/BERUniprotParser.pm

Cheers,
   Chris

On Wed, 18 May 2011, Jason Kwan wrote:

Hi Everyone,

Hopefully this is almost it as far as problems with this pipeline go.
I have
problems with the parse_evidence.hmmpfam_pre and parse_evidence.ber_pre
components.

For parse_evidence.hmmpfam_pre, it wants to know the
tigr_roles_db_dir.  I
thought this might be a directory containing roles_data.db that I got
from
the manatee distribution, but I don't think this is it.  It is looking
for
tigr_roles_db_dir/hmm/PFAM_role_id.txt, which I don't have.  I followed
Aaron's instructions from his message on 5/13/11, Aaron: was there
something
you also had to do with this module to illiminate use of TIGR roles?

For parse_evidence.ber_pre, initially there was an error saying that
the
username and password arguments were required.  Thinking this would be
for a
database that it was going to create, I just set username=test and
password=test.  I now get this error:

return value: 2
message: Error executing script exit value: 2Could not create parser
object
(PFunc::EvidenceParser::BERUniprotParser). [DBI
connect('host=jabba;packetSize=8092','test',...) failed: Unknown MySQL
server
host 'jabba' (1) at /opt/ergatis/lib/perl5//UnirefClusters/Database.pm
line
241 ] at /opt/ergatis/bin/parse_evidence.pl line 197.

So is this trying to connect to an existing mysql database that I
should have
set up on localhost?

Incidently, I found an error in the template for this pipeline.  In my
version the section for priam_ec_assignment.default reads:

<commandSet>
<state>incomplete</state>
<name>priam_ec_assignment.default</name>
</commandSet>

...and I changed the first line to<commandSet type="serial">  to get
the
component to show up.

Thanks to everyone for the help so far.  Hopefully these should be
that last
things for this pipeline as I'm quite near the end.

Jason Kwan

On 5/18/11 7:42 AM, Joshua Orvis wrote:
  Jason -

  It looks like there was a bug in the software.config file that has
already
  been fixed, but you got it before the fix was made.  In your
  software.config file you should have this section for ps_scan (it
gets
  imported into your component config at runtime):

  [component ps_scan]
  $;PS_SCAN_EXEC$;=/usr/local/bin/ps_scan.pl<http://ps_scan.pl>
  $;PROSITE_DAT_PATH$;=/usr/local/projects/db/prosite/prosite.dat
  $;PFSCAN_EXEC$;=/usr/local/bin/pfscan
  $;PSA2MSA_EXEC$;=/usr/local/bin/psa2msa

  In your version it looks like the last variable was $;PSA2MSA$;
rather
  than $;PSA2MSA_EXEC$;  This made the value blank, since in the
component's
  ps_scan.i1.xml file it has a line like this:

  <arg>$;OTHER_OPTS$; -d '$;PROSITE_DAT$;' $;I_FILE_PATH$; --pfscan
  $;PFSCAN_EXEC$; --psa2msa $;PSA2MSA_EXEC$;</arg>

  Notice at the end there that the variable it was looking for was
  $;PSA2MSA_EXEC$;.  Make sure these all agree, and you should be set.

  Joshua



  2011/5/18 Jason Kwan<[hidden email][hidden email]>

      Hi Everyone,

      I'm doing test runs of the prokaryotic annotation pipeline
      currently.  When it gets to the ps_scan component I get the
error
      message:

      return value: 1
      message: Error executing script exit value: 1Option psa2msa
      requires an argument

      ...because the command does not include anything for the psa2msa
      argument:

      /opt/ps_scan/ps_scan.pl<http://ps_scan.pl>  -s -d
      '/opt/db/prosite.dat'
      
/usr/local/projects/lin_actino/output_repository/translate_sequence/123
_final_polypeptides/i1/g5/NODE_5_length_7187_cov_357.polypeptide.284042
.1.fsa
      --pfscan /opt/ps_scan/pfscan --psa2msa

      However, the ps_scan paths are all present and correct in the
      config section of this component:


            [component ps_scan]

          * $;PS_SCAN_EXEC$; =
/opt/ps_scan/ps_scan.pl<http://ps_scan.pl>
          * $;PROSITE_DAT_PATH$; = /opt/db/prosite.dat
          * $;PFSCAN_EXEC$; = /opt/ps_scan/pfscan
          * $;PSA2MSA$; = /opt/ps_scan/psa2msa

      Can anyone help me find exactly where the $;PSA2MSA$; variable
is
      not being passed on?

      Thanks again for your help

      Jason Kwan


      
-----------------------------------------------------------------------
-------
      What Every C/C++ and Fortran developer Should Know!
      Read this article and learn how Intel has extended the reach of
its
      next-generation tools to help Windows* and Linux* C/C++ and
Fortran
      developers boost performance applications - including clusters.
      http://p.sf.net/sfu/intel-dev2devmay
      _______________________________________________
      Ergatis-users mailing list
      [hidden email]
      [hidden email]
      https://lists.sourceforge.net/lists/listinfo/ergatis-users



          

--------------------------------------------------------------------------
----
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users