PERL5LIB

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

PERL5LIB

Mariette
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Joshua Orvis
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Joshua Orvis
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Mariette

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

Pipeline.pm (12K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Mariette

Joshua,

there is something I'm missing. I did modified the Pipeline.pm so the PERL5LIB is settled. When I take a look at the pipeline.XXX.run.sh script written down,
the variable is correctly set ! Is this script used to process the workflow ?

Jérôme







Le 19/04/2012 10:16, Mariette a écrit :

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Joshua Orvis
Jérôme -

The script looks right, and that was going to be my next question.  Yes, Ergatis writes that shell script and then executes it, and within that script RunWorkflow is called.  Has anything else changed in your SGE configuration?  You could replace the RunWorkflow line in the script with your own qsub test that just prints out the environment once it's placed on a node.  This would allow you to see if it's within the SGE job submission that your job is being lost.  Out of curiosity, do you have the 'submit_pipelines_as_jobs' option turned on in the ergatis.ini file?

Joshua



2012/4/20 Mariette <[hidden email]>

Joshua,

there is something I'm missing. I did modified the Pipeline.pm so the PERL5LIB is settled. When I take a look at the pipeline.XXX.run.sh script written down,
the variable is correctly set ! Is this script used to process the workflow ?

Jérôme







Le 19/04/2012 10:16, Mariette a écrit :

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Mariette

Hi everyone,

I'm still facing the same probleme ... I moved on by making a sed to all ergatis scripts to add the right PERL5LIB path, but this is just a patch! I need to do the same with the
LD_LIBRARY_PATH and the PATH variables ... but it's working with non of them!

The script produced (pipeline.*.run.sh) is correctly written down, and the execution with an echo right after display the right path !

What we changed is the full cluster, so we are running the open sge version and no longer the sun sge.

any idea on debugging ?
thanks,

Jerome






Le 20/04/2012 17:08, Joshua Orvis a écrit :
Jérôme -

The script looks right, and that was going to be my next question.  Yes, Ergatis writes that shell script and then executes it, and within that script RunWorkflow is called.  Has anything else changed in your SGE configuration?  You could replace the RunWorkflow line in the script with your own qsub test that just prints out the environment once it's placed on a node.  This would allow you to see if it's within the SGE job submission that your job is being lost.  Out of curiosity, do you have the 'submit_pipelines_as_jobs' option turned on in the ergatis.ini file?

Joshua



2012/4/20 Mariette <[hidden email]>

Joshua,

there is something I'm missing. I did modified the Pipeline.pm so the PERL5LIB is settled. When I take a look at the pipeline.XXX.run.sh script written down,
the variable is correctly set ! Is this script used to process the workflow ?

Jérôme







Le 19/04/2012 10:16, Mariette a écrit :

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Mariette

Hi again,

so I made a component with just a env as execution, and the paths are not right !
What are the steps after the execution of the script ?

Jerome






Le 02/05/2012 16:48, Mariette a écrit :

Hi everyone,

I'm still facing the same probleme ... I moved on by making a sed to all ergatis scripts to add the right PERL5LIB path, but this is just a patch! I need to do the same with the
LD_LIBRARY_PATH and the PATH variables ... but it's working with non of them!

The script produced (pipeline.*.run.sh) is correctly written down, and the execution with an echo right after display the right path !

What we changed is the full cluster, so we are running the open sge version and no longer the sun sge.

any idea on debugging ?
thanks,

Jerome






Le 20/04/2012 17:08, Joshua Orvis a écrit :
Jérôme -

The script looks right, and that was going to be my next question.  Yes, Ergatis writes that shell script and then executes it, and within that script RunWorkflow is called.  Has anything else changed in your SGE configuration?  You could replace the RunWorkflow line in the script with your own qsub test that just prints out the environment once it's placed on a node.  This would allow you to see if it's within the SGE job submission that your job is being lost.  Out of curiosity, do you have the 'submit_pipelines_as_jobs' option turned on in the ergatis.ini file?

Joshua



2012/4/20 Mariette <[hidden email]>

Joshua,

there is something I'm missing. I did modified the Pipeline.pm so the PERL5LIB is settled. When I take a look at the pipeline.XXX.run.sh script written down,
the variable is correctly set ! Is this script used to process the workflow ?

Jérôme







Le 19/04/2012 10:16, Mariette a écrit :

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Joshua Orvis
Jerome -

I think you should take a step back here.  Take Ergatis and Workflow Engine completely out of this.  What happens if you write a shell script that sets an ENV variable and then does the equivalent of a qsub to run your script that prints your environment to a file?  Is the ENV retained?

Joshua

On Wed, May 2, 2012 at 10:05 AM, Mariette <[hidden email]> wrote:

Hi again,

so I made a component with just a env as execution, and the paths are not right !
What are the steps after the execution of the script ?

Jerome






Le 02/05/2012 16:48, Mariette a écrit :

Hi everyone,

I'm still facing the same probleme ... I moved on by making a sed to all ergatis scripts to add the right PERL5LIB path, but this is just a patch! I need to do the same with the
LD_LIBRARY_PATH and the PATH variables ... but it's working with non of them!

The script produced (pipeline.*.run.sh) is correctly written down, and the execution with an echo right after display the right path !

What we changed is the full cluster, so we are running the open sge version and no longer the sun sge.

any idea on debugging ?
thanks,

Jerome






Le 20/04/2012 17:08, Joshua Orvis a écrit :
Jérôme -

The script looks right, and that was going to be my next question.  Yes, Ergatis writes that shell script and then executes it, and within that script RunWorkflow is called.  Has anything else changed in your SGE configuration?  You could replace the RunWorkflow line in the script with your own qsub test that just prints out the environment once it's placed on a node.  This would allow you to see if it's within the SGE job submission that your job is being lost.  Out of curiosity, do you have the 'submit_pipelines_as_jobs' option turned on in the ergatis.ini file?

Joshua



2012/4/20 Mariette <[hidden email]>

Joshua,

there is something I'm missing. I did modified the Pipeline.pm so the PERL5LIB is settled. When I take a look at the pipeline.XXX.run.sh script written down,
the variable is correctly set ! Is this script used to process the workflow ?

Jérôme







Le 19/04/2012 10:16, Mariette a écrit :

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Mariette

Joshua,

they are pretty close, SGE adds some variables (SGE_BINARY_PATH, SGE_CWD_PATH ...), but the PATH, the LD_LIBRARY_PATH are the same within or not a qsub.

Jerome




Le 02/05/2012 17:29, Joshua Orvis a écrit :
Jerome -

I think you should take a step back here.  Take Ergatis and Workflow Engine completely out of this.  What happens if you write a shell script that sets an ENV variable and then does the equivalent of a qsub to run your script that prints your environment to a file?  Is the ENV retained?

Joshua

On Wed, May 2, 2012 at 10:05 AM, Mariette <[hidden email]> wrote:

Hi again,

so I made a component with just a env as execution, and the paths are not right !
What are the steps after the execution of the script ?

Jerome






Le 02/05/2012 16:48, Mariette a écrit :

Hi everyone,

I'm still facing the same probleme ... I moved on by making a sed to all ergatis scripts to add the right PERL5LIB path, but this is just a patch! I need to do the same with the
LD_LIBRARY_PATH and the PATH variables ... but it's working with non of them!

The script produced (pipeline.*.run.sh) is correctly written down, and the execution with an echo right after display the right path !

What we changed is the full cluster, so we are running the open sge version and no longer the sun sge.

any idea on debugging ?
thanks,

Jerome






Le 20/04/2012 17:08, Joshua Orvis a écrit :
Jérôme -

The script looks right, and that was going to be my next question.  Yes, Ergatis writes that shell script and then executes it, and within that script RunWorkflow is called.  Has anything else changed in your SGE configuration?  You could replace the RunWorkflow line in the script with your own qsub test that just prints out the environment once it's placed on a node.  This would allow you to see if it's within the SGE job submission that your job is being lost.  Out of curiosity, do you have the 'submit_pipelines_as_jobs' option turned on in the ergatis.ini file?

Joshua



2012/4/20 Mariette <[hidden email]>

Joshua,

there is something I'm missing. I did modified the Pipeline.pm so the PERL5LIB is settled. When I take a look at the pipeline.XXX.run.sh script written down,
the variable is correctly set ! Is this script used to process the workflow ?

Jérôme







Le 19/04/2012 10:16, Mariette a écrit :

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Mariette

Joshua,

Just so you know, we are running ergatis on a red hat 6 system, have you make it run on it ?

I get a litle further. From the pipeline.X.run.sh file, I added the folowing lines:

sudo -E -u ng6 echo $LD_LIBRARY_PATH > /work/tmp/tt.ld
sudo -E -u ng6 env > /work/tmp/tt.env

the /work/tmp/tt.ld file does contain the right path! However the /work/tmp/tt.env does not contain the LD_LIBARY_PATH at all!

An other point, within ergatis, a simple component running a 'env' command return a LD_LIBRARY_PATH set to:
LD_LIBRARY_PATH=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64
I tryed to figure out where that path could come from ... any idea ?

thanks for your time!

Jérôme





Le 03/05/2012 10:02, Mariette a écrit :

Joshua,

they are pretty close, SGE adds some variables (SGE_BINARY_PATH, SGE_CWD_PATH ...), but the PATH, the LD_LIBRARY_PATH are the same within or not a qsub.

Jerome




Le 02/05/2012 17:29, Joshua Orvis a écrit :
Jerome -

I think you should take a step back here.  Take Ergatis and Workflow Engine completely out of this.  What happens if you write a shell script that sets an ENV variable and then does the equivalent of a qsub to run your script that prints your environment to a file?  Is the ENV retained?

Joshua

On Wed, May 2, 2012 at 10:05 AM, Mariette <[hidden email]> wrote:

Hi again,

so I made a component with just a env as execution, and the paths are not right !
What are the steps after the execution of the script ?

Jerome






Le 02/05/2012 16:48, Mariette a écrit :

Hi everyone,

I'm still facing the same probleme ... I moved on by making a sed to all ergatis scripts to add the right PERL5LIB path, but this is just a patch! I need to do the same with the
LD_LIBRARY_PATH and the PATH variables ... but it's working with non of them!

The script produced (pipeline.*.run.sh) is correctly written down, and the execution with an echo right after display the right path !

What we changed is the full cluster, so we are running the open sge version and no longer the sun sge.

any idea on debugging ?
thanks,

Jerome






Le 20/04/2012 17:08, Joshua Orvis a écrit :
Jérôme -

The script looks right, and that was going to be my next question.  Yes, Ergatis writes that shell script and then executes it, and within that script RunWorkflow is called.  Has anything else changed in your SGE configuration?  You could replace the RunWorkflow line in the script with your own qsub test that just prints out the environment once it's placed on a node.  This would allow you to see if it's within the SGE job submission that your job is being lost.  Out of curiosity, do you have the 'submit_pipelines_as_jobs' option turned on in the ergatis.ini file?

Joshua



2012/4/20 Mariette <[hidden email]>

Joshua,

there is something I'm missing. I did modified the Pipeline.pm so the PERL5LIB is settled. When I take a look at the pipeline.XXX.run.sh script written down,
the variable is correctly set ! Is this script used to process the workflow ?

Jérôme







Le 19/04/2012 10:16, Mariette a écrit :

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Joshua Orvis
Jerome -

This information is helpful.  In the last test that you mentioned, when running in Ergatis, was that step distributed on the grid at all or just run locally?

JO

On Thu, May 3, 2012 at 7:31 AM, Mariette <[hidden email]> wrote:

Joshua,

Just so you know, we are running ergatis on a red hat 6 system, have you make it run on it ?

I get a litle further. From the pipeline.X.run.sh file, I added the folowing lines:

sudo -E -u ng6 echo $LD_LIBRARY_PATH > /work/tmp/tt.ld
sudo -E -u ng6 env > /work/tmp/tt.env

the /work/tmp/tt.ld file does contain the right path! However the /work/tmp/tt.env does not contain the LD_LIBARY_PATH at all!

An other point, within ergatis, a simple component running a 'env' command return a LD_LIBRARY_PATH set to:
LD_LIBRARY_PATH=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64
I tryed to figure out where that path could come from ... any idea ?


thanks for your time!

Jérôme





Le 03/05/2012 10:02, Mariette a écrit :

Joshua,

they are pretty close, SGE adds some variables (SGE_BINARY_PATH, SGE_CWD_PATH ...), but the PATH, the LD_LIBRARY_PATH are the same within or not a qsub.

Jerome




Le 02/05/2012 17:29, Joshua Orvis a écrit :
Jerome -

I think you should take a step back here.  Take Ergatis and Workflow Engine completely out of this.  What happens if you write a shell script that sets an ENV variable and then does the equivalent of a qsub to run your script that prints your environment to a file?  Is the ENV retained?

Joshua

On Wed, May 2, 2012 at 10:05 AM, Mariette <[hidden email]> wrote:

Hi again,

so I made a component with just a env as execution, and the paths are not right !
What are the steps after the execution of the script ?

Jerome






Le 02/05/2012 16:48, Mariette a écrit :

Hi everyone,

I'm still facing the same probleme ... I moved on by making a sed to all ergatis scripts to add the right PERL5LIB path, but this is just a patch! I need to do the same with the
LD_LIBRARY_PATH and the PATH variables ... but it's working with non of them!

The script produced (pipeline.*.run.sh) is correctly written down, and the execution with an echo right after display the right path !

What we changed is the full cluster, so we are running the open sge version and no longer the sun sge.

any idea on debugging ?
thanks,

Jerome






Le 20/04/2012 17:08, Joshua Orvis a écrit :
Jérôme -

The script looks right, and that was going to be my next question.  Yes, Ergatis writes that shell script and then executes it, and within that script RunWorkflow is called.  Has anything else changed in your SGE configuration?  You could replace the RunWorkflow line in the script with your own qsub test that just prints out the environment once it's placed on a node.  This would allow you to see if it's within the SGE job submission that your job is being lost.  Out of curiosity, do you have the 'submit_pipelines_as_jobs' option turned on in the ergatis.ini file?

Joshua



2012/4/20 Mariette <[hidden email]>

Joshua,

there is something I'm missing. I did modified the Pipeline.pm so the PERL5LIB is settled. When I take a look at the pipeline.XXX.run.sh script written down,
the variable is correctly set ! Is this script used to process the workflow ?

Jérôme







Le 19/04/2012 10:16, Mariette a écrit :

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Mariette

Joshua,

We got the same behaviour for both!

On our old system, we did the same (adding sudo -E -u ng6 echo $LD_LIBRARY_PATH > /work/tmp/tt.ld; sudo -E -u ng6 env > /work/tmp/tt.env on the pipeline.X.run.sh script) and we got the same result (nothing on the env, but ok when using the $LD_LIBRARY_PATH)

We run the same component (executing the 'env') on our old system what lead to a correct LD_LIBARY_PATH.

What we tested also is to take off the sudo prefix from the pipeline.X.run.sh script, here the LD_LIBARY_PATH is ok! So we are pretty sure it's coming from the sudo. What we did is to add the LD_LIBRARY_PATH to the env_keep variable without any success!

Defaults    env_reset
Defaults    env_keep =  "COLORS DISPLAY HOSTNAME HISTSIZE INPUTRC KDEDIR LS_COLORS"
...
Defaults    env_keep += "LC_TIME LC_ALL LANGUAGE LINGUAS _XKB_CHARSET XAUTHORITY LD_LIBRARY_PATH"
Defaults    env_keep += "PATH "

Something weird is that the PATH is ok in the pipeline execution, but not the LD_LIBARY_PATH and PERL5LIB variables, even if we did the same with those variables !

How this variables are retreived within tigr workflow ? There is any changed since last workflow version using those variables ?

Jerome







Le 03/05/2012 15:44, Joshua Orvis a écrit :
Jerome -

This information is helpful.  In the last test that you mentioned, when running in Ergatis, was that step distributed on the grid at all or just run locally?

JO

On Thu, May 3, 2012 at 7:31 AM, Mariette <[hidden email]> wrote:

Joshua,

Just so you know, we are running ergatis on a red hat 6 system, have you make it run on it ?

I get a litle further. From the pipeline.X.run.sh file, I added the folowing lines:

sudo -E -u ng6 echo $LD_LIBRARY_PATH > /work/tmp/tt.ld
sudo -E -u ng6 env > /work/tmp/tt.env

the /work/tmp/tt.ld file does contain the right path! However the /work/tmp/tt.env does not contain the LD_LIBARY_PATH at all!

An other point, within ergatis, a simple component running a 'env' command return a LD_LIBRARY_PATH set to:
LD_LIBRARY_PATH=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64
I tryed to figure out where that path could come from ... any idea ?


thanks for your time!

Jérôme





Le 03/05/2012 10:02, Mariette a écrit :

Joshua,

they are pretty close, SGE adds some variables (SGE_BINARY_PATH, SGE_CWD_PATH ...), but the PATH, the LD_LIBRARY_PATH are the same within or not a qsub.

Jerome




Le 02/05/2012 17:29, Joshua Orvis a écrit :
Jerome -

I think you should take a step back here.  Take Ergatis and Workflow Engine completely out of this.  What happens if you write a shell script that sets an ENV variable and then does the equivalent of a qsub to run your script that prints your environment to a file?  Is the ENV retained?

Joshua

On Wed, May 2, 2012 at 10:05 AM, Mariette <[hidden email]> wrote:

Hi again,

so I made a component with just a env as execution, and the paths are not right !
What are the steps after the execution of the script ?

Jerome






Le 02/05/2012 16:48, Mariette a écrit :

Hi everyone,

I'm still facing the same probleme ... I moved on by making a sed to all ergatis scripts to add the right PERL5LIB path, but this is just a patch! I need to do the same with the
LD_LIBRARY_PATH and the PATH variables ... but it's working with non of them!

The script produced (pipeline.*.run.sh) is correctly written down, and the execution with an echo right after display the right path !

What we changed is the full cluster, so we are running the open sge version and no longer the sun sge.

any idea on debugging ?
thanks,

Jerome






Le 20/04/2012 17:08, Joshua Orvis a écrit :
Jérôme -

The script looks right, and that was going to be my next question.  Yes, Ergatis writes that shell script and then executes it, and within that script RunWorkflow is called.  Has anything else changed in your SGE configuration?  You could replace the RunWorkflow line in the script with your own qsub test that just prints out the environment once it's placed on a node.  This would allow you to see if it's within the SGE job submission that your job is being lost.  Out of curiosity, do you have the 'submit_pipelines_as_jobs' option turned on in the ergatis.ini file?

Joshua



2012/4/20 Mariette <[hidden email]>

Joshua,

there is something I'm missing. I did modified the Pipeline.pm so the PERL5LIB is settled. When I take a look at the pipeline.XXX.run.sh script written down,
the variable is correctly set ! Is this script used to process the workflow ?

Jérôme







Le 19/04/2012 10:16, Mariette a écrit :

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: PERL5LIB

Mariette

Hi again,

we have some other informations. From the Pipeline.pm, we changed the sudo command by:
  sudo -s -E -u <user> ... instead of sudo -E -u <user>: here it's working when running the pipeline.X.run.sh script within a terminal.
However I got a "no tty error" from the ergatis website, as ergatis is run as apache, this does sound right. My guess would be that
our apache user should have in is env the LD_LIBRARY_PATH correctly settled.

Does this make sense to you ?

Jerome




Le 03/05/2012 16:03, Mariette a écrit :

Joshua,

We got the same behaviour for both!

On our old system, we did the same (adding sudo -E -u ng6 echo $LD_LIBRARY_PATH > /work/tmp/tt.ld; sudo -E -u ng6 env > /work/tmp/tt.env on the pipeline.X.run.sh script) and we got the same result (nothing on the env, but ok when using the $LD_LIBRARY_PATH)

We run the same component (executing the 'env') on our old system what lead to a correct LD_LIBARY_PATH.

What we tested also is to take off the sudo prefix from the pipeline.X.run.sh script, here the LD_LIBARY_PATH is ok! So we are pretty sure it's coming from the sudo. What we did is to add the LD_LIBRARY_PATH to the env_keep variable without any success!

Defaults    env_reset
Defaults    env_keep =  "COLORS DISPLAY HOSTNAME HISTSIZE INPUTRC KDEDIR LS_COLORS"
...
Defaults    env_keep += "LC_TIME LC_ALL LANGUAGE LINGUAS _XKB_CHARSET XAUTHORITY LD_LIBRARY_PATH"
Defaults    env_keep += "PATH "

Something weird is that the PATH is ok in the pipeline execution, but not the LD_LIBARY_PATH and PERL5LIB variables, even if we did the same with those variables !

How this variables are retreived within tigr workflow ? There is any changed since last workflow version using those variables ?

Jerome







Le 03/05/2012 15:44, Joshua Orvis a écrit :
Jerome -

This information is helpful.  In the last test that you mentioned, when running in Ergatis, was that step distributed on the grid at all or just run locally?

JO

On Thu, May 3, 2012 at 7:31 AM, Mariette <[hidden email]> wrote:

Joshua,

Just so you know, we are running ergatis on a red hat 6 system, have you make it run on it ?

I get a litle further. From the pipeline.X.run.sh file, I added the folowing lines:

sudo -E -u ng6 echo $LD_LIBRARY_PATH > /work/tmp/tt.ld
sudo -E -u ng6 env > /work/tmp/tt.env

the /work/tmp/tt.ld file does contain the right path! However the /work/tmp/tt.env does not contain the LD_LIBARY_PATH at all!

An other point, within ergatis, a simple component running a 'env' command return a LD_LIBRARY_PATH set to:
LD_LIBRARY_PATH=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64
I tryed to figure out where that path could come from ... any idea ?


thanks for your time!

Jérôme





Le 03/05/2012 10:02, Mariette a écrit :

Joshua,

they are pretty close, SGE adds some variables (SGE_BINARY_PATH, SGE_CWD_PATH ...), but the PATH, the LD_LIBRARY_PATH are the same within or not a qsub.

Jerome




Le 02/05/2012 17:29, Joshua Orvis a écrit :
Jerome -

I think you should take a step back here.  Take Ergatis and Workflow Engine completely out of this.  What happens if you write a shell script that sets an ENV variable and then does the equivalent of a qsub to run your script that prints your environment to a file?  Is the ENV retained?

Joshua

On Wed, May 2, 2012 at 10:05 AM, Mariette <[hidden email]> wrote:

Hi again,

so I made a component with just a env as execution, and the paths are not right !
What are the steps after the execution of the script ?

Jerome






Le 02/05/2012 16:48, Mariette a écrit :

Hi everyone,

I'm still facing the same probleme ... I moved on by making a sed to all ergatis scripts to add the right PERL5LIB path, but this is just a patch! I need to do the same with the
LD_LIBRARY_PATH and the PATH variables ... but it's working with non of them!

The script produced (pipeline.*.run.sh) is correctly written down, and the execution with an echo right after display the right path !

What we changed is the full cluster, so we are running the open sge version and no longer the sun sge.

any idea on debugging ?
thanks,

Jerome






Le 20/04/2012 17:08, Joshua Orvis a écrit :
Jérôme -

The script looks right, and that was going to be my next question.  Yes, Ergatis writes that shell script and then executes it, and within that script RunWorkflow is called.  Has anything else changed in your SGE configuration?  You could replace the RunWorkflow line in the script with your own qsub test that just prints out the environment once it's placed on a node.  This would allow you to see if it's within the SGE job submission that your job is being lost.  Out of curiosity, do you have the 'submit_pipelines_as_jobs' option turned on in the ergatis.ini file?

Joshua



2012/4/20 Mariette <[hidden email]>

Joshua,

there is something I'm missing. I did modified the Pipeline.pm so the PERL5LIB is settled. When I take a look at the pipeline.XXX.run.sh script written down,
the variable is correctly set ! Is this script used to process the workflow ?

Jérôme







Le 19/04/2012 10:16, Mariette a écrit :

Joshua,
the file is in attachement. The error returned is:

Can't locate Config/IniFiles.pm in @INC (@INC contains: /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5//x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/ /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5/5.10.0 /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8/x86_64-linux-thread-multi /usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/5.8.8) at /usr/local/bioinfo/src/ergatisng6/v2r17b2/bin/replace_template_keys.pl line 17

Paths I provided to Pipeline.pm are not in @INC.

thanks for your time!
Jérôme


Le 13/04/2012 20:32, Joshua Orvis a écrit :
Jerome -

Do you mind sending me your modified Pipeline.pm file?  I don't see any changes here that would cause the ENV changes you make in the _setup_environment() method of Pipeline.pm to not make it to the execution of the create_template_keys.pl script.  I'd like to test it locally.

Joshua


On Fri, Apr 13, 2012 at 1:14 PM, Joshua Orvis <[hidden email]> wrote:
Jérôme -

I'm taking a look at this.

Joshua

On Fri, Apr 13, 2012 at 10:24 AM, Mariette <[hidden email]> wrote:
Hi everyone,

the environment we are running ergatis on is a bit tricky. The perl libs
are not located on the usual perl path.
usually when I have this kind of trouble with ergatis, I set the path
within the Pipeline.pm file. However it doesn't
work with the new version.

The script crashing is the create_template_keys, when I add the correct
lib path to this wrapper line:
export
PERL5LIB=/usr/local/bioinfo/src/ergatisng6/v2r17b2/lib/perl5/:/path/....

it does work! I would have guessed the Pipeline.pm was adding what I
wanted bu this one is not called in the
create_template_keys.pl script ...

How can I manage to fix this problem, without adding to all the wrapper
my path ?
thanks,

Jérôme


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users




------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/


_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users