Creating a component with custom SGE parameters?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Creating a component with custom SGE parameters?

Aaron Gussman
Does anyone have experience creating an ergatis component with custom
qsub parameters?  SGE here is configured with default constraints that
automatically kill long-running jobs or jobs that use too much memory.
 You can override these settings with the qsub command by specifying
new values.  The catch is that I only want to change it for some
components (and even then, have it as a configurable option), rather
than just hacking the qsub command directly.

I don't recall anyone ever having done this, but I thought I'd ask
before I started.

Thanks,
Aaron

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Creating a component with custom SGE parameters?

Sam Angiuoli
Hi Aaron,
Try adding passthrough to dceSpec
<dceSpec>
 <passthrough>-someargs</passthrough>
/>

On Wed, Aug 18, 2010 at 9:37 AM, Aaron Gussman <[hidden email]> wrote:

> Does anyone have experience creating an ergatis component with custom
> qsub parameters?  SGE here is configured with default constraints that
> automatically kill long-running jobs or jobs that use too much memory.
>  You can override these settings with the qsub command by specifying
> new values.  The catch is that I only want to change it for some
> components (and even then, have it as a configurable option), rather
> than just hacking the qsub command directly.
>
> I don't recall anyone ever having done this, but I thought I'd ask
> before I started.
>
> Thanks,
> Aaron
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by
>
> Make an app they can't live without
> Enter the BlackBerry Developer Challenge
> http://p.sf.net/sfu/RIM-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Creating a component with custom SGE parameters?

Aaron Gussman
Thanks for the info Sam!

This is more complicated than I thought.  The command is being run as
part of an iterator.  It's the only command in the iterator, and it
was of type <type>RunUnixCommand</type>.  I changed it to
RunDistributedCommand and added the <dceSpec> and the <passthrough>
commands worked.  But, the problem is that the job is now occupying
two grid slots: one for the command itself and one for RunWorkflow.

Ideally I would want to add the <dceSpec> to where RunWorkflow is
submitted to the grid.  After looking around though, I can't figure
out where that is.

Thanks,
Aaron

On Wed, Aug 18, 2010 at 9:51 AM, Sam Angiuoli <[hidden email]> wrote:

> Hi Aaron,
> Try adding passthrough to dceSpec
> <dceSpec>
>  <passthrough>-someargs</passthrough>
> />
>
> On Wed, Aug 18, 2010 at 9:37 AM, Aaron Gussman <[hidden email]> wrote:
>> Does anyone have experience creating an ergatis component with custom
>> qsub parameters?  SGE here is configured with default constraints that
>> automatically kill long-running jobs or jobs that use too much memory.
>>  You can override these settings with the qsub command by specifying
>> new values.  The catch is that I only want to change it for some
>> components (and even then, have it as a configurable option), rather
>> than just hacking the qsub command directly.
>>
>> I don't recall anyone ever having done this, but I thought I'd ask
>> before I started.
>>
>> Thanks,
>> Aaron
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by
>>
>> Make an app they can't live without
>> Enter the BlackBerry Developer Challenge
>> http://p.sf.net/sfu/RIM-dev2dev
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by
>
> Make an app they can't live without
> Enter the BlackBerry Developer Challenge
> http://p.sf.net/sfu/RIM-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Creating a component with custom SGE parameters?

Joshua Orvis
Aaron -

For normal iterators the commands that actually do the submission on the grid are the i1.xml.gz files for each component.

JO



On Wed, Aug 18, 2010 at 12:27 PM, Aaron Gussman <[hidden email]> wrote:
Thanks for the info Sam!

This is more complicated than I thought.  The command is being run as
part of an iterator.  It's the only command in the iterator, and it
was of type <type>RunUnixCommand</type>.  I changed it to
RunDistributedCommand and added the <dceSpec> and the <passthrough>
commands worked.  But, the problem is that the job is now occupying
two grid slots: one for the command itself and one for RunWorkflow.

Ideally I would want to add the <dceSpec> to where RunWorkflow is
submitted to the grid.  After looking around though, I can't figure
out where that is.

Thanks,
Aaron

On Wed, Aug 18, 2010 at 9:51 AM, Sam Angiuoli <[hidden email]> wrote:
> Hi Aaron,
> Try adding passthrough to dceSpec
> <dceSpec>
>  <passthrough>-someargs</passthrough>
> />
>
> On Wed, Aug 18, 2010 at 9:37 AM, Aaron Gussman <[hidden email]> wrote:
>> Does anyone have experience creating an ergatis component with custom
>> qsub parameters?  SGE here is configured with default constraints that
>> automatically kill long-running jobs or jobs that use too much memory.
>>  You can override these settings with the qsub command by specifying
>> new values.  The catch is that I only want to change it for some
>> components (and even then, have it as a configurable option), rather
>> than just hacking the qsub command directly.
>>
>> I don't recall anyone ever having done this, but I thought I'd ask
>> before I started.
>>
>> Thanks,
>> Aaron
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by
>>
>> Make an app they can't live without
>> Enter the BlackBerry Developer Challenge
>> http://p.sf.net/sfu/RIM-dev2dev
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by
>
> Make an app they can't live without
> Enter the BlackBerry Developer Challenge
> http://p.sf.net/sfu/RIM-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Creating a component with custom SGE parameters?

Joshua Orvis
 I mean, they're IN the i1.xml.gz files.  I'm sure you figured that out.

JO



On Wed, Aug 18, 2010 at 2:12 PM, Joshua Orvis <[hidden email]> wrote:
Aaron -

For normal iterators the commands that actually do the submission on the grid are the i1.xml.gz files for each component.

JO




On Wed, Aug 18, 2010 at 12:27 PM, Aaron Gussman <[hidden email]> wrote:
Thanks for the info Sam!

This is more complicated than I thought.  The command is being run as
part of an iterator.  It's the only command in the iterator, and it
was of type <type>RunUnixCommand</type>.  I changed it to
RunDistributedCommand and added the <dceSpec> and the <passthrough>
commands worked.  But, the problem is that the job is now occupying
two grid slots: one for the command itself and one for RunWorkflow.

Ideally I would want to add the <dceSpec> to where RunWorkflow is
submitted to the grid.  After looking around though, I can't figure
out where that is.

Thanks,
Aaron

On Wed, Aug 18, 2010 at 9:51 AM, Sam Angiuoli <[hidden email]> wrote:
> Hi Aaron,
> Try adding passthrough to dceSpec
> <dceSpec>
>  <passthrough>-someargs</passthrough>
> />
>
> On Wed, Aug 18, 2010 at 9:37 AM, Aaron Gussman <[hidden email]> wrote:
>> Does anyone have experience creating an ergatis component with custom
>> qsub parameters?  SGE here is configured with default constraints that
>> automatically kill long-running jobs or jobs that use too much memory.
>>  You can override these settings with the qsub command by specifying
>> new values.  The catch is that I only want to change it for some
>> components (and even then, have it as a configurable option), rather
>> than just hacking the qsub command directly.
>>
>> I don't recall anyone ever having done this, but I thought I'd ask
>> before I started.
>>
>> Thanks,
>> Aaron
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by
>>
>> Make an app they can't live without
>> Enter the BlackBerry Developer Challenge
>> http://p.sf.net/sfu/RIM-dev2dev
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by
>
> Make an app they can't live without
> Enter the BlackBerry Developer Challenge
> http://p.sf.net/sfu/RIM-dev2dev
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users