dynamic queue name

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

dynamic queue name

Jaysheel Bhavsar
Hey guys,
        I there a way to dynamically define queue name?  Currently queue name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to pass the queue name based on the pipeline I am executing, along with h_vmem and h_stack parameters.

Thanks
Jaysheel
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Mahurkar, Anup
Jaysheel,

I am working on this feature right now. Should have it ready early next week.

Anup

On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]> wrote:

> Hey guys,
>    I there a way to dynamically define queue name?  Currently queue name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to pass the queue name based on the pipeline I am executing, along with h_vmem and h_stack parameters.
>
> Thanks
> Jaysheel
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Mahurkar, Anup
Jaysheel,

I have added the code to configure the default queue name that workflow
uses as well as specifying a queue name in the workflow that will be
honored. You will need to update the HTC and Workflow code. In the
htc.conf file specify the default queue name to be used for workflow jobs.
If nothing is specified in the command, it will look for the property
"sge_def_queue" in the config file.

We have made slight changes to epilog, prolog, and sge_submitter scripts
as well. So your best bet is to reinstall workflow.

Anup


On 5/11/12 4:44 PM, "Mahurkar, Anup" <[hidden email]> wrote:

>Jaysheel,
>
>I am working on this feature right now. Should have it ready early next
>week.
>
>Anup
>
>On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]>
>wrote:
>
>> Hey guys,
>>    I there a way to dynamically define queue name?  Currently queue
>>name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to
>>pass the queue name based on the pipeline I am executing, along with
>>h_vmem and h_stack parameters.
>>
>> Thanks
>> Jaysheel
>>
>>-------------------------------------------------------------------------
>>-----
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond.
>>Discussions
>> will include endpoint security, mobile security and the latest in
>>malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
>--------------------------------------------------------------------------
>----
>Live Security Virtual Conference
>Exclusive live event will cover all the ways today's security and
>threat landscape has changed and how IT managers can respond. Discussions
>will include endpoint security, mobile security and the latest in malware
>threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>_______________________________________________
>Ergatis-users mailing list
>[hidden email]
>https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Jaysheel Bhavsar
Hey
        Anup, I don't see a releaser version should I download workflow-3.1.x branch?

Jaysheel


On May 18, 2012, at 2:30 PM, Mahurkar, Anup wrote:

> Jaysheel,
>
> I have added the code to configure the default queue name that workflow
> uses as well as specifying a queue name in the workflow that will be
> honored. You will need to update the HTC and Workflow code. In the
> htc.conf file specify the default queue name to be used for workflow jobs.
> If nothing is specified in the command, it will look for the property
> "sge_def_queue" in the config file.
>
> We have made slight changes to epilog, prolog, and sge_submitter scripts
> as well. So your best bet is to reinstall workflow.
>
> Anup
>
>
> On 5/11/12 4:44 PM, "Mahurkar, Anup" <[hidden email]> wrote:
>
>> Jaysheel,
>>
>> I am working on this feature right now. Should have it ready early next
>> week.
>>
>> Anup
>>
>> On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]>
>> wrote:
>>
>>> Hey guys,
>>>   I there a way to dynamically define queue name?  Currently queue
>>> name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to
>>> pass the queue name based on the pipeline I am executing, along with
>>> h_vmem and h_stack parameters.
>>>
>>> Thanks
>>> Jaysheel
>>>
>>> -------------------------------------------------------------------------
>>> -----
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond.
>>> Discussions
>>> will include endpoint security, mobile security and the latest in
>>> malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Ergatis-users mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>> --------------------------------------------------------------------------
>> ----
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Mahurkar, Anup
Yes. It is in the branch.

-----Original Message-----
From: Jaysheel Bhavsar [mailto:[hidden email]]
Sent: Monday, May 21, 2012 1:44 PM
To: [hidden email]
Subject: Re: [Ergatis-users] dynamic queue name

Hey
        Anup, I don't see a releaser version should I download workflow-3.1.x branch?

Jaysheel


On May 18, 2012, at 2:30 PM, Mahurkar, Anup wrote:

> Jaysheel,
>
> I have added the code to configure the default queue name that
> workflow uses as well as specifying a queue name in the workflow that
> will be honored. You will need to update the HTC and Workflow code. In
> the htc.conf file specify the default queue name to be used for workflow jobs.
> If nothing is specified in the command, it will look for the property
> "sge_def_queue" in the config file.
>
> We have made slight changes to epilog, prolog, and sge_submitter
> scripts as well. So your best bet is to reinstall workflow.
>
> Anup
>
>
> On 5/11/12 4:44 PM, "Mahurkar, Anup" <[hidden email]> wrote:
>
>> Jaysheel,
>>
>> I am working on this feature right now. Should have it ready early
>> next week.
>>
>> Anup
>>
>> On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]>
>> wrote:
>>
>>> Hey guys,
>>>   I there a way to dynamically define queue name?  Currently queue
>>> name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to
>>> pass the queue name based on the pipeline I am executing, along with
>>> h_vmem and h_stack parameters.
>>>
>>> Thanks
>>> Jaysheel
>>>
>>> --------------------------------------------------------------------
>>> -----
>>> -----
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond.
>>> Discussions
>>> will include endpoint security, mobile security and the latest in
>>> malware threats.
>>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Ergatis-users mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>> ---------------------------------------------------------------------
>> -----
>> ----
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond.
>> Discussions will include endpoint security, mobile security and the
>> latest in malware threats.
>> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
>
> ----------------------------------------------------------------------
> --------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond.
> Discussions will include endpoint security, mobile security and the
> latest in malware threats.
> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Jaysheel Bhavsar
In reply to this post by Mahurkar, Anup
Hey Anup,
        I got the workflow-3.1.x branch and re-installed workflow.  I updated sge_def_queue and sge_pri_queue in ${WORKFLOW_HOME}/server-conf/htc.conf file but jobs are still being submitted to default queue.  I don't see -q option in the qsub command in sge_submitter.sh. Is there something I missed?

Jaysheel


On May 18, 2012, at 2:30 PM, Mahurkar, Anup wrote:

> Jaysheel,
>
> I have added the code to configure the default queue name that workflow
> uses as well as specifying a queue name in the workflow that will be
> honored. You will need to update the HTC and Workflow code. In the
> htc.conf file specify the default queue name to be used for workflow jobs.
> If nothing is specified in the command, it will look for the property
> "sge_def_queue" in the config file.
>
> We have made slight changes to epilog, prolog, and sge_submitter scripts
> as well. So your best bet is to reinstall workflow.
>
> Anup
>
>
> On 5/11/12 4:44 PM, "Mahurkar, Anup" <[hidden email]> wrote:
>
>> Jaysheel,
>>
>> I am working on this feature right now. Should have it ready early next
>> week.
>>
>> Anup
>>
>> On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]>
>> wrote:
>>
>>> Hey guys,
>>>   I there a way to dynamically define queue name?  Currently queue
>>> name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to
>>> pass the queue name based on the pipeline I am executing, along with
>>> h_vmem and h_stack parameters.
>>>
>>> Thanks
>>> Jaysheel
>>>
>>> -------------------------------------------------------------------------
>>> -----
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond.
>>> Discussions
>>> will include endpoint security, mobile security and the latest in
>>> malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Ergatis-users mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>> --------------------------------------------------------------------------
>> ----
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Mahurkar, Anup
Jaysheel,

The -q will be specified in the sge_job.sh script for each of the jobs. Is
it not there? If not could you send me the log file?

Anup

On 5/21/12 3:52 PM, "Jaysheel Bhavsar" <[hidden email]> wrote:

>Hey Anup,
> I got the workflow-3.1.x branch and re-installed workflow.  I updated
>sge_def_queue and sge_pri_queue in ${WORKFLOW_HOME}/server-conf/htc.conf
>file but jobs are still being submitted to default queue.  I don't see -q
>option in the qsub command in sge_submitter.sh. Is there something I
>missed?
>
>Jaysheel
>
>
>On May 18, 2012, at 2:30 PM, Mahurkar, Anup wrote:
>
>> Jaysheel,
>>
>> I have added the code to configure the default queue name that workflow
>> uses as well as specifying a queue name in the workflow that will be
>> honored. You will need to update the HTC and Workflow code. In the
>> htc.conf file specify the default queue name to be used for workflow
>>jobs.
>> If nothing is specified in the command, it will look for the property
>> "sge_def_queue" in the config file.
>>
>> We have made slight changes to epilog, prolog, and sge_submitter scripts
>> as well. So your best bet is to reinstall workflow.
>>
>> Anup
>>
>>
>> On 5/11/12 4:44 PM, "Mahurkar, Anup" <[hidden email]>
>>wrote:
>>
>>> Jaysheel,
>>>
>>> I am working on this feature right now. Should have it ready early next
>>> week.
>>>
>>> Anup
>>>
>>> On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]>
>>> wrote:
>>>
>>>> Hey guys,
>>>>   I there a way to dynamically define queue name?  Currently queue
>>>> name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to
>>>> pass the queue name based on the pipeline I am executing, along with
>>>> h_vmem and h_stack parameters.
>>>>
>>>> Thanks
>>>> Jaysheel
>>>>
>>>>
>>>>-----------------------------------------------------------------------
>>>>--
>>>> -----
>>>> Live Security Virtual Conference
>>>> Exclusive live event will cover all the ways today's security and
>>>> threat landscape has changed and how IT managers can respond.
>>>> Discussions
>>>> will include endpoint security, mobile security and the latest in
>>>> malware
>>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>>> _______________________________________________
>>>> Ergatis-users mailing list
>>>> [hidden email]
>>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>>
>>>
>>>------------------------------------------------------------------------
>>>--
>>> ----
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond.
>>>Discussions
>>> will include endpoint security, mobile security and the latest in
>>>malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Ergatis-users mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>>
>>
>>-------------------------------------------------------------------------
>>-----
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond.
>>Discussions
>> will include endpoint security, mobile security and the latest in
>>malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
>
>--------------------------------------------------------------------------
>----
>Live Security Virtual Conference
>Exclusive live event will cover all the ways today's security and
>threat landscape has changed and how IT managers can respond. Discussions
>will include endpoint security, mobile security and the latest in malware
>threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>_______________________________________________
>Ergatis-users mailing list
>[hidden email]
>https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Jaysheel Bhavsar
Hey Anup,
I don't see any -q option in sge_job.sh script  written in mock_server_base.  In the log file I see a call to she_job.sh which looks like this

qsub -b n -C '#SGE_DIRECTIVE' /projects/bsi/RandD/ngs/ergatis/mock_server_base/working/0/4521/sge_job.sh

Which come from sge_submitter.sh from what I can tell.

Jaysheel


On May 21, 2012, at 8:22 PM, Mahurkar, Anup wrote:

Jaysheel,

The -q will be specified in the sge_job.sh script for each of the jobs. Is
it not there? If not could you send me the log file?

Anup

On 5/21/12 3:52 PM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I got the workflow-3.1.x branch and re-installed workflow.  I updated
sge_def_queue and sge_pri_queue in ${WORKFLOW_HOME}/server-conf/htc.conf
file but jobs are still being submitted to default queue.  I don't see -q
option in the qsub command in sge_submitter.sh. Is there something I
missed?

Jaysheel


On May 18, 2012, at 2:30 PM, Mahurkar, Anup wrote:

Jaysheel,

I have added the code to configure the default queue name that workflow
uses as well as specifying a queue name in the workflow that will be
honored. You will need to update the HTC and Workflow code. In the
htc.conf file specify the default queue name to be used for workflow
jobs.
If nothing is specified in the command, it will look for the property
"sge_def_queue" in the config file.

We have made slight changes to epilog, prolog, and sge_submitter scripts
as well. So your best bet is to reinstall workflow.

Anup


On 5/11/12 4:44 PM, "Mahurkar, Anup" <[hidden email]>
wrote:

Jaysheel,

I am working on this feature right now. Should have it ready early next
week.

Anup

On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]>
wrote:

Hey guys,
 I there a way to dynamically define queue name?  Currently queue
name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to
pass the queue name based on the pipeline I am executing, along with
h_vmem and h_stack parameters.

Thanks
Jaysheel


-----------------------------------------------------------------------
--
-----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------
--
----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



-------------------------------------------------------------------------
-----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


--------------------------------------------------------------------------
----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Mahurkar, Anup
Can you send me the log file an the shell script?

Anup

On May 22, 2012, at 11:54 AM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I don't see any -q option in sge_job.sh script  written in mock_server_base.  In the log file I see a call to she_job.sh which looks like this

qsub -b n -C '#SGE_DIRECTIVE' /projects/bsi/RandD/ngs/ergatis/mock_server_base/working/0/4521/sge_job.sh

Which come from sge_submitter.sh from what I can tell.

Jaysheel


On May 21, 2012, at 8:22 PM, Mahurkar, Anup wrote:

Jaysheel,

The -q will be specified in the sge_job.sh script for each of the jobs. Is
it not there? If not could you send me the log file?

Anup

On 5/21/12 3:52 PM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I got the workflow-3.1.x branch and re-installed workflow.  I updated
sge_def_queue and sge_pri_queue in ${WORKFLOW_HOME}/server-conf/htc.conf
file but jobs are still being submitted to default queue.  I don't see -q
option in the qsub command in sge_submitter.sh. Is there something I
missed?

Jaysheel


On May 18, 2012, at 2:30 PM, Mahurkar, Anup wrote:

Jaysheel,

I have added the code to configure the default queue name that workflow
uses as well as specifying a queue name in the workflow that will be
honored. You will need to update the HTC and Workflow code. In the
htc.conf file specify the default queue name to be used for workflow
jobs.
If nothing is specified in the command, it will look for the property
"sge_def_queue" in the config file.

We have made slight changes to epilog, prolog, and sge_submitter scripts
as well. So your best bet is to reinstall workflow.

Anup


On 5/11/12 4:44 PM, "Mahurkar, Anup" <[hidden email]>
wrote:

Jaysheel,

I am working on this feature right now. Should have it ready early next
week.

Anup

On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]>
wrote:

Hey guys,
 I there a way to dynamically define queue name?  Currently queue
name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to
pass the queue name based on the pipeline I am executing, along with
h_vmem and h_stack parameters.

Thanks
Jaysheel


-----------------------------------------------------------------------
--
-----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------
--
----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



-------------------------------------------------------------------------
-----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


--------------------------------------------------------------------------
----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Jaysheel Bhavsar
Hey Anup,
Attached is the pipeline.xml.log and sge_job.sh file.

Jaysheel




On May 22, 2012, at 10:59 AM, Mahurkar, Anup wrote:

Can you send me the log file an the shell script?

Anup

On May 22, 2012, at 11:54 AM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I don't see any -q option in sge_job.sh script  written in mock_server_base.  In the log file I see a call to she_job.sh which looks like this

qsub -b n -C '#SGE_DIRECTIVE' /projects/bsi/RandD/ngs/ergatis/mock_server_base/working/0/4521/sge_job.sh

Which come from sge_submitter.sh from what I can tell.

Jaysheel


On May 21, 2012, at 8:22 PM, Mahurkar, Anup wrote:

Jaysheel,

The -q will be specified in the sge_job.sh script for each of the jobs. Is
it not there? If not could you send me the log file?

Anup

On 5/21/12 3:52 PM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I got the workflow-3.1.x branch and re-installed workflow.  I updated
sge_def_queue and sge_pri_queue in ${WORKFLOW_HOME}/server-conf/htc.conf
file but jobs are still being submitted to default queue.  I don't see -q
option in the qsub command in sge_submitter.sh. Is there something I
missed?

Jaysheel


On May 18, 2012, at 2:30 PM, Mahurkar, Anup wrote:

Jaysheel,

I have added the code to configure the default queue name that workflow
uses as well as specifying a queue name in the workflow that will be
honored. You will need to update the HTC and Workflow code. In the
htc.conf file specify the default queue name to be used for workflow
jobs.
If nothing is specified in the command, it will look for the property
"sge_def_queue" in the config file.

We have made slight changes to epilog, prolog, and sge_submitter scripts
as well. So your best bet is to reinstall workflow.

Anup


On 5/11/12 4:44 PM, "Mahurkar, Anup" <[hidden email]>
wrote:

Jaysheel,

I am working on this feature right now. Should have it ready early next
week.

Anup

On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]>
wrote:

Hey guys,
 I there a way to dynamically define queue name?  Currently queue
name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to
pass the queue name based on the pipeline I am executing, along with
h_vmem and h_stack parameters.

Thanks
Jaysheel


-----------------------------------------------------------------------
--
-----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------
--
----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



-------------------------------------------------------------------------
-----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


--------------------------------------------------------------------------
----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

pipeline.xml.log (5M) Download Attachment
sge_job.sh (9K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Jaysheel Bhavsar
In reply to this post by Mahurkar, Anup
Hey Anup, 
I was looking through Ergatis cgi files to figure out when queue option is set, and looks like in Pipeline.pm if pipeline_submission_queue is set its add to runprefix which is written to pipeline.ID.run.sh file.  None of my pipeline.ID.run.sh have -q option.  I not sure if I am on track here.

Below is the content of pipeline.ID.run.sh file.

Thanks
Jaysheel


#!/bin/bash

export SCRIPT_NAME="/cgi/run_pipeline.cgi"
export REQUEST_METHOD="POST"
export SCRIPT_FILENAME="/projects/bsi/RandD/ngs/ergatis/html/cgi/run_pipeline.cgi"
export WF_TEMPLATE="/projects/bsi/RandD/ngs/ergatis/workflow/templates"
export QUERY_STRING=""
export REMOTE_PORT="56650"
export WF_ROOT="/projects/bsi/RandD/ngs/ergatis/workflow"
export SGE_CELL="default"
export vappio_data_placement="0"
export HMMER_NCPU="1"
export CONTENT_TYPE="application/x-www-form-urlencoded"
export PERL5LIB="/home/m101236/perl5/lib/perl5/x86_64-linux-thread-multi:/home/m101236/perl5/lib/perl5::/usr/local/biotools/perl/5.14.2/lib/site_perl/5.14.2/x86_64-linux-thread-multi:/usr/local/biotools/perl/5.14.2/lib/site_perl/5.14.2:/usr/local/biotools/perl/5.14.2/lib/5.14.2/x86_64-linux-thread-multi:/usr/local/biotools/perl/5.14.2/lib/5.14.2"
export PATH="/projects/bsi/RandD/ngs/ergatis/workflow:/projects/bsi/RandD/ngs/ergatis/workflow/bin:/projects/bsi/RandD/ngs/ergatis/workflow/add-ons/bin:/projects/bsi/RandD/ngs/ergatis/workflow:/projects/bsi/RandD/ngs/ergatis/workflow/bin:/projects/bsi/RandD/ngs/ergatis/workflow/add-ons/bin:/home/m101236/perl5/bin:/usr/local/biotools/blast/blast-2.2.13/bin:/usr/local/biotools/hmmer/hmmer-3.0/bin:/usr/local/biotools/perl/5.14.2/bin:/home/biotools/sas/linux_SAS_9.2/bin:/usr/local/biotools/misc/ver1/bin:/usr/java/jdk1.6.0_05/bin:/usr/local/biotools/emboss/EMBOSS-5.0.0/bin:/usr/local/biotools/subversion/1.4.6/bin:/home/sge6_2/bin/lx24-amd64:/home/sge6_2/bin/lx24-amd64:/usr/local/biotools/python/2.7/bin:/usr/local/biotools/washu/ver1/bin:/usr/lib64/qt-3.3/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/biotools/git/1.7.9.2/bin:/home/m101236/bin:/projects/bsi/bictools/apps/alignment/samtools/samtools-0.1.18:/projects/bsi/bictools/apps/alignment/blat/34"
export GATEWAY_INTERFACE="CGI/1.1"
export DOCUMENT_ROOT="/projects/bsi/RandD/ngs/ergatis/html"
export HMM_SCRIPTS="/usr/local/devel/ANNOTATION/hmm/bin"
export LD_LIBRARY_PATH=""
export CONTENT_LENGTH="708"
export REMOTE_ADDR="172.31.194.244"
export WISECONFIGDIR="/usr/local/devel/ANNOTATION/EGC_utilities/WISE2/wise2.2.0/wisecfg"
export SGE_EXECD_PORT="537"
export REQUEST_URI="/cgi/run_pipeline.cgi"
export SGE_ARCH="lx26-amd64"
export WF_ROOT_INSTALL="/projects/bsi/RandD/ngs/ergatis/workflow"
export TMPDIR="/tmp"
export SGE_ROOT="/home/sge6_2"
export TERMCAP=""
export vappio_root="/opt/vappio_scripts"

  /projects/bsi/RandD/ngs/ergatis/workflow/RunWorkflow -i /projects/bsi/RandD/ngs/ergatis/projects/etest/workflow/runtime/pipeline/22/pipeline.xml -m 1 --init-heap=100m --max-heap=1024m  --logconf=/projects/bsi/RandD/ngs/ergatis/workflow/log4j.properties >& /projects/bsi/RandD/ngs/ergatis/projects/etest/workflow/runtime/pipeline/22/pipeline.xml.run.out




On May 22, 2012, at 10:59 AM, Mahurkar, Anup wrote:

Can you send me the log file an the shell script?

Anup

On May 22, 2012, at 11:54 AM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I don't see any -q option in sge_job.sh script  written in mock_server_base.  In the log file I see a call to she_job.sh which looks like this

qsub -b n -C '#SGE_DIRECTIVE' /projects/bsi/RandD/ngs/ergatis/mock_server_base/working/0/4521/sge_job.sh

Which come from sge_submitter.sh from what I can tell.

Jaysheel


On May 21, 2012, at 8:22 PM, Mahurkar, Anup wrote:

Jaysheel,

The -q will be specified in the sge_job.sh script for each of the jobs. Is
it not there? If not could you send me the log file?

Anup

On 5/21/12 3:52 PM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I got the workflow-3.1.x branch and re-installed workflow.  I updated
sge_def_queue and sge_pri_queue in ${WORKFLOW_HOME}/server-conf/htc.conf
file but jobs are still being submitted to default queue.  I don't see -q
option in the qsub command in sge_submitter.sh. Is there something I
missed?

Jaysheel


On May 18, 2012, at 2:30 PM, Mahurkar, Anup wrote:

Jaysheel,

I have added the code to configure the default queue name that workflow
uses as well as specifying a queue name in the workflow that will be
honored. You will need to update the HTC and Workflow code. In the
htc.conf file specify the default queue name to be used for workflow
jobs.
If nothing is specified in the command, it will look for the property
"sge_def_queue" in the config file.

We have made slight changes to epilog, prolog, and sge_submitter scripts
as well. So your best bet is to reinstall workflow.

Anup


On 5/11/12 4:44 PM, "Mahurkar, Anup" <[hidden email]>
wrote:

Jaysheel,

I am working on this feature right now. Should have it ready early next
week.

Anup

On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]>
wrote:

Hey guys,
 I there a way to dynamically define queue name?  Currently queue
name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to
pass the queue name based on the pipeline I am executing, along with
h_vmem and h_stack parameters.

Thanks
Jaysheel


-----------------------------------------------------------------------
--
-----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------
--
----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



-------------------------------------------------------------------------
-----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


--------------------------------------------------------------------------
----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Jaysheel Bhavsar
In reply to this post by Mahurkar, Anup
ooop please disregard my previous email.

Jaysheel


On May 22, 2012, at 10:59 AM, Mahurkar, Anup wrote:

Can you send me the log file an the shell script?

Anup

On May 22, 2012, at 11:54 AM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I don't see any -q option in sge_job.sh script  written in mock_server_base.  In the log file I see a call to she_job.sh which looks like this

qsub -b n -C '#SGE_DIRECTIVE' /projects/bsi/RandD/ngs/ergatis/mock_server_base/working/0/4521/sge_job.sh

Which come from sge_submitter.sh from what I can tell.

Jaysheel


On May 21, 2012, at 8:22 PM, Mahurkar, Anup wrote:

Jaysheel,

The -q will be specified in the sge_job.sh script for each of the jobs. Is
it not there? If not could you send me the log file?

Anup

On 5/21/12 3:52 PM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I got the workflow-3.1.x branch and re-installed workflow.  I updated
sge_def_queue and sge_pri_queue in ${WORKFLOW_HOME}/server-conf/htc.conf
file but jobs are still being submitted to default queue.  I don't see -q
option in the qsub command in sge_submitter.sh. Is there something I
missed?

Jaysheel


On May 18, 2012, at 2:30 PM, Mahurkar, Anup wrote:

Jaysheel,

I have added the code to configure the default queue name that workflow
uses as well as specifying a queue name in the workflow that will be
honored. You will need to update the HTC and Workflow code. In the
htc.conf file specify the default queue name to be used for workflow
jobs.
If nothing is specified in the command, it will look for the property
"sge_def_queue" in the config file.

We have made slight changes to epilog, prolog, and sge_submitter scripts
as well. So your best bet is to reinstall workflow.

Anup


On 5/11/12 4:44 PM, "Mahurkar, Anup" <[hidden email]>
wrote:

Jaysheel,

I am working on this feature right now. Should have it ready early next
week.

Anup

On May 11, 2012, at 4:39 PM, "Jaysheel Bhavsar" <[hidden email]>
wrote:

Hey guys,
 I there a way to dynamically define queue name?  Currently queue
name is set in $WORKFLOW_HOME/bin/sge_submitter.sh.  I would like to
pass the queue name based on the pipeline I am executing, along with
h_vmem and h_stack parameters.

Thanks
Jaysheel


-----------------------------------------------------------------------
--
-----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------
--
----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



-------------------------------------------------------------------------
-----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond.
Discussions
will include endpoint security, mobile security and the latest in
malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


--------------------------------------------------------------------------
----
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Mahurkar, Anup
In reply to this post by Jaysheel Bhavsar
Jaysheel,

Could you confirm that this snippet of code exists in the SgeRunner class? It should be in the createSubmitFile method.

I do not see the debugging lines in the log file you sent me. Hence the question.

Anup

// If the user has specified a particular queue (Config.CmdClass), then set the queue name

// with the -q directive else use the default configured in the htc.conf

logger.debug("Queue name specified as: " + config.getCmdClass());

if (config.getCmdClass() != null ) {

p_out.println(SGE_DIRECTIVE + " -q " + config.getCmdClass());

}

else if (HTCConfig.getProperty("sge_def_queue") != null) {

p_out.println(SGE_DIRECTIVE + " -q " + HTCConfig.getProperty("sge_def_queue"));

}

else {

logger.warn("No queue specified in config file. Jobs might fail if epilog/prolog not on all queues");

}



From: Jaysheel Bhavsar <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tue, 22 May 2012 11:40:44 -0500
To: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] dynamic queue name

Hey Anup,
Attached is the pipeline.xml.log and sge_job.sh file.

Jaysheel

------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Jaysheel Bhavsar
Hey Anup,
I got latest version of tigr-htc and confirmed that the code snippet is there, compiled and replaced new jar file.  Now however, I get an error 

sge_submit result Unable to read script file because of error: ERROR! -q option must have queue list argument
In sge_submit.sh file I see SGE_DIRECTIVE -q but there is not value for queue

# config variables #SGE_DIRECTIVE -l arch=lx* #SGE_DIRECTIVE -l mem_total=1024M #SGE_DIRECTIVE -p -20 #SGE_DIRECTIVE -q #SGE_DIRECTIVE -r y #SGE_DIRECTIVE -N RunWorkflow

I have confirmed that $WORKFLOW_HOME/server-conf/htc.conf has properties 

# Configuration parameters used by the SGERunner class sge_def_queue=centos6-ergatis-test sge_pri_queue=centos6-ergatis-test


Jaysheel


On May 22, 2012, at 8:33 PM, Mahurkar, Anup wrote:

Jaysheel,

Could you confirm that this snippet of code exists in the SgeRunner class? It should be in the createSubmitFile method.

I do not see the debugging lines in the log file you sent me. Hence the question.

Anup

// If the user has specified a particular queue (Config.CmdClass), then set the queue name
// with the -q directive else use the default configured in the htc.conf
logger.debug("Queue name specified as: " + config.getCmdClass());
if (config.getCmdClass() != null ) {
p_out.println(SGE_DIRECTIVE + " -q " + config.getCmdClass());
}
else if (HTCConfig.getProperty("sge_def_queue") != null) {
p_out.println(SGE_DIRECTIVE + " -q " + HTCConfig.getProperty("sge_def_queue"));
}
else {
logger.warn("No queue specified in config file. Jobs might fail if epilog/prolog not on all queues");
}


From: Jaysheel Bhavsar <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tue, 22 May 2012 11:40:44 -0500
To: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] dynamic queue name

Hey Anup,
Attached is the pipeline.xml.log and sge_job.sh file.

Jaysheel

------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Mahurkar, Anup
Can you send me the pipeline and log file. 

Anup

On May 23, 2012, at 1:02 PM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I got latest version of tigr-htc and confirmed that the code snippet is there, compiled and replaced new jar file.  Now however, I get an error 

sge_submit result Unable to read script file because of error: ERROR! -q option must have queue list argument
In sge_submit.sh file I see SGE_DIRECTIVE -q but there is not value for queue

# config variables #SGE_DIRECTIVE -l arch=lx* #SGE_DIRECTIVE -l mem_total=1024M #SGE_DIRECTIVE -p -20 #SGE_DIRECTIVE -q #SGE_DIRECTIVE -r y #SGE_DIRECTIVE -N RunWorkflow

I have confirmed that $WORKFLOW_HOME/server-conf/htc.conf has properties 

# Configuration parameters used by the SGERunner class sge_def_queue=centos6-ergatis-test sge_pri_queue=centos6-ergatis-test


Jaysheel


On May 22, 2012, at 8:33 PM, Mahurkar, Anup wrote:

Jaysheel,

Could you confirm that this snippet of code exists in the SgeRunner class? It should be in the createSubmitFile method.

I do not see the debugging lines in the log file you sent me. Hence the question.

Anup

// If the user has specified a particular queue (Config.CmdClass), then set the queue name
// with the -q directive else use the default configured in the htc.conf
logger.debug("Queue name specified as: " + config.getCmdClass());
if (config.getCmdClass() != null ) {
p_out.println(SGE_DIRECTIVE + " -q " + config.getCmdClass());
}
else if (HTCConfig.getProperty("sge_def_queue") != null) {
p_out.println(SGE_DIRECTIVE + " -q " + HTCConfig.getProperty("sge_def_queue"));
}
else {
logger.warn("No queue specified in config file. Jobs might fail if epilog/prolog not on all queues");
}


From: Jaysheel Bhavsar <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tue, 22 May 2012 11:40:44 -0500
To: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] dynamic queue name

Hey Anup,
Attached is the pipeline.xml.log and sge_job.sh file.

Jaysheel

------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: dynamic queue name

Jaysheel Bhavsar
Hey Anup,
Attached at the files you requested.  I think the problem is in 
logger.debug("Queue name specified as: " + config.getCmdClass());
if (config.getCmdClass() != null ) {
p_out.println(SGE_DIRECTIVE + " -q " + config.getCmdClass());
}

This value config.getCmdClass isn't null.  To test I created two more parameters in htc.conf and added appropriate statements in SgeRunner, they are set correctly (see h_vmem, and h_stack in pipeline.xml.log)

Jaysheel




On May 23, 2012, at 12:25 PM, Mahurkar, Anup wrote:

Can you send me the pipeline and log file. 

Anup

On May 23, 2012, at 1:02 PM, "Jaysheel Bhavsar" <[hidden email]> wrote:

Hey Anup,
I got latest version of tigr-htc and confirmed that the code snippet is there, compiled and replaced new jar file.  Now however, I get an error 

sge_submit result Unable to read script file because of error: ERROR! -q option must have queue list argument
In sge_submit.sh file I see SGE_DIRECTIVE -q but there is not value for queue

# config variables #SGE_DIRECTIVE -l arch=lx* #SGE_DIRECTIVE -l mem_total=1024M #SGE_DIRECTIVE -p -20 #SGE_DIRECTIVE -q #SGE_DIRECTIVE -r y #SGE_DIRECTIVE -N RunWorkflow

I have confirmed that $WORKFLOW_HOME/server-conf/htc.conf has properties 

# Configuration parameters used by the SGERunner class sge_def_queue=centos6-ergatis-test sge_pri_queue=centos6-ergatis-test


Jaysheel


On May 22, 2012, at 8:33 PM, Mahurkar, Anup wrote:

Jaysheel,

Could you confirm that this snippet of code exists in the SgeRunner class? It should be in the createSubmitFile method.

I do not see the debugging lines in the log file you sent me. Hence the question.

Anup

// If the user has specified a particular queue (Config.CmdClass), then set the queue name
// with the -q directive else use the default configured in the htc.conf
logger.debug("Queue name specified as: " + config.getCmdClass());
if (config.getCmdClass() != null ) {
p_out.println(SGE_DIRECTIVE + " -q " + config.getCmdClass());
}
else if (HTCConfig.getProperty("sge_def_queue") != null) {
p_out.println(SGE_DIRECTIVE + " -q " + HTCConfig.getProperty("sge_def_queue"));
}
else {
logger.warn("No queue specified in config file. Jobs might fail if epilog/prolog not on all queues");
}


From: Jaysheel Bhavsar <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Tue, 22 May 2012 11:40:44 -0500
To: "[hidden email]" <[hidden email]>
Subject: Re: [Ergatis-users] dynamic queue name

Hey Anup,
Attached is the pipeline.xml.log and sge_job.sh file.

Jaysheel

------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users

pipeline.xml (12K) Download Attachment
pipeline.xml.log (1021K) Download Attachment