features question

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

features question

Steve Fischer
Hi-

I have had a look at the ergatis paper and the docs available on the
sourceforge site.  I am trying to see if ergatis has some features i am
intersted in.

does ergatis have these features:
   1) undo.  from the paper it seems i can point to a spot in the graph
and ask ergatis to failure recover from that spot.  does ergatis undo
the work back up to that spot, or does it just re-run forward from there?

   2) template nesting.  seems like i can define a template, which is a
reusable graph.  can i nest these within larger graphs, like
subroutines?  are they parameterized?

   3) changing the graph.  if i have a long-running workflow, can i
pause it and add/delete/change components (as long as they are not
completed), and then resume?

   4) fixing a component.  if a component fails because of an error in
its implementation, can i correct its implementation and have the
corrected version seen by the currently running workflow?

thanks,
steve

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: features question

Joshua Orvis
Steve -

Good to hear from you again.  I'll try to answer each of your questions below:

1)  As you mentioned, Ergatis does have failure recovery from any spot where an error occurs.  But any 'undo' functionality as you described, where you want to back up to a certain spot and 'undo' commands that were reported as successful is currently limited.  In a pipeline you can 'reset' a component so that it will start over completely whether the steps were reported as successful or not, but that 'undo' functionality is then limited in scope to any given component withi a pipeline, not any given particular step.  (At least with the interface.  See Q3 response for more info on how you actually CAN 'undo' any steps you want.)

2)  Any level of nesting of is allowed within the pipelines.  There could be some tricky bits about your grid configuration to allow this to happen completely, but nothing within the Workflow XML that would prevent it.  For example, if you're using SunGrid Engine and place a job on a node and then want that node to launch other jobs itself you'd have to configure all your execute nodes to also be submit hosts, which I don't think is done that often.

3) Here I would say this is possible for developers, but not something built into the web interface.  You can stop a pipeline and then manipulate the Workflow XML in any manner you choose before restarting it (even making changes to 'completed' sections and marking them for re-execution).  I know of at least one group who routinely stops their pipelines when necessary, refactors the XML describing them, and restarts as necessary because of specific needs of their architecture.

4)  This question wasn't completely clear to me.  If you have a running workflow and one component fails because of a design problem, you can certainly fix the component, but any still-running workflow won't see the update until you restart it.  It doesn't constantly check back on itself to restart failed steps without an explicit command to restart. 

To be more specific, when you configure a pipeline using the interface and hit 'Run' some code reads your configuration, your input files, and writes out the pipeline XML to be run.  If the error in your component implementation involves, say, an error within a Perl script you wrote, you could fix the error, click 're-run' on the interface and it would try those steps again.  If, on the other hand, your error was that you specified incorrect options to your Perl script then that mistake is represented within the XML defining the work to be done.  In that case the XML needs to be corrected.  This could be done programatically, or you can fix the component implementation and then use the 'reset component' button on the interface.


Maybe this was too much information, but hopefully I covered what you needed. 

Joshua




On Thu, Jun 17, 2010 at 1:24 PM, Steve Fischer <[hidden email]> wrote:
Hi-

I have had a look at the ergatis paper and the docs available on the
sourceforge site.  I am trying to see if ergatis has some features i am
intersted in.

does ergatis have these features:
  1) undo.  from the paper it seems i can point to a spot in the graph
and ask ergatis to failure recover from that spot.  does ergatis undo
the work back up to that spot, or does it just re-run forward from there?

  2) template nesting.  seems like i can define a template, which is a
reusable graph.  can i nest these within larger graphs, like
subroutines?  are they parameterized?

  3) changing the graph.  if i have a long-running workflow, can i
pause it and add/delete/change components (as long as they are not
completed), and then resume?

  4) fixing a component.  if a component fails because of an error in
its implementation, can i correct its implementation and have the
corrected version seen by the currently running workflow?

thanks,
steve

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit.  See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Ergatis at BOSC/ISMB

Chris Hemmerich

Will any Ergatis or CloVR developers be at BOSC/ISMB or the fall GMOD
meeting? I'd also like to share experiences with any other users, and in
particular I'd like to pick the brains of anyone managing large Ergatis
installations.

Cheers,
  Chris

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Ergatis at BOSC/ISMB

Joshua Orvis
Chris -

I'll be at BOSC and in Boston starting Thursday morning and until late Saturday night.  I'd love to get together sometime.  I'll e-mail you my contact info.

Joshua


On Wed, Jul 7, 2010 at 9:50 AM, Chris Hemmerich <[hidden email]> wrote:

Will any Ergatis or CloVR developers be at BOSC/ISMB or the fall GMOD
meeting? I'd also like to share experiences with any other users, and in
particular I'd like to pick the brains of anyone managing large Ergatis
installations.

Cheers,
 Chris

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users


------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Ergatis at BOSC/ISMB

Sam Angiuoli
In reply to this post by Chris Hemmerich
Chris and Josh,
We should certainly meet up.  I'll be there at the Biosharing SIG and
the rest of ISMB, including the cloud and bioinformatics core
workshops on Sunday. CloVR project will have a poster (Poster # U64)
on Monday.  Malcolm is another CloVR/Ergatis developer that will be
there.

-Sam

On Wed, Jul 7, 2010 at 10:50 AM, Chris Hemmerich
<[hidden email]> wrote:

>
> Will any Ergatis or CloVR developers be at BOSC/ISMB or the fall GMOD
> meeting? I'd also like to share experiences with any other users, and in
> particular I'd like to pick the brains of anyone managing large Ergatis
> installations.
>
> Cheers,
>  Chris
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Ergatis at BOSC/ISMB

Dave Clements
Hello all,

I'd like to point out that Chris is presenting a poster at ISMB on ISGA, a pipeline service that uses Ergatis:

  I10: ISGA - An Intuitive Web Server for Prokaryotic Genome Annotation and Other Analyses,
  Monday 12:30-2:40.

And there is a whole bunch of other GMOD related content too.  See http://gmod.org/wiki/ISMB_2010.

Dave C.


On Wed, Jul 7, 2010 at 8:19 AM, Sam Angiuoli <[hidden email]> wrote:
Chris and Josh,
We should certainly meet up.  I'll be there at the Biosharing SIG and
the rest of ISMB, including the cloud and bioinformatics core
workshops on Sunday. CloVR project will have a poster (Poster # U64)
on Monday.  Malcolm is another CloVR/Ergatis developer that will be
there.

-Sam

On Wed, Jul 7, 2010 at 10:50 AM, Chris Hemmerich
<[hidden email]> wrote:
>
> Will any Ergatis or CloVR developers be at BOSC/ISMB or the fall GMOD
> meeting? I'd also like to share experiences with any other users, and in
> particular I'd like to pick the brains of anyone managing large Ergatis
> installations.
>
> Cheers,
>  Chris
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users



--
http://gmod.org/wiki/GMOD_News
http://gmod.org/wiki/Calendar
http://gmod.org/wiki/Help_Desk_Feedback

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users
Reply | Threaded
Open this post in threaded view
|

Re: Ergatis at BOSC/ISMB

Chris Hemmerich
In reply to this post by Sam Angiuoli

That sounds like a good idea. I'm getting into Boston late tonight, lunch
or dinner on Friday or Saturday sound good?

  Chris

On Wed, 7 Jul 2010, Sam Angiuoli wrote:

> Chris and Josh,
> We should certainly meet up.  I'll be there at the Biosharing SIG and
> the rest of ISMB, including the cloud and bioinformatics core
> workshops on Sunday. CloVR project will have a poster (Poster # U64)
> on Monday.  Malcolm is another CloVR/Ergatis developer that will be
> there.
>
> -Sam
>
> On Wed, Jul 7, 2010 at 10:50 AM, Chris Hemmerich
> <[hidden email]> wrote:
>>
>> Will any Ergatis or CloVR developers be at BOSC/ISMB or the fall GMOD
>> meeting? I'd also like to share experiences with any other users, and in
>> particular I'd like to pick the brains of anyone managing large Ergatis
>> installations.
>>
>> Cheers,
>>  Chris
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by Sprint
>> What will you do first with EVO, the first 4G phone?
>> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
>> _______________________________________________
>> Ergatis-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Ergatis-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/ergatis-users
>
------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Ergatis-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-users