|
Hello there,
I'm currently trying to connect ergatis and workflow with the SGE. My test pipeline is an exact copy of the pipeline described in the "pipeline creation tutorial" (formatdb and blastall). The problem occurred during the blastall run. After the job is distributed on the cluster the job runs, finishes and creates all output_data except the bsml.list, raw.list and btab.list. The program doesn't finish and runs on and on and on... The entry.log contains the following: I~~~ wrapper script starting job I~~~ Job Process id is 15233 F~~~623~~~106~~~Mon Sep 13 12:01:02 CEST 2010~~~command finished~~~0 The stdout and stderr of ergatis contains the following: The Workflow is invoked. View Workflow Monitor to check the progress. log4j:WARN No appenders could be found for logger (org.tigr.antware.shared.util.AppConfig). log4j:WARN Please initialize the log4j system properly. WARN 12:01:02:957 [Thread: (1) Monitor Command 623] Command taskFinished:1065 Task 0 finished but it was never started... The SGE command qacct tells me the job finished without an error. It is obvious that the process officially never started but the data is there. Furthermore the program seems to think it didn't finish. Thanks for your help/advices, Richard Schobesberger ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
|
It appears that the SGE epilog and prolog scripts have not been installed or
installed incorrectly. These scripts write the start and finish lines on the event.log file and workflow is detecting that these are missing. Can you check the SGE configuration to see if these are installed? On 9/13/10 8:24 AM, "Schobesberger Richard - S0910595006" <[hidden email]> wrote: > Hello there, > > I'm currently trying to connect ergatis and workflow with the SGE. > > My test pipeline is an exact copy of the pipeline described in the "pipeline > creation tutorial" (formatdb and blastall). > The problem occurred during the blastall run. After the job is distributed on > the cluster the job runs, finishes and creates all output_data except the > bsml.list, raw.list and btab.list. > The program doesn't finish and runs on and on and on... > The entry.log contains the following: > > I~~~ wrapper script starting job > I~~~ Job Process id is 15233 > F~~~623~~~106~~~Mon Sep 13 12:01:02 CEST 2010~~~command finished~~~0 > > The stdout and stderr of ergatis contains the following: > > The Workflow is invoked. View Workflow Monitor to check the progress. > log4j:WARN No appenders could be found for logger > (org.tigr.antware.shared.util.AppConfig). > log4j:WARN Please initialize the log4j system properly. > WARN 12:01:02:957 [Thread: (1) Monitor Command 623] Command taskFinished:1065 > Task 0 finished but it was never started... > > The SGE command qacct tells me the job finished without an error. > > It is obvious that the process officially never started but the data is there. > Furthermore the program seems to think it didn't finish. > > Thanks for your help/advices, > Richard Schobesberger > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > Ergatis-users mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/ergatis-users ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
|
In reply to this post by Schobesberger Richard - S0910595006
Hi,
Thanks, you were correct, I forgot to put the prolog and epilog path into the queue. The entry.log now contains this: I~~~prolog starting I~~~htc id sge id[.task id] date message hostname S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on for ~~~ I~~~prolog ending I~~~ wrapper script starting job I~~~ Job Process id is 16208 F~~~769~~~110~~~Mon Sep 13 15:27:47 CEST 2010~~~command finished~~~0 I~~~epilog starting I~~~htc id sge id[.task id] date message hostname T~~~769~~~~~~Mon Sep 13 15:27:47 CEST 2010~~~job finished on for ~~~ I~~~epilog ending But the problem remains that the program doesn't finish running The ergatis still prints: The Workflow is invoked. View Workflow Monitor to check the progress. log4j:WARN No appenders could be found for logger (org.tigr.antware.shared.util.AppConfig). log4j:WARN Please initialize the log4j system properly. WARN 15:27:49:246 [Thread: (1) Monitor Command 769] Command taskFinished:1065 Task 0 finished but it was never started... Probably its because the program doesn't know on which host the script is running since the prolog file wrote: S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on (here should be the hostname) for ~~~ Thanks for your help/advice, Richard Schobesberger >It appears that the SGE epilog and prolog scripts have not been installed or >installed incorrectly. These scripts write the start and finish lines on the >event.log file and workflow is detecting that these are missing. Can you >check the SGE configuration to see if these are installed? ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
|
In reply to this post by Schobesberger Richard - S0910595006
Hi,
Thanks, you were correct, I forgot to put the prolog and epilog path into the queue. The entry.log now contains this: I~~~prolog starting I~~~htc id sge id[.task id] date message hostname S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on for ~~~ I~~~prolog ending I~~~ wrapper script starting job I~~~ Job Process id is 16208 F~~~769~~~110~~~Mon Sep 13 15:27:47 CEST 2010~~~command finished~~~0 I~~~epilog starting I~~~htc id sge id[.task id] date message hostname T~~~769~~~~~~Mon Sep 13 15:27:47 CEST 2010~~~job finished on for ~~~ I~~~epilog ending But the problem remains that the program doesn't finish running The ergatis still prints: The Workflow is invoked. View Workflow Monitor to check the progress. log4j:WARN No appenders could be found for logger (org.tigr.antware.shared.util.AppConfig). log4j:WARN Please initialize the log4j system properly. WARN 15:27:49:246 [Thread: (1) Monitor Command 769] Command taskFinished:1065 Task 0 finished but it was never started... Probably its because the program doesn't know on which host the script is running since the prolog file wrote: S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on (here should be the hostname) for ~~~ Thanks for your help/advice, Richard Schobesberger >It appears that the SGE epilog and prolog scripts have not been installed or >installed incorrectly. These scripts write the start and finish lines on the >event.log file and workflow is detecting that these are missing. Can you >check the SGE configuration to see if these are installed? ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
|
In reply to this post by Schobesberger Richard - S0910595006
Right. When you install prolog you have to specify the parameters passed to
the script and one of them is the HOSTNAME. I am not near the documentation, but could you check? On 9/13/10 9:38 AM, "Schobesberger Richard - S0910595006" <[hidden email]> wrote: > Hi, > > Thanks, you were correct, I forgot to put the prolog and epilog path into the > queue. > The entry.log now contains this: > I~~~prolog starting > I~~~htc id sge id[.task id] date message hostname > S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on for ~~~ > I~~~prolog ending > I~~~ wrapper script starting job > I~~~ Job Process id is 16208 > F~~~769~~~110~~~Mon Sep 13 15:27:47 CEST 2010~~~command finished~~~0 > I~~~epilog starting > I~~~htc id sge id[.task id] date message hostname > T~~~769~~~~~~Mon Sep 13 15:27:47 CEST 2010~~~job finished on for ~~~ > I~~~epilog ending > > But the problem remains that the program doesn't finish running > The ergatis still prints: > > The Workflow is invoked. View Workflow Monitor to check the progress. > log4j:WARN No appenders could be found for logger > (org.tigr.antware.shared.util.AppConfig). > log4j:WARN Please initialize the log4j system properly. > WARN 15:27:49:246 [Thread: (1) Monitor Command 769] Command taskFinished:1065 > Task 0 finished but it was never started... > > Probably its because the program doesn't know on which host the script is > running since the prolog file wrote: > S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on (here should be > the hostname) for ~~~ > > Thanks for your help/advice, > Richard Schobesberger > > >> It appears that the SGE epilog and prolog scripts have not been installed or >> installed incorrectly. These scripts write the start and finish lines on the >> event.log file and workflow is detecting that these are missing. Can you >> check the SGE configuration to see if these are installed? > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > Ergatis-users mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/ergatis-users ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
|
Hi,
The prolog script of workflow has a list of parameters and one of them is the hostname. But I'm not sure which script or program calls prolog in the end. Is this done by SGE? Richard ________________________________________ Von: Mahurkar, Anup [[hidden email]] Gesendet: Montag, 13. September 2010 16:04 An: [hidden email] Betreff: Re: [Ergatis-users] Task finished but it was never started Right. When you install prolog you have to specify the parameters passed to the script and one of them is the HOSTNAME. I am not near the documentation, but could you check? On 9/13/10 9:38 AM, "Schobesberger Richard - S0910595006" <[hidden email]> wrote: > Hi, > > Thanks, you were correct, I forgot to put the prolog and epilog path into the > queue. > The entry.log now contains this: > I~~~prolog starting > I~~~htc id sge id[.task id] date message hostname > S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on for ~~~ > I~~~prolog ending > I~~~ wrapper script starting job > I~~~ Job Process id is 16208 > F~~~769~~~110~~~Mon Sep 13 15:27:47 CEST 2010~~~command finished~~~0 > I~~~epilog starting > I~~~htc id sge id[.task id] date message hostname > T~~~769~~~~~~Mon Sep 13 15:27:47 CEST 2010~~~job finished on for ~~~ > I~~~epilog ending > > But the problem remains that the program doesn't finish running > The ergatis still prints: > > The Workflow is invoked. View Workflow Monitor to check the progress. > log4j:WARN No appenders could be found for logger > (org.tigr.antware.shared.util.AppConfig). > log4j:WARN Please initialize the log4j system properly. > WARN 15:27:49:246 [Thread: (1) Monitor Command 769] Command taskFinished:1065 > Task 0 finished but it was never started... > > Probably its because the program doesn't know on which host the script is > running since the prolog file wrote: > S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on (here should be > the hostname) for ~~~ > > Thanks for your help/advice, > Richard Schobesberger > > >> It appears that the SGE epilog and prolog scripts have not been installed or >> installed incorrectly. These scripts write the start and finish lines on the >> event.log file and workflow is detecting that these are missing. Can you >> check the SGE configuration to see if these are installed? > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > Ergatis-users mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/ergatis-users ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
|
Yes, it is called by SGE.
On 9/13/10 10:29 AM, "Schobesberger Richard - S0910595006" <[hidden email]> wrote: > Hi, > > The prolog script of workflow has a list of parameters and one of them is the > hostname. > But I'm not sure which script or program calls prolog in the end. > Is this done by SGE? > > Richard > ________________________________________ > Von: Mahurkar, Anup [[hidden email]] > Gesendet: Montag, 13. September 2010 16:04 > An: [hidden email] > Betreff: Re: [Ergatis-users] Task finished but it was never started > > Right. When you install prolog you have to specify the parameters passed to > the script and one of them is the HOSTNAME. I am not near the documentation, > but could you check? > > > On 9/13/10 9:38 AM, "Schobesberger Richard - S0910595006" > <[hidden email]> wrote: > >> Hi, >> >> Thanks, you were correct, I forgot to put the prolog and epilog path into the >> queue. >> The entry.log now contains this: >> I~~~prolog starting >> I~~~htc id sge id[.task id] date message hostname >> S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on for ~~~ >> I~~~prolog ending >> I~~~ wrapper script starting job >> I~~~ Job Process id is 16208 >> F~~~769~~~110~~~Mon Sep 13 15:27:47 CEST 2010~~~command finished~~~0 >> I~~~epilog starting >> I~~~htc id sge id[.task id] date message hostname >> T~~~769~~~~~~Mon Sep 13 15:27:47 CEST 2010~~~job finished on for ~~~ >> I~~~epilog ending >> >> But the problem remains that the program doesn't finish running >> The ergatis still prints: >> >> The Workflow is invoked. View Workflow Monitor to check the progress. >> log4j:WARN No appenders could be found for logger >> (org.tigr.antware.shared.util.AppConfig). >> log4j:WARN Please initialize the log4j system properly. >> WARN 15:27:49:246 [Thread: (1) Monitor Command 769] Command taskFinished:1065 >> Task 0 finished but it was never started... >> >> Probably its because the program doesn't know on which host the script is >> running since the prolog file wrote: >> S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on (here should be >> the hostname) for ~~~ >> >> Thanks for your help/advice, >> Richard Schobesberger >> >> >>> It appears that the SGE epilog and prolog scripts have not been installed or >>> installed incorrectly. These scripts write the start and finish lines on the >>> event.log file and workflow is detecting that these are missing. Can you >>> check the SGE configuration to see if these are installed? >> >> - >> Start uncovering the many advantages of virtual appliances >> and start using them to simplify application deployment and >> accelerate your shift to cloud computing >> http://p.sf.net/sfu/novell-sfdev2dev >> _______________________________________________ >> Ergatis-users mailing list >> [hidden email] >> https://lists.sourceforge.net/lists/listinfo/ergatis-users > > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > Ergatis-users mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/ergatis-users > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > Ergatis-users mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/ergatis-users ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
|
Thank you very much! It works now!
Richard ________________________________________ Von: Mahurkar, Anup [[hidden email]] Gesendet: Montag, 13. September 2010 16:38 An: [hidden email] Betreff: Re: [Ergatis-users] Task finished but it was never started Yes, it is called by SGE. On 9/13/10 10:29 AM, "Schobesberger Richard - S0910595006" <[hidden email]> wrote: > Hi, > > The prolog script of workflow has a list of parameters and one of them is the > hostname. > But I'm not sure which script or program calls prolog in the end. > Is this done by SGE? > > Richard > ________________________________________ > Von: Mahurkar, Anup [[hidden email]] > Gesendet: Montag, 13. September 2010 16:04 > An: [hidden email] > Betreff: Re: [Ergatis-users] Task finished but it was never started > > Right. When you install prolog you have to specify the parameters passed to > the script and one of them is the HOSTNAME. I am not near the documentation, > but could you check? > > > On 9/13/10 9:38 AM, "Schobesberger Richard - S0910595006" > <[hidden email]> wrote: > >> Hi, >> >> Thanks, you were correct, I forgot to put the prolog and epilog path into the >> queue. >> The entry.log now contains this: >> I~~~prolog starting >> I~~~htc id sge id[.task id] date message hostname >> S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on for ~~~ >> I~~~prolog ending >> I~~~ wrapper script starting job >> I~~~ Job Process id is 16208 >> F~~~769~~~110~~~Mon Sep 13 15:27:47 CEST 2010~~~command finished~~~0 >> I~~~epilog starting >> I~~~htc id sge id[.task id] date message hostname >> T~~~769~~~~~~Mon Sep 13 15:27:47 CEST 2010~~~job finished on for ~~~ >> I~~~epilog ending >> >> But the problem remains that the program doesn't finish running >> The ergatis still prints: >> >> The Workflow is invoked. View Workflow Monitor to check the progress. >> log4j:WARN No appenders could be found for logger >> (org.tigr.antware.shared.util.AppConfig). >> log4j:WARN Please initialize the log4j system properly. >> WARN 15:27:49:246 [Thread: (1) Monitor Command 769] Command taskFinished:1065 >> Task 0 finished but it was never started... >> >> Probably its because the program doesn't know on which host the script is >> running since the prolog file wrote: >> S~~~769~~~~~~Mon Sep 13 15:27:37 CEST 2010~~~job started on (here should be >> the hostname) for ~~~ >> >> Thanks for your help/advice, >> Richard Schobesberger >> >> >>> It appears that the SGE epilog and prolog scripts have not been installed or >>> installed incorrectly. These scripts write the start and finish lines on the >>> event.log file and workflow is detecting that these are missing. Can you >>> check the SGE configuration to see if these are installed? >> >> - >> Start uncovering the many advantages of virtual appliances >> and start using them to simplify application deployment and >> accelerate your shift to cloud computing >> http://p.sf.net/sfu/novell-sfdev2dev >> _______________________________________________ >> Ergatis-users mailing list >> [hidden email] >> https://lists.sourceforge.net/lists/listinfo/ergatis-users > > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > Ergatis-users mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/ergatis-users > > ------------------------------------------------------------------------------ > Start uncovering the many advantages of virtual appliances > and start using them to simplify application deployment and > accelerate your shift to cloud computing > http://p.sf.net/sfu/novell-sfdev2dev > _______________________________________________ > Ergatis-users mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/ergatis-users ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
|
Hello everybody,
First question: Is it possible to clone a pipeline without adding an empty identation ? Other question: When the "kill" button just do nothing (pipeline still alive on the cluster) where should I investigate ? As far as I know, it never worked on our cluster. Thanks, Nallias ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
|
Nallias -
That nesting of a cloned pipeline within a serial set is an artifact that needs to be fixed. It's there because you could conceivably add your pipeline and then other components (or pipelines) afterwards, and all of these would need to be contained in some sort of set, but it doesn't make sense for it to stay that way if there is only one thing inside of the set. When you hit 'kill' to end your pipeline there should be a pipeline.xml.kill.log in the same location as your pipeline.xml file. Can you see what's there? Joshua
On Mon, Sep 13, 2010 at 11:08 AM, Nicolas Allias <[hidden email]> wrote: Hello everybody, ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
|
Thanks for the quick replies !
You're right, there is no kill.log in this directory: $ ls workflow/runtime/pipeline/1 config_run.default.config config_run.list.config pipeline.layout pipeline.xml pipeline.xml.comment pipeline.xml.log pipeline.xml.run.out Next step ? Joshua Orvis wrote: > Nallias - > > That nesting of a cloned pipeline within a serial set is an artifact > that needs to be fixed. It's there because you could conceivably add > your pipeline and then other components (or pipelines) afterwards, and > all of these would need to be contained in some sort of set, but it > doesn't make sense for it to stay that way if there is only one thing > inside of the set. > > When you hit 'kill' to end your pipeline there should be a > pipeline.xml.kill.log in the same location as your pipeline.xml file. > Can you see what's there? > > Joshua > > > > On Mon, Sep 13, 2010 at 11:08 AM, Nicolas Allias > <[hidden email] > <mailto:[hidden email]>> wrote: > > Hello everybody, > > First question: > Is it possible to clone a pipeline without adding an empty > identation ? > > Other question: > When the "kill" button just do nothing (pipeline still alive on the > cluster) where should I investigate ? > As far as I know, it never worked on our cluster. > > Thanks, > > Nallias > ------------------------------------------------------------------------------ Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing http://p.sf.net/sfu/novell-sfdev2dev _______________________________________________ Ergatis-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/ergatis-users |
| Powered by Nabble | Edit this page |
