[Ergatis-devel] "submit_pipelines_as_jobs" feedback

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[Ergatis-devel] "submit_pipelines_as_jobs" feedback

Chris Hemmerich

I've been playing with this today and it is a great feature for us where
ergatis is running on a different OS than our compute nodes.

I ran across two things I had to hack around to get it to run at our site. If
you can help me come up with good solutions, I'm happy to implement and commit
them.

1) qsub is not being called with a full path. Can the full path be pieced
     together from existing configuration information? Or should it be a new
     ergatis.config entry?

2) A working directory isn't passed to qsub so our home directory was
     being flooded with logs and tmp files. I just changed the call in
     Pipeline.pm to use the created $run_dir, but I'm not sure this is a
     good or universal solution.

  $runprefix = "qsub -wd $run_dir ";

Thanks,
   Chris

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
Ergatis-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-devel
Reply | Threaded
Open this post in threaded view
|

Re: [Ergatis-devel] "submit_pipelines_as_jobs" feedback

Joshua Orvis
Chris -

These are good suggestions, and I've incorporated them into SVN revisions 6812 and 6813.  There is now a new configuration option [grid]:sge_qsub in the ergatis.ini file.

Joshua



On Wed, Feb 23, 2011 at 11:55 PM, Chris Hemmerich <[hidden email]> wrote:

I've been playing with this today and it is a great feature for us where
ergatis is running on a different OS than our compute nodes.

I ran across two things I had to hack around to get it to run at our site. If
you can help me come up with good solutions, I'm happy to implement and commit
them.

1) qsub is not being called with a full path. Can the full path be pieced
    together from existing configuration information? Or should it be a new
    ergatis.config entry?

2) A working directory isn't passed to qsub so our home directory was
    being flooded with logs and tmp files. I just changed the call in
    Pipeline.pm to use the created $run_dir, but I'm not sure this is a
    good or universal solution.

 $runprefix = "qsub -wd $run_dir ";

Thanks,
  Chris

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev
_______________________________________________
Ergatis-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-devel


------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in
Real-Time with Splunk. Collect, index and harness all the fast moving IT data
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
Ergatis-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/ergatis-devel