learning pulsar for galaxy

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

learning pulsar for galaxy

Wong, Kai Leung
Hello,

We are working on a project in providing a Cloud based research platform for Health Informatics and Bioinformatics. Long story short, one of the components to be provided by the project is a Galaxy Server for our genomics researchers; and on the other hand, GWU also has a HPC system for high performance and high throughput computing.

Our Galaxy Server for this project is hosing in the Cloud (Azure) and the HPC cluster is onsite at GW. To allow a galaxy server to utilize external compute power of a HPC, galaxy has provided job runner plugins of DRMAA for job schedulers as well as Pulsar. For our situation, the HPC is a remote system w.r.t. the Galaxy server in the cloud and thus there isn't a shared file system between them. It seems to me that Pulsar would be the appropriate solution for us. Am I on the right track?

Any suggestions are much appreciated!

Best,
Adam
---
Adam K. L. Wong, PhD.
High Performance Computing Specialist for Genomics
Division of Information Technology
The George Washington University
Email: [hidden email]

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: learning pulsar for galaxy

Björn Grüning-3
Hi,

> Hello,
>
> We are working on a project in providing a Cloud based research platform
> for Health Informatics and Bioinformatics. Long story short, one of the
> components to be provided by the project is a Galaxy Server for our
> genomics researchers; and on the other hand, GWU also has a HPC system
> for high performance and high throughput computing.
>
> Our Galaxy Server for this project is hosing in the Cloud (Azure) and
> the HPC cluster is onsite at GW. To allow a galaxy server to utilize
> external compute power of a HPC, galaxy has provided job runner plugins
> of DRMAA for job schedulers as well as Pulsar. For our situation, the
> HPC is a remote system w.r.t. the Galaxy server in the cloud and thus
> there isn't a shared file system between them. It seems to me that
> Pulsar would be the appropriate solution for us. Am I on the right track?


You are on the right track, indeed! Pulsar is what you are searching for :)

Cheers,
Bjoern


> Any suggestions are much appreciated!
>
> Best,
> Adam
> ---
> Adam K. L. Wong, PhD.
> High Performance Computing Specialist for Genomics
> Division of Information Technology
> The George Washington University
> Email: [hidden email] <mailto:[hidden email]>
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>    https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>    http://galaxyproject.org/search/
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Reply | Threaded
Open this post in threaded view
|

Re: learning pulsar for galaxy

Dan Fornika
I'm also interested in a similar setup. One detail that I'd be interested in hearing about is how to minimize the cost involved in shuttling data in & out of the cloud with this sort of setup.

Dan

On Wed, Oct 3, 2018 at 9:38 AM Björn Grüning <[hidden email]> wrote:
Hi,

> Hello,
>
> We are working on a project in providing a Cloud based research platform
> for Health Informatics and Bioinformatics. Long story short, one of the
> components to be provided by the project is a Galaxy Server for our
> genomics researchers; and on the other hand, GWU also has a HPC system
> for high performance and high throughput computing.
>
> Our Galaxy Server for this project is hosing in the Cloud (Azure) and
> the HPC cluster is onsite at GW. To allow a galaxy server to utilize
> external compute power of a HPC, galaxy has provided job runner plugins
> of DRMAA for job schedulers as well as Pulsar. For our situation, the
> HPC is a remote system w.r.t. the Galaxy server in the cloud and thus
> there isn't a shared file system between them. It seems to me that
> Pulsar would be the appropriate solution for us. Am I on the right track?


You are on the right track, indeed! Pulsar is what you are searching for :)

Cheers,
Bjoern


> Any suggestions are much appreciated!
>
> Best,
> Adam
> ---
> Adam K. L. Wong, PhD.
> High Performance Computing Specialist for Genomics
> Division of Information Technology
> The George Washington University
> Email: [hidden email] <mailto:[hidden email]>
>
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>    https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>    http://galaxyproject.org/search/
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/