[galaxy-dev] learning pulsar for galaxy

[Wong, Kai Leung]

Hello,

We are working on a project in providing a Cloud based research platform
for Health Informatics and Bioinformatics. Long story short, one of the
components to be provided by the project is a Galaxy Server for our genomics
researchers; and on the other hand, GWU also has a HPC system for high
performance and high throughput computing.

Our Galaxy Server for this project is hosing in the Cloud (Azure) and the
HPC cluster is onsite at GW. To allow a galaxy server to utilize external
compute power of a HPC, galaxy has provided job runner plugins of DRMAA for
job schedulers as well as Pulsar. For our situation, the HPC is a remote
system w.r.t. the Galaxy server in the cloud and thus there isn't a shared
file system between them. It seems to me that Pulsar would be the
appropriate solution for us. Am I on the right track?

Any suggestions are much appreciated!

Best,
Adam
---
Adam K. L. Wong, PhD.
High Performance Computing Specialist for Genomics
Division of Information Technology
The George Washington University
Email: ***@gwu.edu

[Björn Grüning]

Hi,

Post by Wong, Kai Leung
Hello,
We are working on a project in providing a Cloud based research platform
for Health Informatics and Bioinformatics. Long story short, one of the
components to be provided by the project is a Galaxy Server for our
genomics researchers; and on the other hand, GWU also has a HPC system
for high performance and high throughput computing.
Our Galaxy Server for this project is hosing in the Cloud (Azure) and
the HPC cluster is onsite at GW. To allow a galaxy server to utilize
external compute power of a HPC, galaxy has provided job runner plugins
of DRMAA for job schedulers as well as Pulsar. For our situation, the
HPC is a remote system w.r.t. the Galaxy server in the cloud and thus
there isn't a shared file system between them. It seems to me that
Pulsar would be the appropriate solution for us. Am I on the right track?

You are on the right track, indeed! Pulsar is what you are searching for :)

Cheers,
Bjoern

Post by Wong, Kai Leung
Any suggestions are much appreciated!
Best,
Adam
---
Adam K. L. Wong, PhD.
High Performance Computing Specialist for Genomics
Division of Information Technology
The George Washington University
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
https://lists.galaxyproject.org/
http://galaxyproject.org/search/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified

[Dan Fornika]

I'm also interested in a similar setup. One detail that I'd be interested
in hearing about is how to minimize the cost involved in shuttling data in
& out of the cloud with this sort of setup.

Dan

Post by Björn Grüning
Hi,

Post by Wong, Kai Leung
Hello,
We are working on a project in providing a Cloud based research platform
for Health Informatics and Bioinformatics. Long story short, one of the
components to be provided by the project is a Galaxy Server for our
genomics researchers; and on the other hand, GWU also has a HPC system
for high performance and high throughput computing.
Our Galaxy Server for this project is hosing in the Cloud (Azure) and
the HPC cluster is onsite at GW. To allow a galaxy server to utilize
external compute power of a HPC, galaxy has provided job runner plugins
of DRMAA for job schedulers as well as Pulsar. For our situation, the
HPC is a remote system w.r.t. the Galaxy server in the cloud and thus
there isn't a shared file system between them. It seems to me that
Pulsar would be the appropriate solution for us. Am I on the right track?

You are on the right track, indeed! Pulsar is what you are searching for :)
Cheers,
Bjoern

Post by Wong, Kai Leung
Any suggestions are much appreciated!
Best,
Adam
---
Adam K. L. Wong, PhD.
High Performance Computing Specialist for Genomics
Division of Information Technology
The George Washington University
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
https://lists.galaxyproject.org/
http://galaxyproject.org/search/

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
https://lists.galaxyproject.org/
http://galaxyproject.org/search/