Quantcast

drmaa on univa grid engine

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

drmaa on univa grid engine

Matthias Bernt
Dear list,

I'm struggling to get jobs running on our cluster (UGE).

In the galaxy log I see:

galaxy.jobs.runners.drmaa DEBUG 2017-04-13 12:24:57,799 (10) submitting
file
/gpfs1/data/galaxy_server/galaxy-dev/database/jobs_directory/000/10/galaxy_10.sh

galaxy.jobs.runners.drmaa DEBUG 2017-04-13 12:24:57,799 (10)  native
specification is: -l h_rt=60 -l h_vmem=1G -pe smp 2

and I get

galaxy.jobs.runners.drmaa WARNING 2017-04-13 12:24:57,805 (10)
drmaa.Session.runJob() failed, will retry: code 17: error: no suitable
queues

But manual submission works:

qsub -l h_rt=60 -l h_vmem=1G -pe smp 2
/gpfs1/data/galaxy_server/galaxy-dev/database/jobs_directory/000/10/galaxy_10.sh
Your job 161150 ("galaxy_10.sh") has been submitted

So I checked if I can use the drmaa library to submit using the
following python script (based on the example2 included in the library):

"""
#!/usr/bin/env python

from __future__ import print_function
import drmaa
import os


def main():
     """Submit a job.
     Note, need file called sleeper.sh in current directory.
     """
     s = drmaa.Session()
     s.initialize()
     print('Creating job template')
     jt = s.createJobTemplate()
     jt.remoteCommand = os.getcwd() + '/sleeper.sh'
     jt.args = ['42','Simon says:']
     jt.joinFiles=True
     jt.nativeSpecification = "-l h_rt=60 -l h_vmem=1G -pe smp 2"

     jobid = s.runJob(jt)
     print('Your job has been submitted with id ' + jobid)

     print('Cleaning up')
     s.deleteJobTemplate(jt)
     s.exit()

if __name__=='__main__':
     main()
"""

This gives me the same error:

drmaa.errors.DeniedByDrmException: code 17: error: no suitable queues

So it seems to be a problem related to the drmaa python library.
Unfortunately I can not try without any native specs, since time, memory
and cores need to be specifies (a suitable queue is chosen automatically).

I found an earlier post with the same error (but this seemed to be
caused by colons in the file name):
http://dev.list.galaxyproject.org/Galaxy-with-Univa-Grid-Engine-UGE-instead-of-SGE-tt4657848.html#a4657952

Does anyone has suggestions for me?

Best regards,
Matthias

--

-------------------------------------------
Matthias Bernt
Bioinformatics Service
Molekulare Systembiologie (MOLSYB)
Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
Helmholtz Centre for Environmental Research GmbH - UFZ
Permoserstraße 15, 04318 Leipzig, Germany
Phone +49 341 235 482296,
[hidden email], www.ufz.de

Sitz der Gesellschaft/Registered Office: Leipzig
Registergericht/Registration Office: Amtsgericht Leipzig
Handelsregister Nr./Trade Register Nr.: B 4703
Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board:
MinDirig Wilfried Kraus
Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
Prof. Dr. Dr. h.c. Georg Teutsch
Administrative Geschäftsführerin/ Administrative Managing Director:
Prof. Dr. Heike Graßmann
-------------------------------------------


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/

smime.p7s (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: drmaa on univa grid engine

Marius van den Beek
Hi Matthias,

in addition to the link you posted there is also this here that looks similar:

Perhaps qsub is filling in a default queue, and drmaa is not doing that?
You could try adding a queue to the nativeSpecification.

Good luck,
Marius

On 13 April 2017 at 15:02, Matthias Bernt <[hidden email]> wrote:
Dear list,

I'm struggling to get jobs running on our cluster (UGE).

In the galaxy log I see:

galaxy.jobs.runners.drmaa DEBUG 2017-04-13 12:24:57,799 (10) submitting file /gpfs1/data/galaxy_server/galaxy-dev/database/jobs_directory/000/10/galaxy_10.sh

galaxy.jobs.runners.drmaa DEBUG 2017-04-13 12:24:57,799 (10)  native specification is: -l h_rt=60 -l h_vmem=1G -pe smp 2

and I get

galaxy.jobs.runners.drmaa WARNING 2017-04-13 12:24:57,805 (10) drmaa.Session.runJob() failed, will retry: code 17: error: no suitable queues

But manual submission works:

qsub -l h_rt=60 -l h_vmem=1G -pe smp 2 /gpfs1/data/galaxy_server/galaxy-dev/database/jobs_directory/000/10/galaxy_10.sh
Your job 161150 ("galaxy_10.sh") has been submitted

So I checked if I can use the drmaa library to submit using the following python script (based on the example2 included in the library):

"""
#!/usr/bin/env python

from __future__ import print_function
import drmaa
import os


def main():
    """Submit a job.
    Note, need file called sleeper.sh in current directory.
    """
    s = drmaa.Session()
    s.initialize()
    print('Creating job template')
    jt = s.createJobTemplate()
    jt.remoteCommand = os.getcwd() + '/sleeper.sh'
    jt.args = ['42','Simon says:']
    jt.joinFiles=True
    jt.nativeSpecification = "-l h_rt=60 -l h_vmem=1G -pe smp 2"

    jobid = s.runJob(jt)
    print('Your job has been submitted with id ' + jobid)

    print('Cleaning up')
    s.deleteJobTemplate(jt)
    s.exit()

if __name__=='__main__':
    main()
"""

This gives me the same error:

drmaa.errors.DeniedByDrmException: code 17: error: no suitable queues

So it seems to be a problem related to the drmaa python library. Unfortunately I can not try without any native specs, since time, memory and cores need to be specifies (a suitable queue is chosen automatically).

I found an earlier post with the same error (but this seemed to be caused by colons in the file name): http://dev.list.galaxyproject.org/Galaxy-with-Univa-Grid-Engine-UGE-instead-of-SGE-tt4657848.html#a4657952

Does anyone has suggestions for me?

Best regards,
Matthias

--

-------------------------------------------
Matthias Bernt
Bioinformatics Service
Molekulare Systembiologie (MOLSYB)
Helmholtz-Zentrum für Umweltforschung GmbH - UFZ/
Helmholtz Centre for Environmental Research GmbH - UFZ
Permoserstraße 15, 04318 Leipzig, Germany
Phone <a href="tel:%2B49%20341%20235%20482296" value="+49341235482296" target="_blank">+49 341 235 482296,
[hidden email], www.ufz.de

Sitz der Gesellschaft/Registered Office: Leipzig
Registergericht/Registration Office: Amtsgericht Leipzig
Handelsregister Nr./Trade Register Nr.: B 4703
Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: MinDirig Wilfried Kraus
Wissenschaftlicher Geschäftsführer/Scientific Managing Director:
Prof. Dr. Dr. h.c. Georg Teutsch
Administrative Geschäftsführerin/ Administrative Managing Director:
Prof. Dr. Heike Graßmann
-------------------------------------------


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/
Loading...