Running MAKER with MPI support and SLURM scheduler

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Running MAKER with MPI support and SLURM scheduler

Siller, Karsten H (khs3z)
Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies: VERIFIED
External Programs: VERIFIED
External C Libraries: VERIFIED
MPI SUPPORT: ENABLED
MWAS Web Interface: DISABLED
MAKER PACKAGE: CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we’re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.”  The processes appear to run ok, but we want to make sure there’s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there’s an issue either with the MAKER MPI configuration or with the scheduler.  We’re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten


Karsten Siller
University of Virginia



_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

smime.p7s (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Running MAKER with MPI support and SLURM scheduler

Carson Holt-2
Launch with OpenMPI’s mpiexec command. The slurm stun command you are using won’t launch it as expected.

—Carson

On Mar 12, 2019, at 6:39 AM, Siller, Karsten H (khs3z) <[hidden email]> wrote:

Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies: VERIFIED
External Programs: VERIFIED
External C Libraries: VERIFIED
MPI SUPPORT: ENABLED
MWAS Web Interface: DISABLED
MAKER PACKAGE: CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we’re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.”  The processes appear to run ok, but we want to make sure there’s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there’s an issue either with the MAKER MPI configuration or with the scheduler.  We’re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten


Karsten Siller
University of Virginia


_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Reply | Threaded
Open this post in threaded view
|

Re: Running MAKER with MPI support and SLURM scheduler

Siller, Karsten H (khs3z)
Ok, we’ll try it with mpiexec. Thanks for your help.

Karsten


From: Carson Holt <[hidden email]>
Sent: Tuesday, March 12, 2019 11:07 AM
To: Siller, Karsten H (khs3z)
Cc: [hidden email]
Subject: Re: [maker-devel] Running MAKER with MPI support and SLURM scheduler
 
Launch with OpenMPI’s mpiexec command. The slurm stun command you are using won’t launch it as expected.

—Carson

On Mar 12, 2019, at 6:39 AM, Siller, Karsten H (khs3z) <[hidden email]> wrote:

Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies:VERIFIED
External Programs:VERIFIED
External C Libraries:VERIFIED
MPI SUPPORT:ENABLED
MWAS Web Interface:DISABLED
MAKER PACKAGE:CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we’re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.”  The processes appear to run ok, but we want to make sure there’s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there’s an issue either with the MAKER MPI configuration or with the scheduler.  We’re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten


Karsten Siller
University of Virginia


_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org