some suggestion

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

some suggestion

zhushilin@frasergen.com
Dear developer,

It seems that MAKER can only run in the general hard disk which support structrued data, as SQLite was used.
When running in lustre filesystem, we got  I/O error and nothing was written to .db file which saves the gff information.

Maybe the best way is to check the filesystem automatically and give the different strategy to store the information in gff files.

Best wishes

Shilin Zhu   R&D director

DEPT. of Bioinformatics

Wuhan Frasergen Bioinformatics Co., Ltd

B8 buildingBiolake666 Gaoxin RoadWuhan East Lake High-tech ZoneWuhan 430075China

T: 027-87224705│M: +86 18502745140 

F: 027-87224785│E: [hidden email]

W: http://www.frasergen.com


Disclaimer

This e-mail is intended to be used only by persons entitled to receive such information and may contain information that is confidential, proprietary, and/or legally privileged. If you are not the intended recipient, you are hereby  notified that any use, retention, disclosure, dissemination, copying, or taking any other action in reliance on contents of this e-mail is prohibited. If you have received this e-mail in error, please immediately contact the sender and delete the e-mail from your mailbox or any other storage mechanism. Thank you!


_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: some suggestion

Carson Holt-2
The system we commonly use with MAKER is Lustre and we get no issues. We also commonly run MAKER on TACC which uses Lustre for all it’s file systems. So there is no Lustre limitation. MAKER does require that each node also have a local temporary directory for some operations which can generate high IOPS or require that traditional flock support (cannot be guaranteed on some NFS systems). These operations occur in the location specified by TMP= in the control file. Perhaps you are attempting to set your TMP value in the control files to a Lustre space which can overload the MDS (metadata server) used by Lustre. Make sure you do not set TMP to a shared location. Your working directory can be a shared Lustre space and result files will be stored there, but IO operations that are not safe for shared spaces will occur in TMP, and TMP must be set to a local storage location (usually /tmp).

--Carson



On Jul 12, 2017, at 12:19 AM, [hidden email] wrote:

Dear developer,

It seems that MAKER can only run in the general hard disk which support structrued data, as SQLite was used.
When running in lustre filesystem, we got  I/O error and nothing was written to .db file which saves the gff information.

Maybe the best way is to check the filesystem automatically and give the different strategy to store the information in gff files.

Best wishes

Shilin Zhu   R&D director
DEPT. of Bioinformatics
Wuhan Frasergen Bioinformatics Co., Ltd
B8 buildingBiolake666 Gaoxin RoadWuhan East Lake High-tech ZoneWuhan 430075China
T: 027-87224705│M: +86 18502745140 
F: 027-87224785│E: [hidden email]
W: <a href="x-msg://93/www.frasergen.com" class="">http://www.frasergen.com

Disclaimer
This e-mail is intended to be used only by persons entitled to receive such information and may contain information that is confidential, proprietary, and/or legally privileged. If you are not the intended recipient, you are hereby  notified that any use, retention, disclosure, dissemination, copying, or taking any other action in reliance on contents of this e-mail is prohibited. If you have received this e-mail in error, please immediately contact the sender and delete the e-mail from your mailbox or any other storage mechanism. Thank you!
_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org


_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: some suggestion

Fields, Christopher J

We’ve worked with MAKER on Lustre and GPFS w/o significant issues (we’re also in the process of setting up MAKER on cluster using Ceph).  I’ve always found the trickiest part in initial MAKER setup and testing is making sure the proper tempfile space is set (point to local disk or /dev/shm) and wrangling MPI issues.

 

chris

 

From: maker-devel <[hidden email]> on behalf of Carson Holt <[hidden email]>
Date: Tuesday, July 18, 2017 at 12:06 AM
To: "[hidden email]" <[hidden email]>
Cc: maker-devel <[hidden email]>
Subject: Re: [maker-devel] some suggestion

 

The system we commonly use with MAKER is Lustre and we get no issues. We also commonly run MAKER on TACC which uses Lustre for all it’s file systems. So there is no Lustre limitation. MAKER does require that each node also have a local temporary directory for some operations which can generate high IOPS or require that traditional flock support (cannot be guaranteed on some NFS systems). These operations occur in the location specified by TMP= in the control file. Perhaps you are attempting to set your TMP value in the control files to a Lustre space which can overload the MDS (metadata server) used by Lustre. Make sure you do not set TMP to a shared location. Your working directory can be a shared Lustre space and result files will be stored there, but IO operations that are not safe for shared spaces will occur in TMP, and TMP must be set to a local storage location (usually /tmp).

 

--Carson

 

 

 

On Jul 12, 2017, at 12:19 AM, [hidden email] wrote:

 

Dear developer,

 

It seems that MAKER can only run in the general hard disk which support structrued data, as SQLite was used.

When running in lustre filesystem, we got  I/O error and nothing was written to .db file which saves the gff information.

 

Maybe the best way is to check the filesystem automatically and give the different strategy to store the information in gff files.

 

Best wishes


Shilin Zhu   R&D director

DEPT. of Bioinformatics

Wuhan Frasergen Bioinformatics Co., Ltd

B8 buildingBiolake666 Gaoxin RoadWuhan East Lake High-tech ZoneWuhan 430075China

T: 027-87224705│M: +86 18502745140 

F: 027-87224785│E: [hidden email]

W: <a href="x-msg://93/www.frasergen.com">http://www.frasergen.com


Disclaimer

This e-mail is intended to be used only by persons entitled to receive such information and may contain information that is confidential, proprietary, and/or legally privileged. If you are not the intended recipient, you are hereby  notified that any use, retention, disclosure, dissemination, copying, or taking any other action in reliance on contents of this e-mail is prohibited. If you have received this e-mail in error, please immediately contact the sender and delete the e-mail from your mailbox or any other storage mechanism. Thank you!

_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Loading...