questions on master_datastore_index.log file

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

questions on master_datastore_index.log file

Qihua Liang
Dear Maker Develop Team,

I have successfully run Maker for several times before. But I came across a strange thing days ago when I ran Maker again on a different assembly with the same input files and settings.

I saw the message of "Maker is now finished!!!” but got empty GFF3 and no fasta files. And then I checked the master_datastore_index.log and realized that there are a lot of “failed”s and “retry”s and “failed” again. What does this mean? Since I used same inputs as previous successful runs, could you provide some instructions on how to debug and solve it?

Thank you so much
Qihua
_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Reply | Threaded
Open this post in threaded view
|

Re: questions on master_datastore_index.log file

Ence,daniel
Hi, Before we can give any help to debug it, we need the error messages. These should be in the same file that the “maker is finished” message is in. Look for the first error message (the one closest to the top of the file) and send that to the mailing list.

Thanks,
Daniel


> On Jan 3, 2018, at 8:52 PM, Qihua Liang <[hidden email]> wrote:
>
> Dear Maker Develop Team,
>
> I have successfully run Maker for several times before. But I came across a strange thing days ago when I ran Maker again on a different assembly with the same input files and settings.
>
> I saw the message of "Maker is now finished!!!” but got empty GFF3 and no fasta files. And then I checked the master_datastore_index.log and realized that there are a lot of “failed”s and “retry”s and “failed” again. What does this mean? Since I used same inputs as previous successful runs, could you provide some instructions on how to debug and solve it?
>
> Thank you so much
> Qihua
> _______________________________________________
> maker-devel mailing list
> [hidden email]
> https://urldefense.proofpoint.com/v2/url?u=http-3A__box290.bluehost.com_mailman_listinfo_maker-2Ddevel-5Fyandell-2Dlab.org&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=12jzlNvGVD0AlPJ4E7cTlw1Dvu6n9cb4kMCobJ28XPs&m=nUDCP_0kFOhDYlTHgOpWtf_zdL77aQFeQwYOGIQwP8c&s=9Z4T1hdtxOyIjpn6f70qhrQRuGsZxXdV-oLSJF1zGkY&e= 

_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Reply | Threaded
Open this post in threaded view
|

Re: questions on master_datastore_index.log file

Qihua Liang
Hi Ence,

When I searched for “E/error” in the output file, here is what first showed up:
Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
        Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
        Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
        Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
        Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179

Is this what you may need?

Qihua

On Jan 4, 2018, at 6:16 AM, Ence,daniel <[hidden email]> wrote:

Hi, Before we can give any help to debug it, we need the error messages. These should be in the same file that the “maker is finished” message is in. Look for the first error message (the one closest to the top of the file) and send that to the mailing list.

Thanks,
Daniel


On Jan 3, 2018, at 8:52 PM, Qihua Liang <[hidden email]> wrote:

Dear Maker Develop Team,

I have successfully run Maker for several times before. But I came across a strange thing days ago when I ran Maker again on a different assembly with the same input files and settings.

I saw the message of "Maker is now finished!!!” but got empty GFF3 and no fasta files. And then I checked the master_datastore_index.log and realized that there are a lot of “failed”s and “retry”s and “failed” again. What does this mean? Since I used same inputs as previous successful runs, could you provide some instructions on how to debug and solve it?

Thank you so much
Qihua
_______________________________________________
maker-devel mailing list
[hidden email]
https://urldefense.proofpoint.com/v2/url?u=http-3A__box290.bluehost.com_mailman_listinfo_maker-2Ddevel-5Fyandell-2Dlab.org&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=12jzlNvGVD0AlPJ4E7cTlw1Dvu6n9cb4kMCobJ28XPs&m=nUDCP_0kFOhDYlTHgOpWtf_zdL77aQFeQwYOGIQwP8c&s=9Z4T1hdtxOyIjpn6f70qhrQRuGsZxXdV-oLSJF1zGkY&e=



_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Reply | Threaded
Open this post in threaded view
|

Re: questions on master_datastore_index.log file

Carson Holt-2
That’s the stack trace. The error is going to be a few lines further back. It would be best to get a few hundred lines right around the area you are showing.

—Carson

On Jan 4, 2018, at 2:36 PM, Qihua Liang <[hidden email]> wrote:

Hi Ence,

When I searched for “E/error” in the output file, here is what first showed up:
Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
        Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
        Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
        Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
        Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179

Is this what you may need?

Qihua

On Jan 4, 2018, at 6:16 AM, Ence,daniel <[hidden email]> wrote:

Hi, Before we can give any help to debug it, we need the error messages. These should be in the same file that the “maker is finished” message is in. Look for the first error message (the one closest to the top of the file) and send that to the mailing list.

Thanks,
Daniel


On Jan 3, 2018, at 8:52 PM, Qihua Liang <[hidden email]> wrote:

Dear Maker Develop Team,

I have successfully run Maker for several times before. But I came across a strange thing days ago when I ran Maker again on a different assembly with the same input files and settings.

I saw the message of "Maker is now finished!!!” but got empty GFF3 and no fasta files. And then I checked the master_datastore_index.log and realized that there are a lot of “failed”s and “retry”s and “failed” again. What does this mean? Since I used same inputs as previous successful runs, could you provide some instructions on how to debug and solve it?

Thank you so much
Qihua
_______________________________________________
maker-devel mailing list
[hidden email]
https://urldefense.proofpoint.com/v2/url?u=http-3A__box290.bluehost.com_mailman_listinfo_maker-2Ddevel-5Fyandell-2Dlab.org&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=12jzlNvGVD0AlPJ4E7cTlw1Dvu6n9cb4kMCobJ28XPs&m=nUDCP_0kFOhDYlTHgOpWtf_zdL77aQFeQwYOGIQwP8c&s=9Z4T1hdtxOyIjpn6f70qhrQRuGsZxXdV-oLSJF1zGkY&e=


_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org


_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Reply | Threaded
Open this post in threaded view
|

Re: questions on master_datastore_index.log file

Qihua Liang
Hi Carson,

I am pasting more lines of error messages. I notice an error of "ERROR: Can't open seq file: /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/query.masked.gff.seq”, the seq name of “ScsGwly” is ">ScsGwly_6124;HRSCAF=6247”, is it because of the seq naming that makes the temp file name weird?

Thanks
Qihua

#--------- command -------------#
Widget::blastx:
/24-2/home/qliang/0.soft/maker/bin/../exe/blast/bin/blastx -db /tmp/maker_nJDkCL/te_proteins%2Efasta.mpi.10.9 -query /tmp/maker_nJDkCL/0/ScsG
wly_5932%3BHRSCAF=6050.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes 
-soft_masking true -lcase_masking -show_gis -out /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/D7/4
A/ScsGwly_5932%3BHRSCAF=6050//theVoid.ScsGwly_5932%3BHRSCAF=6050/0/ScsGwly_5932%3BHRSCAF=6050.0.te_proteins%2Efasta.repeatrunner.temp_dir/te_
proteins%2Efasta.mpi.10.9.repeatrunner
#-------------------------------#
deleted:0 hits
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
preparing masked sequence
preparing ab-inits
running  snap.
#--------- command -------------#
Widget::snap:
/24-2/home/qliang/0.soft/maker/exe/snap/snap /home/qliang/cowpea/annotation/09.tingting/4.Abintio/2.CEGMA/3.maker/maker1.hmm/maker1.snap.hmm 
/tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0 > /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0.maker1%2Esnap%2Eh
mm.snap
#-------------------------------#
scoring....decoding.10.20.30.40.50.60.70.80.90.100 done
scoring....decoding.10.20.30.40.50.60.70.80.90.100 done
running  augustus.
#--------- command -------------#
Widget::augustus:
/usr/local/augustus.2.7/bin/augustus --species=cowpea_new --UTR=off /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0 > /tmp/maker
_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0.cowpea_new.augustus
#-------------------------------#
deleted:0 hits
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
deleted:0 hits
doing blastx repeats
running  blast search.
#--------- command -------------#
Widget::blastx:
/24-2/home/qliang/0.soft/maker/bin/../exe/blast/bin/blastx -db /tmp/maker_mvdRkd/te_proteins%2Efasta.mpi.10.6 -query /tmp/maker_mvdRkd/0/chr10.75 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/09/chr10//theVoid.chr10/7/chr10.75.te_proteins%2Efasta.repeatrunner.temp_dir/te_proteins%2Efasta.mpi.10.6.repeatrunner
#-------------------------------#
doing blastx repeats
re reading blast report.
/24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/0/ScsGwly_6124%3BHRSCAF=6247.0.te_proteins%2Efasta.repeatrunner
deleted:0 hits
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
ERROR: Can't open seq file: /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/query.masked.gff.seq
No such file or directory

 at /24-2/home/qliang/0.soft/maker/bin/../lib/Dumper/GFF/GFFV3.pm line 199.
        Dumper::GFF::GFFV3::finalize(Dumper::GFF::GFFV3=HASH(0x5000ab8)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 700
        Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
        Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
        Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
        Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
        Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179
        Process::MpiTiers::next_chunk(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 286
        Process::MpiTiers::run_all(Process::MpiTiers=HASH(0x4fb3350), 0) called at /home/qliang/0.soft/maker/bin/maker line 695
--> rank=NA, hostname=H4
ERROR: Failed while builing masking tiers
--> rank=NA, hostname=H4
--> rank=NA, hostname=H4
ERROR: Can not get next level
running  genemark.
#--------- command -------------#
Widget::genemark:
/24-2/home/qliang/0.soft/PerlPackages/ActivePerl-5.22/bin/perl-static /24-2/home/qliang/0.soft/maker/bin/../lib/Widget/genemark/gmhmm_wrap -m /home/qliang/cowpea/annotation/05.CEGMA/2.genemask/output/gmhmm.mod -g /24-2/home/qliang/0.soft/makerPackages/gm_et_linux_64/gmes_petap/gmhmme3 -p /24-2/home/qliang/0.soft/makerPackages/gm_et_linux_64/gmes_petap/probuild -o /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_nomask.0.gmhmm%2Emod.genemark /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_nomask.0
#-------------------------------#
FAILED CONTIG:ScsGwly_6124;HRSCAF=6247

examining contents of the fasta file and run log



--Next Contig--

#---------------------------------------------------------------------
Now starting the contig!!
SeqID: ScsGwly_6140;HRSCAF=6263
Length: 1247
#---------------------------------------------------------------------




On Jan 5, 2018, at 7:22 PM, Carson Holt <[hidden email]> wrote:

That’s the stack trace. The error is going to be a few lines further back. It would be best to get a few hundred lines right around the area you are showing.

—Carson

On Jan 4, 2018, at 2:36 PM, Qihua Liang <[hidden email]> wrote:

Hi Ence,

When I searched for “E/error” in the output file, here is what first showed up:
Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
        Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
        Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
        Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
        Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179

Is this what you may need?

Qihua

On Jan 4, 2018, at 6:16 AM, Ence,daniel <[hidden email]> wrote:

Hi, Before we can give any help to debug it, we need the error messages. These should be in the same file that the “maker is finished” message is in. Look for the first error message (the one closest to the top of the file) and send that to the mailing list.

Thanks,
Daniel


On Jan 3, 2018, at 8:52 PM, Qihua Liang <[hidden email]> wrote:

Dear Maker Develop Team,

I have successfully run Maker for several times before. But I came across a strange thing days ago when I ran Maker again on a different assembly with the same input files and settings.

I saw the message of "Maker is now finished!!!” but got empty GFF3 and no fasta files. And then I checked the master_datastore_index.log and realized that there are a lot of “failed”s and “retry”s and “failed” again. What does this mean? Since I used same inputs as previous successful runs, could you provide some instructions on how to debug and solve it?

Thank you so much
Qihua
_______________________________________________
maker-devel mailing list
[hidden email]
https://urldefense.proofpoint.com/v2/url?u=http-3A__box290.bluehost.com_mailman_listinfo_maker-2Ddevel-5Fyandell-2Dlab.org&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=12jzlNvGVD0AlPJ4E7cTlw1Dvu6n9cb4kMCobJ28XPs&m=nUDCP_0kFOhDYlTHgOpWtf_zdL77aQFeQwYOGIQwP8c&s=9Z4T1hdtxOyIjpn6f70qhrQRuGsZxXdV-oLSJF1zGkY&e=


_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org



_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Reply | Threaded
Open this post in threaded view
|

Re: questions on master_datastore_index.log file

Carson Holt-2
Your contig names may create issues. Specifically the ‘;’ character, but you should also remove the ‘=‘ character. However, I believe your problem may be IO. If you are running under MPI or are running multiple jobs, the disk one of the machines may have that location unmounted, it may be full, you may have hit a system file quota limit, or the IO load is slowing it is not actually finished writing the file when MAKER tries to read it. If IO load, is the issue, then you just need to run fewer processes. The other possibilities would mean you need to make space, fix the mount, or raise any quotas on your systems.

Thanks,
Carson

On Jan 6, 2018, at 4:09 PM, Qihua Liang <[hidden email]> wrote:

Hi Carson,

I am pasting more lines of error messages. I notice an error of "ERROR: Can't open seq file: /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/query.masked.gff.seq”, the seq name of “ScsGwly” is ">ScsGwly_6124;HRSCAF=6247”, is it because of the seq naming that makes the temp file name weird?

Thanks
Qihua

#--------- command -------------#
Widget::blastx:
/24-2/home/qliang/0.soft/maker/bin/../exe/blast/bin/blastx -db /tmp/maker_nJDkCL/te_proteins%2Efasta.mpi.10.9 -query /tmp/maker_nJDkCL/0/ScsG
wly_5932%3BHRSCAF=6050.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes 
-soft_masking true -lcase_masking -show_gis -out /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/D7/4
A/ScsGwly_5932%3BHRSCAF=6050//theVoid.ScsGwly_5932%3BHRSCAF=6050/0/ScsGwly_5932%3BHRSCAF=6050.0.te_proteins%2Efasta.repeatrunner.temp_dir/te_
proteins%2Efasta.mpi.10.9.repeatrunner
#-------------------------------#
deleted:0 hits
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
preparing masked sequence
preparing ab-inits
running  snap.
#--------- command -------------#
Widget::snap:
/24-2/home/qliang/0.soft/maker/exe/snap/snap /home/qliang/cowpea/annotation/09.tingting/4.Abintio/2.CEGMA/3.maker/maker1.hmm/maker1.snap.hmm 
/tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0 > /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0.maker1%2Esnap%2Eh
mm.snap
#-------------------------------#
scoring....decoding.10.20.30.40.50.60.70.80.90.100 done
scoring....decoding.10.20.30.40.50.60.70.80.90.100 done
running  augustus.
#--------- command -------------#
Widget::augustus:
/usr/local/augustus.2.7/bin/augustus --species=cowpea_new --UTR=off /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0 > /tmp/maker
_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0.cowpea_new.augustus
#-------------------------------#
deleted:0 hits
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
deleted:0 hits
doing blastx repeats
running  blast search.
#--------- command -------------#
Widget::blastx:
/24-2/home/qliang/0.soft/maker/bin/../exe/blast/bin/blastx -db /tmp/maker_mvdRkd/te_proteins%2Efasta.mpi.10.6 -query /tmp/maker_mvdRkd/0/chr10.75 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/09/chr10//theVoid.chr10/7/chr10.75.te_proteins%2Efasta.repeatrunner.temp_dir/te_proteins%2Efasta.mpi.10.6.repeatrunner
#-------------------------------#
doing blastx repeats
re reading blast report.
/24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/0/ScsGwly_6124%3BHRSCAF=6247.0.te_proteins%2Efasta.repeatrunner
deleted:0 hits
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
ERROR: Can't open seq file: /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/query.masked.gff.seq
No such file or directory

 at /24-2/home/qliang/0.soft/maker/bin/../lib/Dumper/GFF/GFFV3.pm line 199.
        Dumper::GFF::GFFV3::finalize(Dumper::GFF::GFFV3=HASH(0x5000ab8)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 700
        Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
        Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
        Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
        Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
        Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179
        Process::MpiTiers::next_chunk(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 286
        Process::MpiTiers::run_all(Process::MpiTiers=HASH(0x4fb3350), 0) called at /home/qliang/0.soft/maker/bin/maker line 695
--> rank=NA, hostname=H4
ERROR: Failed while builing masking tiers
--> rank=NA, hostname=H4
--> rank=NA, hostname=H4
ERROR: Can not get next level
running  genemark.
#--------- command -------------#
Widget::genemark:
/24-2/home/qliang/0.soft/PerlPackages/ActivePerl-5.22/bin/perl-static /24-2/home/qliang/0.soft/maker/bin/../lib/Widget/genemark/gmhmm_wrap -m /home/qliang/cowpea/annotation/05.CEGMA/2.genemask/output/gmhmm.mod -g /24-2/home/qliang/0.soft/makerPackages/gm_et_linux_64/gmes_petap/gmhmme3 -p /24-2/home/qliang/0.soft/makerPackages/gm_et_linux_64/gmes_petap/probuild -o /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_nomask.0.gmhmm%2Emod.genemark /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_nomask.0
#-------------------------------#
FAILED CONTIG:ScsGwly_6124;HRSCAF=6247

examining contents of the fasta file and run log



--Next Contig--

#---------------------------------------------------------------------
Now starting the contig!!
SeqID: ScsGwly_6140;HRSCAF=6263
Length: 1247
#---------------------------------------------------------------------




On Jan 5, 2018, at 7:22 PM, Carson Holt <[hidden email]> wrote:

That’s the stack trace. The error is going to be a few lines further back. It would be best to get a few hundred lines right around the area you are showing.

—Carson

On Jan 4, 2018, at 2:36 PM, Qihua Liang <[hidden email]> wrote:

Hi Ence,

When I searched for “E/error” in the output file, here is what first showed up:
Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
        Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
        Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
        Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
        Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179

Is this what you may need?

Qihua

On Jan 4, 2018, at 6:16 AM, Ence,daniel <[hidden email]> wrote:

Hi, Before we can give any help to debug it, we need the error messages. These should be in the same file that the “maker is finished” message is in. Look for the first error message (the one closest to the top of the file) and send that to the mailing list.

Thanks,
Daniel


On Jan 3, 2018, at 8:52 PM, Qihua Liang <[hidden email]> wrote:

Dear Maker Develop Team,

I have successfully run Maker for several times before. But I came across a strange thing days ago when I ran Maker again on a different assembly with the same input files and settings.

I saw the message of "Maker is now finished!!!” but got empty GFF3 and no fasta files. And then I checked the master_datastore_index.log and realized that there are a lot of “failed”s and “retry”s and “failed” again. What does this mean? Since I used same inputs as previous successful runs, could you provide some instructions on how to debug and solve it?

Thank you so much
Qihua
_______________________________________________
maker-devel mailing list
[hidden email]
https://urldefense.proofpoint.com/v2/url?u=http-3A__box290.bluehost.com_mailman_listinfo_maker-2Ddevel-5Fyandell-2Dlab.org&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=12jzlNvGVD0AlPJ4E7cTlw1Dvu6n9cb4kMCobJ28XPs&m=nUDCP_0kFOhDYlTHgOpWtf_zdL77aQFeQwYOGIQwP8c&s=9Z4T1hdtxOyIjpn6f70qhrQRuGsZxXdV-oLSJF1zGkY&e=


_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org



_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Reply | Threaded
Open this post in threaded view
|

Re: questions on master_datastore_index.log file

Qihua Liang
Hi Carson,

I just check with the system administrator and we think the disk space should be working fine. And actually I also ran another attempt with much fewer processors days ago and I am having the same issues.

Maybe I will try renaming the contig names to see how the new attempt works? Or any other suggestions?

Thank you!
Qihua

On Jan 9, 2018, at 9:14 AM, Carson Holt <[hidden email]> wrote:

Your contig names may create issues. Specifically the ‘;’ character, but you should also remove the ‘=‘ character. However, I believe your problem may be IO. If you are running under MPI or are running multiple jobs, the disk one of the machines may have that location unmounted, it may be full, you may have hit a system file quota limit, or the IO load is slowing it is not actually finished writing the file when MAKER tries to read it. If IO load, is the issue, then you just need to run fewer processes. The other possibilities would mean you need to make space, fix the mount, or raise any quotas on your systems.

Thanks,
Carson

On Jan 6, 2018, at 4:09 PM, Qihua Liang <[hidden email]> wrote:

Hi Carson,

I am pasting more lines of error messages. I notice an error of "ERROR: Can't open seq file: /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/query.masked.gff.seq”, the seq name of “ScsGwly” is ">ScsGwly_6124;HRSCAF=6247”, is it because of the seq naming that makes the temp file name weird?

Thanks
Qihua

#--------- command -------------#
Widget::blastx:
/24-2/home/qliang/0.soft/maker/bin/../exe/blast/bin/blastx -db /tmp/maker_nJDkCL/te_proteins%2Efasta.mpi.10.9 -query /tmp/maker_nJDkCL/0/ScsG
wly_5932%3BHRSCAF=6050.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes 
-soft_masking true -lcase_masking -show_gis -out /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/D7/4
A/ScsGwly_5932%3BHRSCAF=6050//theVoid.ScsGwly_5932%3BHRSCAF=6050/0/ScsGwly_5932%3BHRSCAF=6050.0.te_proteins%2Efasta.repeatrunner.temp_dir/te_
proteins%2Efasta.mpi.10.9.repeatrunner
#-------------------------------#
deleted:0 hits
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
preparing masked sequence
preparing ab-inits
running  snap.
#--------- command -------------#
Widget::snap:
/24-2/home/qliang/0.soft/maker/exe/snap/snap /home/qliang/cowpea/annotation/09.tingting/4.Abintio/2.CEGMA/3.maker/maker1.hmm/maker1.snap.hmm 
/tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0 > /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0.maker1%2Esnap%2Eh
mm.snap
#-------------------------------#
scoring....decoding.10.20.30.40.50.60.70.80.90.100 done
scoring....decoding.10.20.30.40.50.60.70.80.90.100 done
running  augustus.
#--------- command -------------#
Widget::augustus:
/usr/local/augustus.2.7/bin/augustus --species=cowpea_new --UTR=off /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0 > /tmp/maker
_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0.cowpea_new.augustus
#-------------------------------#
deleted:0 hits
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
deleted:0 hits
doing blastx repeats
running  blast search.
#--------- command -------------#
Widget::blastx:
/24-2/home/qliang/0.soft/maker/bin/../exe/blast/bin/blastx -db /tmp/maker_mvdRkd/te_proteins%2Efasta.mpi.10.6 -query /tmp/maker_mvdRkd/0/chr10.75 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/09/chr10//theVoid.chr10/7/chr10.75.te_proteins%2Efasta.repeatrunner.temp_dir/te_proteins%2Efasta.mpi.10.6.repeatrunner
#-------------------------------#
doing blastx repeats
re reading blast report.
/24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/0/ScsGwly_6124%3BHRSCAF=6247.0.te_proteins%2Efasta.repeatrunner
deleted:0 hits
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
ERROR: Can't open seq file: /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/query.masked.gff.seq
No such file or directory

 at /24-2/home/qliang/0.soft/maker/bin/../lib/Dumper/GFF/GFFV3.pm line 199.
        Dumper::GFF::GFFV3::finalize(Dumper::GFF::GFFV3=HASH(0x5000ab8)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 700
        Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
        Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
        Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
        Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
        Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179
        Process::MpiTiers::next_chunk(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 286
        Process::MpiTiers::run_all(Process::MpiTiers=HASH(0x4fb3350), 0) called at /home/qliang/0.soft/maker/bin/maker line 695
--> rank=NA, hostname=H4
ERROR: Failed while builing masking tiers
--> rank=NA, hostname=H4
--> rank=NA, hostname=H4
ERROR: Can not get next level
running  genemark.
#--------- command -------------#
Widget::genemark:
/24-2/home/qliang/0.soft/PerlPackages/ActivePerl-5.22/bin/perl-static /24-2/home/qliang/0.soft/maker/bin/../lib/Widget/genemark/gmhmm_wrap -m /home/qliang/cowpea/annotation/05.CEGMA/2.genemask/output/gmhmm.mod -g /24-2/home/qliang/0.soft/makerPackages/gm_et_linux_64/gmes_petap/gmhmme3 -p /24-2/home/qliang/0.soft/makerPackages/gm_et_linux_64/gmes_petap/probuild -o /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_nomask.0.gmhmm%2Emod.genemark /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_nomask.0
#-------------------------------#
FAILED CONTIG:ScsGwly_6124;HRSCAF=6247

examining contents of the fasta file and run log



--Next Contig--

#---------------------------------------------------------------------
Now starting the contig!!
SeqID: ScsGwly_6140;HRSCAF=6263
Length: 1247
#---------------------------------------------------------------------




On Jan 5, 2018, at 7:22 PM, Carson Holt <[hidden email]> wrote:

That’s the stack trace. The error is going to be a few lines further back. It would be best to get a few hundred lines right around the area you are showing.

—Carson

On Jan 4, 2018, at 2:36 PM, Qihua Liang <[hidden email]> wrote:

Hi Ence,

When I searched for “E/error” in the output file, here is what first showed up:
Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
        Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
        Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
        Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
        Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179

Is this what you may need?

Qihua

On Jan 4, 2018, at 6:16 AM, Ence,daniel <[hidden email]> wrote:

Hi, Before we can give any help to debug it, we need the error messages. These should be in the same file that the “maker is finished” message is in. Look for the first error message (the one closest to the top of the file) and send that to the mailing list.

Thanks,
Daniel


On Jan 3, 2018, at 8:52 PM, Qihua Liang <[hidden email]> wrote:

Dear Maker Develop Team,

I have successfully run Maker for several times before. But I came across a strange thing days ago when I ran Maker again on a different assembly with the same input files and settings.

I saw the message of "Maker is now finished!!!” but got empty GFF3 and no fasta files. And then I checked the master_datastore_index.log and realized that there are a lot of “failed”s and “retry”s and “failed” again. What does this mean? Since I used same inputs as previous successful runs, could you provide some instructions on how to debug and solve it?

Thank you so much
Qihua
_______________________________________________
maker-devel mailing list
[hidden email]
https://urldefense.proofpoint.com/v2/url?u=http-3A__box290.bluehost.com_mailman_listinfo_maker-2Ddevel-5Fyandell-2Dlab.org&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=12jzlNvGVD0AlPJ4E7cTlw1Dvu6n9cb4kMCobJ28XPs&m=nUDCP_0kFOhDYlTHgOpWtf_zdL77aQFeQwYOGIQwP8c&s=9Z4T1hdtxOyIjpn6f70qhrQRuGsZxXdV-oLSJF1zGkY&e=


_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
Reply | Threaded
Open this post in threaded view
|

Re: questions on master_datastore_index.log file

Carson Holt-2
The error is saying exactly that the file MAKER just created does not exist. The only time we ever see this is when using network mounted locations under heavy IO load. Most network storage options use asynchronous IO, which means the system returns success on file operation before they actually complete. So they can say they finished writing a file before it actually exist. So if you try and open it right away, it doesn’t really exist and everything fails. But that only happens if there is heavy IO (lots of things going on in that mount location). So if you are getting persitent failures you may want to try a different work directory, or get your IT to troubleshoot IO load in the directory you are using.

—Carson


On Jan 9, 2018, at 11:10 AM, Qihua Liang <[hidden email]> wrote:

Hi Carson,

I just check with the system administrator and we think the disk space should be working fine. And actually I also ran another attempt with much fewer processors days ago and I am having the same issues.

Maybe I will try renaming the contig names to see how the new attempt works? Or any other suggestions?

Thank you!
Qihua

On Jan 9, 2018, at 9:14 AM, Carson Holt <[hidden email]> wrote:

Your contig names may create issues. Specifically the ‘;’ character, but you should also remove the ‘=‘ character. However, I believe your problem may be IO. If you are running under MPI or are running multiple jobs, the disk one of the machines may have that location unmounted, it may be full, you may have hit a system file quota limit, or the IO load is slowing it is not actually finished writing the file when MAKER tries to read it. If IO load, is the issue, then you just need to run fewer processes. The other possibilities would mean you need to make space, fix the mount, or raise any quotas on your systems.

Thanks,
Carson

On Jan 6, 2018, at 4:09 PM, Qihua Liang <[hidden email]> wrote:

Hi Carson,

I am pasting more lines of error messages. I notice an error of "ERROR: Can't open seq file: /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/query.masked.gff.seq”, the seq name of “ScsGwly” is ">ScsGwly_6124;HRSCAF=6247”, is it because of the seq naming that makes the temp file name weird?

Thanks
Qihua

#--------- command -------------#
Widget::blastx:
/24-2/home/qliang/0.soft/maker/bin/../exe/blast/bin/blastx -db /tmp/maker_nJDkCL/te_proteins%2Efasta.mpi.10.9 -query /tmp/maker_nJDkCL/0/ScsG
wly_5932%3BHRSCAF=6050.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes 
-soft_masking true -lcase_masking -show_gis -out /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/D7/4
A/ScsGwly_5932%3BHRSCAF=6050//theVoid.ScsGwly_5932%3BHRSCAF=6050/0/ScsGwly_5932%3BHRSCAF=6050.0.te_proteins%2Efasta.repeatrunner.temp_dir/te_
proteins%2Efasta.mpi.10.9.repeatrunner
#-------------------------------#
deleted:0 hits
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
preparing masked sequence
preparing ab-inits
running  snap.
#--------- command -------------#
Widget::snap:
/24-2/home/qliang/0.soft/maker/exe/snap/snap /home/qliang/cowpea/annotation/09.tingting/4.Abintio/2.CEGMA/3.maker/maker1.hmm/maker1.snap.hmm 
/tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0 > /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0.maker1%2Esnap%2Eh
mm.snap
#-------------------------------#
scoring....decoding.10.20.30.40.50.60.70.80.90.100 done
scoring....decoding.10.20.30.40.50.60.70.80.90.100 done
running  augustus.
#--------- command -------------#
Widget::augustus:
/usr/local/augustus.2.7/bin/augustus --species=cowpea_new --UTR=off /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0 > /tmp/maker
_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0.cowpea_new.augustus
#-------------------------------#
deleted:0 hits
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
deleted:0 hits
doing blastx repeats
running  blast search.
#--------- command -------------#
Widget::blastx:
/24-2/home/qliang/0.soft/maker/bin/../exe/blast/bin/blastx -db /tmp/maker_mvdRkd/te_proteins%2Efasta.mpi.10.6 -query /tmp/maker_mvdRkd/0/chr10.75 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/09/chr10//theVoid.chr10/7/chr10.75.te_proteins%2Efasta.repeatrunner.temp_dir/te_proteins%2Efasta.mpi.10.6.repeatrunner
#-------------------------------#
doing blastx repeats
re reading blast report.
/24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/0/ScsGwly_6124%3BHRSCAF=6247.0.te_proteins%2Efasta.repeatrunner
deleted:0 hits
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
doing blastx repeats
collecting blastx repeatmasking
processing all repeats
in cluster::shadow_cluster...
...finished clustering.
ERROR: Can't open seq file: /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/query.masked.gff.seq
No such file or directory

 at /24-2/home/qliang/0.soft/maker/bin/../lib/Dumper/GFF/GFFV3.pm line 199.
        Dumper::GFF::GFFV3::finalize(Dumper::GFF::GFFV3=HASH(0x5000ab8)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 700
        Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
        Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
        Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
        Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
        Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179
        Process::MpiTiers::next_chunk(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 286
        Process::MpiTiers::run_all(Process::MpiTiers=HASH(0x4fb3350), 0) called at /home/qliang/0.soft/maker/bin/maker line 695
--> rank=NA, hostname=H4
ERROR: Failed while builing masking tiers
--> rank=NA, hostname=H4
--> rank=NA, hostname=H4
ERROR: Can not get next level
running  genemark.
#--------- command -------------#
Widget::genemark:
/24-2/home/qliang/0.soft/PerlPackages/ActivePerl-5.22/bin/perl-static /24-2/home/qliang/0.soft/maker/bin/../lib/Widget/genemark/gmhmm_wrap -m /home/qliang/cowpea/annotation/05.CEGMA/2.genemask/output/gmhmm.mod -g /24-2/home/qliang/0.soft/makerPackages/gm_et_linux_64/gmes_petap/gmhmme3 -p /24-2/home/qliang/0.soft/makerPackages/gm_et_linux_64/gmes_petap/probuild -o /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_nomask.0.gmhmm%2Emod.genemark /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_nomask.0
#-------------------------------#
FAILED CONTIG:ScsGwly_6124;HRSCAF=6247

examining contents of the fasta file and run log



--Next Contig--

#---------------------------------------------------------------------
Now starting the contig!!
SeqID: ScsGwly_6140;HRSCAF=6263
Length: 1247
#---------------------------------------------------------------------




On Jan 5, 2018, at 7:22 PM, Carson Holt <[hidden email]> wrote:

That’s the stack trace. The error is going to be a few lines further back. It would be best to get a few hundred lines right around the area you are showing.

—Carson

On Jan 4, 2018, at 2:36 PM, Qihua Liang <[hidden email]> wrote:

Hi Ence,

When I searched for “E/error” in the output file, here is what first showed up:
Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
        Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
        Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
        Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
        eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
        Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
        Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179

Is this what you may need?

Qihua

On Jan 4, 2018, at 6:16 AM, Ence,daniel <[hidden email]> wrote:

Hi, Before we can give any help to debug it, we need the error messages. These should be in the same file that the “maker is finished” message is in. Look for the first error message (the one closest to the top of the file) and send that to the mailing list.

Thanks,
Daniel


On Jan 3, 2018, at 8:52 PM, Qihua Liang <[hidden email]> wrote:

Dear Maker Develop Team,

I have successfully run Maker for several times before. But I came across a strange thing days ago when I ran Maker again on a different assembly with the same input files and settings.

I saw the message of "Maker is now finished!!!” but got empty GFF3 and no fasta files. And then I checked the master_datastore_index.log and realized that there are a lot of “failed”s and “retry”s and “failed” again. What does this mean? Since I used same inputs as previous successful runs, could you provide some instructions on how to debug and solve it?

Thank you so much
Qihua
_______________________________________________
maker-devel mailing list
[hidden email]
https://urldefense.proofpoint.com/v2/url?u=http-3A__box290.bluehost.com_mailman_listinfo_maker-2Ddevel-5Fyandell-2Dlab.org&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=12jzlNvGVD0AlPJ4E7cTlw1Dvu6n9cb4kMCobJ28XPs&m=nUDCP_0kFOhDYlTHgOpWtf_zdL77aQFeQwYOGIQwP8c&s=9Z4T1hdtxOyIjpn6f70qhrQRuGsZxXdV-oLSJF1zGkY&e=


_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org





_______________________________________________
maker-devel mailing list
[hidden email]
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org