Gbrowse bp_seqfeature_load WARNING Error

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Gbrowse bp_seqfeature_load WARNING Error

eaalvarado

Hello,

 

I am following the GBrowse NGS Tutorial from http://gmod.org/wiki/GBrowse_NGS_Tutorial#Start but I am getting the following error when running bp_seqfeature_load.pl. I can send you the fasta and gff3 file that I am trying to upload into GBrowse. Do you know a workaround this issue? I would really appreciate your help.

 

 

loading Lsat_1_v4_lg_1_1..252823024.gff3...

 

--------------------- WARNING ---------------------

MSG: ID=3667958 has been used more than once, but it cannot be found in the database.

This can happen if you have specified fast loading, but features sharing the same ID

are not contiguous in the GFF file. This will be loaded as a separate feature.

Line 31: "Lsat_1_v4_lg_1              GeneWise_St    CDS        12608    12931    .               .               0                ID=3667958;Scaffold=Lsat_1_v4_g_1_3464

"

 

STACK Bio::DB::SeqFeature::Store::GFF3Loader::handle_feature /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:592

STACK Bio::DB::SeqFeature::Store::GFF3Loader::load_line /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:415

STACK Bio::DB::SeqFeature::Store::Loader::do_load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:433

STACK Bio::DB::SeqFeature::Store::Loader::load_fh /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:353

STACK Bio::DB::SeqFeature::Store::Loader::load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:243

STACK toplevel /opt/gbrowse/bin/bp_seqfeature_load.pl:252

---------------------------------------------------

 

 

Thank you,

 

-Emilio


------------------------------------------------------------------------------

_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Gbrowse bp_seqfeature_load WARNING Error

Timothy Parnell
Hi Emilio,
As the error explains, your GFF3 file is re-using ID numbers. Normally, these should be unique within the file, and are used to help establish parent-child relationships. When loading, parents ideally should come before children in the GFF3 file so that they can be properly associated. When a child comes before the parent, I think it makes a dummy parent with that ID, and then throws this error when it finally comes across the parent later in the file.

You can try turning off fast loading in the bp_seqfeature_load.pl load program, in which case out-of-order parents and children should be handled smartly. Just be prepared to let it run much, much longer (mysql isn’t so bad, sqlite is horrible with really big files).

Sometimes, sorting the GFF3 by genomic coordinates might also help.

Good luck,
Tim

On Oct 19, 2015, at 5:23 PM, Emilio A. Alvarado Ortiz <[hidden email]<mailto:[hidden email]>> wrote:

Hello,

I am following the GBrowse NGS Tutorial from http://gmod.org/wiki/GBrowse_NGS_Tutorial#Start but I am getting the following error when running bp_seqfeature_load.pl. I can send you the fasta and gff3 file that I am trying to upload into GBrowse. Do you know a workaround this issue? I would really appreciate your help.


loading Lsat_1_v4_lg_1_1..252823024.gff3...

--------------------- WARNING ---------------------
MSG: ID=3667958 has been used more than once, but it cannot be found in the database.
This can happen if you have specified fast loading, but features sharing the same ID
are not contiguous in the GFF file. This will be loaded as a separate feature.
Line 31: "Lsat_1_v4_lg_1              GeneWise_St    CDS        12608    12931    .               .               0                ID=3667958;Scaffold=Lsat_1_v4_g_1_3464
"

STACK Bio::DB::SeqFeature::Store::GFF3Loader::handle_feature /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:592
STACK Bio::DB::SeqFeature::Store::GFF3Loader::load_line /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:415
STACK Bio::DB::SeqFeature::Store::Loader::do_load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:433
STACK Bio::DB::SeqFeature::Store::Loader::load_fh /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:353
STACK Bio::DB::SeqFeature::Store::Loader::load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:243
STACK toplevel /opt/gbrowse/bin/bp_seqfeature_load.pl:252
---------------------------------------------------


Thank you,

-Emilio
------------------------------------------------------------------------------
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]<mailto:[hidden email]>
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

------------------------------------------------------------------------------
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Gbrowse bp_seqfeature_load WARNING Error

Timothy Parnell
Wow, that appears to be a really low level error that is below that of the BioPerl database adaptor. Are you running out of disk space or allocation issues where the mysql files are kept? You should solve any issues with the mysql server before dealing with the perl side of things.

Also, make sure you use -c or create option in bp_seqfeature_load when re-loading a database; this will empty the existing tables before loading. I could imagine you might hit table limits if you keep loading and loading and loading on top of each other. I suppose it’s also possible your GFF3 is so large that you’re just running into the predefined limits of the SeqFeature store tables; these could be changed in the perl code if need be.

I don’t know if this will help, but you may also want to consider increasing the parameters for the mysql server; I use the settings for a “large” system in /etc/my.cnf on my system. That’s mostly just caches and buffer sizes, though.


On Oct 20, 2015, at 12:10 PM, Emilio A. Alvarado Ortiz <[hidden email]<mailto:[hidden email]>> wrote:

Hi Timothy,

Thank you for the quick reply. When I turn off fast loading in the bp_seqfeature_load.pl load program, everything starts loading fine and then I get the same warning along with "The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358. Do you know what is triggering this error? I have attached the log file.

--------------------- WARNING ---------------------
MSG: ID=666531 has been used more than once, but it cannot be found in the database.
This can happen if you have specified fast loading, but features sharing the same ID
are not contiguous in the GFF file. This will be loaded as a separate feature.
Line 385661: "Lsat_1_v4_lg_1 protein2genome match_part 120131127 120131795 486 - . Name=gi%7C225435580%7Cref%7CXP_002283196.1%7C;ID=666531;Target=gi%7C225435580%7Cref%7CXP_002283196.1%7C%20200%20427;Gap=M68%20I1%20M11%20I4%20M27%20I1%20M40%20D1%20M76;Scaffold=Lsat_1_v4_g_1_1079
"

STACK Bio::DB::SeqFeature::Store::GFF3Loader::handle_feature /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:592
STACK Bio::DB::SeqFeature::Store::GFF3Loader::load_line /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:415
STACK Bio::DB::SeqFeature::Store::Loader::do_load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:433
STACK Bio::DB::SeqFeature::Store::Loader::load_fh /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:353
STACK Bio::DB::SeqFeature::Store::Loader::load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:243
STACK toplevel /opt/gbrowse/bin/bp_seqfeature_load.pl:252
---------------------------------------------------
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385661.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385661.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385662.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385662.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385663.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385663.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385664.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385664.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385665.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385665.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385666.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385666.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385667.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385667.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385668.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385668.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385669.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385669.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385670.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385670.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385671.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385671.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385672.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385672.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385673.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385673.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385674.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385674.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385675.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385675.


Thank you,

-Emilio

-----Original Message-----
From: Timothy Parnell [mailto:[hidden email]]
Sent: Tuesday, October 20, 2015 10:18 AM
To: Emilio A. Alvarado Ortiz
Cc: [hidden email]<mailto:[hidden email]>
Subject: Re: [Gmod-gbrowse] Gbrowse bp_seqfeature_load WARNING Error

Hi Emilio,
As the error explains, your GFF3 file is re-using ID numbers. Normally, these should be unique within the file, and are used to help establish parent-child relationships. When loading, parents ideally should come before children in the GFF3 file so that they can be properly associated. When a child comes before the parent, I think it makes a dummy parent with that ID, and then throws this error when it finally comes across the parent later in the file.

You can try turning off fast loading in the bp_seqfeature_load.pl load program, in which case out-of-order parents and children should be handled smartly. Just be prepared to let it run much, much longer (mysql isn’t so bad, sqlite is horrible with really big files).

Sometimes, sorting the GFF3 by genomic coordinates might also help.

Good luck,
Tim

On Oct 19, 2015, at 5:23 PM, Emilio A. Alvarado Ortiz <[hidden email]<mailto:[hidden email]><mailto:[hidden email]>> wrote:

Hello,

I am following the GBrowse NGS Tutorial from http://gmod.org/wiki/GBrowse_NGS_Tutorial#Start but I am getting the following error when running bp_seqfeature_load.pl. I can send you the fasta and gff3 file that I am trying to upload into GBrowse. Do you know a workaround this issue? I would really appreciate your help.


loading Lsat_1_v4_lg_1_1..252823024.gff3...

--------------------- WARNING ---------------------
MSG: ID=3667958 has been used more than once, but it cannot be found in the database.
This can happen if you have specified fast loading, but features sharing the same ID are not contiguous in the GFF file. This will be loaded as a separate feature.
Line 31: "Lsat_1_v4_lg_1              GeneWise_St    CDS        12608    12931    .               .               0            ID=3667958;Scaffold=Lsat_1_v4_g_1_3464
"

STACK Bio::DB::SeqFeature::Store::GFF3Loader::handle_feature /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:592
STACK Bio::DB::SeqFeature::Store::GFF3Loader::load_line /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:415
STACK Bio::DB::SeqFeature::Store::Loader::do_load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:433
STACK Bio::DB::SeqFeature::Store::Loader::load_fh /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:353
STACK Bio::DB::SeqFeature::Store::Loader::load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:243
STACK toplevel /opt/gbrowse/bin/bp_seqfeature_load.pl:252
---------------------------------------------------


Thank you,

-Emilio
------------------------------------------------------------------------------
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]<mailto:[hidden email]><mailto:[hidden email]>
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

<bpsl.log-1>

------------------------------------------------------------------------------
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
Reply | Threaded
Open this post in threaded view
|

Re: Gbrowse bp_seqfeature_load WARNING Error

eaalvarado
Hi Timothy,

I was finally able to upload the fasta and GFF3 file using bp_seqfeature_load. I am using the GBrowse 2.52, VirtualBox Edition (32bit)  downloaded from http://gmod.org/wiki/GBrowse2_VMs. I believe the problem was the because I was running out of the /boot disk space. I increased the space by adding a new hard drive to the VM and the files successfully uploaded (see attached log).  However, now I have another issue, when I try to visualize what I just uploaded, I get a " Not Found" error. I have attached a screenshot of the error message. Do you know why it is not able to find it? I really appreciate your help.


Thank you,

-Emilio

-----Original Message-----
From: Timothy Parnell [mailto:[hidden email]]
Sent: Tuesday, October 20, 2015 12:21 PM
To: Emilio A. Alvarado Ortiz
Cc: Gbrowse (E-mail)
Subject: Re: [Gmod-gbrowse] Gbrowse bp_seqfeature_load WARNING Error

Wow, that appears to be a really low level error that is below that of the BioPerl database adaptor. Are you running out of disk space or allocation issues where the mysql files are kept? You should solve any issues with the mysql server before dealing with the perl side of things.

Also, make sure you use -c or create option in bp_seqfeature_load when re-loading a database; this will empty the existing tables before loading. I could imagine you might hit table limits if you keep loading and loading and loading on top of each other. I suppose it’s also possible your GFF3 is so large that you’re just running into the predefined limits of the SeqFeature store tables; these could be changed in the perl code if need be.

I don’t know if this will help, but you may also want to consider increasing the parameters for the mysql server; I use the settings for a “large” system in /etc/my.cnf on my system. That’s mostly just caches and buffer sizes, though.


On Oct 20, 2015, at 12:10 PM, Emilio A. Alvarado Ortiz <[hidden email]<mailto:[hidden email]>> wrote:

Hi Timothy,

Thank you for the quick reply. When I turn off fast loading in the bp_seqfeature_load.pl load program, everything starts loading fine and then I get the same warning along with "The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358. Do you know what is triggering this error? I have attached the log file.

--------------------- WARNING ---------------------
MSG: ID=666531 has been used more than once, but it cannot be found in the database.
This can happen if you have specified fast loading, but features sharing the same ID are not contiguous in the GFF file. This will be loaded as a separate feature.
Line 385661: "Lsat_1_v4_lg_1 protein2genome match_part 120131127 120131795 486 - . Name=gi%7C225435580%7Cref%7CXP_002283196.1%7C;ID=666531;Target=gi%7C225435580%7Cref%7CXP_002283196.1%7C%20200%20427;Gap=M68%20I1%20M11%20I4%20M27%20I1%20M40%20D1%20M76;Scaffold=Lsat_1_v4_g_1_1079
"

STACK Bio::DB::SeqFeature::Store::GFF3Loader::handle_feature /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:592
STACK Bio::DB::SeqFeature::Store::GFF3Loader::load_line /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:415
STACK Bio::DB::SeqFeature::Store::Loader::do_load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:433
STACK Bio::DB::SeqFeature::Store::Loader::load_fh /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:353
STACK Bio::DB::SeqFeature::Store::Loader::load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:243
STACK toplevel /opt/gbrowse/bin/bp_seqfeature_load.pl:252
---------------------------------------------------
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385661.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385661.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385662.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385662.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385663.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385663.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385664.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385664.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385665.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385665.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385666.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385666.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385667.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385667.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385668.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385668.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385669.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385669.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385670.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385670.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385671.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385671.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385672.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385672.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385673.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385673.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385674.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385674.
DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385675.
Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385675.


Thank you,

-Emilio

-----Original Message-----
From: Timothy Parnell [mailto:[hidden email]]
Sent: Tuesday, October 20, 2015 10:18 AM
To: Emilio A. Alvarado Ortiz
Cc: [hidden email]<mailto:[hidden email]>
Subject: Re: [Gmod-gbrowse] Gbrowse bp_seqfeature_load WARNING Error

Hi Emilio,
As the error explains, your GFF3 file is re-using ID numbers. Normally, these should be unique within the file, and are used to help establish parent-child relationships. When loading, parents ideally should come before children in the GFF3 file so that they can be properly associated. When a child comes before the parent, I think it makes a dummy parent with that ID, and then throws this error when it finally comes across the parent later in the file.

You can try turning off fast loading in the bp_seqfeature_load.pl load program, in which case out-of-order parents and children should be handled smartly. Just be prepared to let it run much, much longer (mysql isn’t so bad, sqlite is horrible with really big files).

Sometimes, sorting the GFF3 by genomic coordinates might also help.

Good luck,
Tim

On Oct 19, 2015, at 5:23 PM, Emilio A. Alvarado Ortiz <[hidden email]<mailto:[hidden email]><mailto:[hidden email]>> wrote:

Hello,

I am following the GBrowse NGS Tutorial from http://gmod.org/wiki/GBrowse_NGS_Tutorial#Start but I am getting the following error when running bp_seqfeature_load.pl. I can send you the fasta and gff3 file that I am trying to upload into GBrowse. Do you know a workaround this issue? I would really appreciate your help.


loading Lsat_1_v4_lg_1_1..252823024.gff3...

--------------------- WARNING ---------------------
MSG: ID=3667958 has been used more than once, but it cannot be found in the database.
This can happen if you have specified fast loading, but features sharing the same ID are not contiguous in the GFF file. This will be loaded as a separate feature.
Line 31: "Lsat_1_v4_lg_1              GeneWise_St    CDS        12608    12931    .               .               0            ID=3667958;Scaffold=Lsat_1_v4_g_1_3464
"

STACK Bio::DB::SeqFeature::Store::GFF3Loader::handle_feature /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:592
STACK Bio::DB::SeqFeature::Store::GFF3Loader::load_line /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:415
STACK Bio::DB::SeqFeature::Store::Loader::do_load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:433
STACK Bio::DB::SeqFeature::Store::Loader::load_fh /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:353
STACK Bio::DB::SeqFeature::Store::Loader::load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:243
STACK toplevel /opt/gbrowse/bin/bp_seqfeature_load.pl:252
---------------------------------------------------


Thank you,

-Emilio
------------------------------------------------------------------------------
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]<mailto:[hidden email]><mailto:[hidden email]>
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

<bpsl.log-1>


------------------------------------------------------------------------------

_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse

bpsl.log2 (155K) Download Attachment
Gbrowse_NotFound_Error.png (78K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Gbrowse bp_seqfeature_load WARNING Error

Timothy Parnell
Hi Emilio,
You should be able to look back at archived messages on this mailing list to find some advice; this comes up a lot. The most likely culprit is that you don’t have the chromosomes (or scaffold or reference sequence) entered in your GFF3 file. Something like
chr1    UCSC    chromosome      1       249250621       .       .       .       Name=chr1;ID=chr1

Tim

> On Oct 20, 2015, at 6:55 PM, Emilio A. Alvarado Ortiz <[hidden email]> wrote:
>
> Hi Timothy,
>
> I was finally able to upload the fasta and GFF3 file using bp_seqfeature_load. I am using the GBrowse 2.52, VirtualBox Edition (32bit)  downloaded from http://gmod.org/wiki/GBrowse2_VMs. I believe the problem was the because I was running out of the /boot disk space. I increased the space by adding a new hard drive to the VM and the files successfully uploaded (see attached log).  However, now I have another issue, when I try to visualize what I just uploaded, I get a " Not Found" error. I have attached a screenshot of the error message. Do you know why it is not able to find it? I really appreciate your help.
>
>
> Thank you,
>
> -Emilio
>
> -----Original Message-----
> From: Timothy Parnell [mailto:[hidden email]]
> Sent: Tuesday, October 20, 2015 12:21 PM
> To: Emilio A. Alvarado Ortiz
> Cc: Gbrowse (E-mail)
> Subject: Re: [Gmod-gbrowse] Gbrowse bp_seqfeature_load WARNING Error
>
> Wow, that appears to be a really low level error that is below that of the BioPerl database adaptor. Are you running out of disk space or allocation issues where the mysql files are kept? You should solve any issues with the mysql server before dealing with the perl side of things.
>
> Also, make sure you use -c or create option in bp_seqfeature_load when re-loading a database; this will empty the existing tables before loading. I could imagine you might hit table limits if you keep loading and loading and loading on top of each other. I suppose it’s also possible your GFF3 is so large that you’re just running into the predefined limits of the SeqFeature store tables; these could be changed in the perl code if need be.
>
> I don’t know if this will help, but you may also want to consider increasing the parameters for the mysql server; I use the settings for a “large” system in /etc/my.cnf on my system. That’s mostly just caches and buffer sizes, though.
>
>
> On Oct 20, 2015, at 12:10 PM, Emilio A. Alvarado Ortiz <[hidden email]<mailto:[hidden email]>> wrote:
>
> Hi Timothy,
>
> Thank you for the quick reply. When I turn off fast loading in the bp_seqfeature_load.pl load program, everything starts loading fine and then I get the same warning along with "The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358. Do you know what is triggering this error? I have attached the log file.
>
> --------------------- WARNING ---------------------
> MSG: ID=666531 has been used more than once, but it cannot be found in the database.
> This can happen if you have specified fast loading, but features sharing the same ID are not contiguous in the GFF file. This will be loaded as a separate feature.
> Line 385661: "Lsat_1_v4_lg_1 protein2genome match_part 120131127 120131795 486 - . Name=gi%7C225435580%7Cref%7CXP_002283196.1%7C;ID=666531;Target=gi%7C225435580%7Cref%7CXP_002283196.1%7C%20200%20427;Gap=M68%20I1%20M11%20I4%20M27%20I1%20M40%20D1%20M76;Scaffold=Lsat_1_v4_g_1_1079
> "
>
> STACK Bio::DB::SeqFeature::Store::GFF3Loader::handle_feature /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:592
> STACK Bio::DB::SeqFeature::Store::GFF3Loader::load_line /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:415
> STACK Bio::DB::SeqFeature::Store::Loader::do_load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:433
> STACK Bio::DB::SeqFeature::Store::Loader::load_fh /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:353
> STACK Bio::DB::SeqFeature::Store::Loader::load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:243
> STACK toplevel /opt/gbrowse/bin/bp_seqfeature_load.pl:252
> ---------------------------------------------------
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385661.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385661.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385662.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385662.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385663.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385663.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385664.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385664.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385665.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385665.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385666.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385666.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385667.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385667.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385668.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385668.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385669.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385669.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385670.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385670.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385671.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385671.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385672.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385672.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385673.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385673.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385674.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385674.
> DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385675.
> Transaction aborted because DBD::mysql::st execute failed: The table 'feature' is full at /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/DBI/mysql.pm line 1358, <GEN1> line 385675.
>
>
> Thank you,
>
> -Emilio
>
> -----Original Message-----
> From: Timothy Parnell [mailto:[hidden email]]
> Sent: Tuesday, October 20, 2015 10:18 AM
> To: Emilio A. Alvarado Ortiz
> Cc: [hidden email]<mailto:[hidden email]>
> Subject: Re: [Gmod-gbrowse] Gbrowse bp_seqfeature_load WARNING Error
>
> Hi Emilio,
> As the error explains, your GFF3 file is re-using ID numbers. Normally, these should be unique within the file, and are used to help establish parent-child relationships. When loading, parents ideally should come before children in the GFF3 file so that they can be properly associated. When a child comes before the parent, I think it makes a dummy parent with that ID, and then throws this error when it finally comes across the parent later in the file.
>
> You can try turning off fast loading in the bp_seqfeature_load.pl load program, in which case out-of-order parents and children should be handled smartly. Just be prepared to let it run much, much longer (mysql isn’t so bad, sqlite is horrible with really big files).
>
> Sometimes, sorting the GFF3 by genomic coordinates might also help.
>
> Good luck,
> Tim
>
> On Oct 19, 2015, at 5:23 PM, Emilio A. Alvarado Ortiz <[hidden email]<mailto:[hidden email]><mailto:[hidden email]>> wrote:
>
> Hello,
>
> I am following the GBrowse NGS Tutorial from http://gmod.org/wiki/GBrowse_NGS_Tutorial#Start but I am getting the following error when running bp_seqfeature_load.pl. I can send you the fasta and gff3 file that I am trying to upload into GBrowse. Do you know a workaround this issue? I would really appreciate your help.
>
>
> loading Lsat_1_v4_lg_1_1..252823024.gff3...
>
> --------------------- WARNING ---------------------
> MSG: ID=3667958 has been used more than once, but it cannot be found in the database.
> This can happen if you have specified fast loading, but features sharing the same ID are not contiguous in the GFF file. This will be loaded as a separate feature.
> Line 31: "Lsat_1_v4_lg_1              GeneWise_St    CDS        12608    12931    .               .               0            ID=3667958;Scaffold=Lsat_1_v4_g_1_3464
> "
>
> STACK Bio::DB::SeqFeature::Store::GFF3Loader::handle_feature /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:592
> STACK Bio::DB::SeqFeature::Store::GFF3Loader::load_line /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/GFF3Loader.pm:415
> STACK Bio::DB::SeqFeature::Store::Loader::do_load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:433
> STACK Bio::DB::SeqFeature::Store::Loader::load_fh /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:353
> STACK Bio::DB::SeqFeature::Store::Loader::load /opt/gbrowse/lib/perl5/Bio/DB/SeqFeature/Store/Loader.pm:243
> STACK toplevel /opt/gbrowse/bin/bp_seqfeature_load.pl:252
> ---------------------------------------------------
>
>
> Thank you,
>
> -Emilio
> ------------------------------------------------------------------------------
> _______________________________________________
> Gmod-gbrowse mailing list
> [hidden email]<mailto:[hidden email]><mailto:[hidden email]>
> https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse
>
> <bpsl.log-1>
>
> <bpsl.log2><Gbrowse_NotFound_Error.png>

------------------------------------------------------------------------------
_______________________________________________
Gmod-gbrowse mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-gbrowse