Unable to load GO into Chado, GO bug or GMOD tools bug or ....

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Unable to load GO into Chado, GO bug or GMOD tools bug or ....

George Hartzell

Hi Everybody,

I’m trying to install Chado following the instructions in the ‘Chado from SVN’ section of the Getting Started page. I have everything working successfully until the make ontologies step. I can successfully use the make ontologies command to install the Relationship Ontology and the Sequence Ontology but am unable to successfully load the Gene Ontology.

It appears that the loading process is trying to insert a row into the cvterm table without values for several fields that are required to be “not null”.

After a long and winding road, the problem seems to be caused by this portion:

property_value: IAO:0000589 "cell and encapsulating structures" xsd:string

of the entry for the GO term GO:0005623 in the go.obo file. [Is IAO the “Information Artifact Ontology?] Is it possible that go.obo needs to provide should some additional information about the IAO, e.g. a ‘name’ and ‘definition’?

What can I do to help make this work?

Thanks,

g.


Here are the details:

When you run make ontologies and select ontologies “1,2,3,4” (for example), this occurs:

fetching files for Relationship Ontology
  +http://www.obofoundry.org/ro/ro.obo
    updated
    loading...done!
fetching files for Sequence Ontology
  +http://song.cvs.sourceforge.net/*checkout*/song/ontology/so.obo
    updated
    loading...done!
fetching files for Gene Ontology
  +http://www.geneontology.org/ontology/gene_ontology.obo
    updated
    loading...System call 'stag-storenode.pl -d 'dbi:Pg:dbname=gmod;host=localhost;port=5432' --user gmod  --password 'secret'  tmp/go/go.oboxml' failed: 256

FATAL: all hosts have already failed -- aborting

If one digs down a bit into what’s happening, one discovers (beyond how annoying the use of “one” as a third person pronoun is…) that the process is dying within stag-storenode.pl, illustrated here:

gmod@precise32:~/src/gmod-schema-chado$ stag-storenode.pl -d 'dbi:Pg:dbname=gmod;host=localhost;port=5432' --user gmod  --password 'secret'  tmp/go/go.oboxml
DBD::Pg::st execute failed: ERROR:  null value in column "cv_id" violates not-null constraint
DETAIL:  Failing row contains (2268, null, null, null, 3088, 0, 0). [for Statement "INSERT INTO cvterm (dbxref_id) VALUES (?)" with ParamValues: 1='3088'] at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 3322.
DBD::Pg::st execute failed: ERROR:  null value in column "cv_id" violates not-null constraint
DETAIL:  Failing row contains (2268, null, null, null, 3088, 0, 0). [for Statement "INSERT INTO cvterm (dbxref_id) VALUES (?)" with ParamValues: 1='3088'] at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 3322.
 at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 3332
    DBIx::DBStag::insertrow('DBIx::DBStag=HASH(0x8f839a4)', 'cvterm', 'HASH(0x9493148)', 'cvterm_id') called at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 1928
    DBIx::DBStag::_storenode('DBIx::DBStag=HASH(0x8f839a4)', 'Data::Stag::StagImpl=ARRAY(0x9918b78)') called at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 1180
    DBIx::DBStag::storenode('DBIx::DBStag=HASH(0x8f839a4)', 'Data::Stag::StagImpl=ARRAY(0x993a4f0)') called at /usr/local/gmod/bin/stag-storenode.pl line 88
    eval {...} called at /usr/local/gmod/bin/stag-storenode.pl line 87
    main::store('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'Data::Stag::StagImpl=ARRAY(0x993a4f0)') called at /usr/local/gmod/bin/stag-storenode.pl line 137
    main::__ANON__('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'Data::Stag::StagImpl=ARRAY(0x993a4f0)') called at /usr/local/gmod/lib/perl5/Data/Stag/BaseHandler.pm line 594
    Data::Stag::BaseHandler::end_event('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'cvterm') called at /usr/local/gmod/lib/perl5/Data/Stag/BaseHandler.pm line 742
    Data::Stag::BaseHandler::end_element('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'HASH(0x980d2a8)') called at /usr/local/gmod/lib/perl5/XML/Parser/PerlSAX.pm line 239
    XML::Parser::PerlSAX::_handle_end('XML::Parser::PerlSAX=HASH(0x974ce40)', 'XML::Parser::Expat=HASH(0x96edf24)', 'cvterm') called at /usr/local/gmod/lib/perl5/XML/Parser/PerlSAX.pm line 79
    XML::Parser::PerlSAX::__ANON__('XML::Parser::Expat=HASH(0x96edf24)', 'cvterm') called at /usr/local/gmod/lib/perl5/i686-linux-gnu-thread-multi-64int/XML/Parser/Expat.pm line 470
    XML::Parser::Expat::parse('XML::Parser::Expat=HASH(0x96edf24)', 'FileHandle=GLOB(0x9777a34)') called at /usr/local/gmod/lib/perl5/i686-linux-gnu-thread-multi-64int/XML/Parser.pm line 187
    eval {...} called at /usr/local/gmod/lib/perl5/i686-linux-gnu-thread-multi-64int/XML/Parser.pm line 186
    XML::Parser::parse('XML::Parser=HASH(0x96396b0)', 'FileHandle=GLOB(0x9777a34)') called at /usr/local/gmod/lib/perl5/XML/Parser/PerlSAX.pm line 146
    XML::Parser::PerlSAX::parse('XML::Parser::PerlSAX=HASH(0x974ce40)', 'Handler', 'Data::Stag::BaseHandler=HASH(0x961c6a4)', 'Source', 'HASH(0x974d124)') called at /usr/local/gmod/lib/perl5/Data/Stag/XMLParser.pm line 69
    Data::Stag::XMLParser::parse_fh('Data::Stag::XMLParser=HASH(0x961c9b0)', 'FileHandle=GLOB(0x9777a34)') called at /usr/local/gmod/lib/perl5/Data/Stag/BaseGenerator.pm line 476
    Data::Stag::BaseGenerator::parse('Data::Stag::XMLParser=HASH(0x961c9b0)', '-file', 'tmp/go/go.oboxml', '-str', undef, '-fh', undef) called at /usr/local/gmod/lib/perl5/Data/Stag/XMLParser.pm line 58
    Data::Stag::XMLParser::parse('Data::Stag::XMLParser=HASH(0x961c9b0)', '-file', 'tmp/go/go.oboxml', '-str', undef, '-fh', undef) called at /usr/local/gmod/lib/perl5/Data/Stag/StagImpl.pm line 275
    Data::Stag::StagImpl::parse('Data::Stag', '-format', undef, '-file', 'tmp/go/go.oboxml', '-handler', 'Data::Stag::BaseHandler=HASH(0x961c6a4)') called at /usr/local/gmod/lib/perl5/Data/Stag.pm line 181
    Data::Stag::AUTOLOAD('Data::Stag', '-format', undef, '-file', 'tmp/go/go.oboxml', '-handler', 'Data::Stag::BaseHandler=HASH(0x961c6a4)') called at /usr/local/gmod/bin/stag-storenode.pl line 143

Drilling down into DBIx::DBStag, one hits a call to _storenode with this value for node, leading to a call to insertrow with these arguments:

  DB<6> c
DBIx::DBStag::_storenode(/usr/local/gmod/lib/perl5/DBIx/DBStag.pm:1188):
1188:        if (!$node) {
$VAR1 = bless( [
                 'cvterm',
                 [
                   bless( [
                            '@',
                            [
                              bless( [
                                       'id',
                                       'IAO:0000589'
                                     ], 'Data::Stag::StagImpl' )
                            ]
                          ], 'Data::Stag::StagImpl' ),
                   bless( [
                            'dbxref_id',
                            'dbxref__IAO:0000589'
                          ], 'Data::Stag::StagImpl' )
                 ]
               ], 'Data::Stag::StagImpl' );
  DB<6> c
DBIx::DBStag::_storenode(/usr/local/gmod/lib/perl5/DBIx/DBStag.pm:1930):
1930:                $id =
1931:                  $self->insertrow($element,
1932:                                   \%store_hash,
1933:                                   $pkcol);
$VAR1 = bless( [
                 'cvterm',
                 [
                   bless( [
                            'dbxref_id',
                            3088
                          ], 'Data::Stag::StagImpl' )
                 ]
               ], 'Data::Stag::StagImpl' );
  DB<6> c

which leads to this SQL being generated:

  DB<5> x $sql
0  'INSERT INTO cvterm (dbxref_id) VALUES (?)'
  DB<6>

and run against the cvterm table, which is defined thusly:

                                           Table "public.cvterm"
       Column        |          Type           |                         Modifiers
---------------------+-------------------------+------------------------------------------------------------
 cvterm_id           | integer                 | not null default nextval('cvterm_cvterm_id_seq'::regclass)
 cv_id               | integer                 | not null
 name                | character varying(1024) | not null
 definition          | text                    |
 dbxref_id           | integer                 | not null
 is_obsolete         | integer                 | not null default 0
 is_relationshiptype | integer                 | not null default 0

Given that neither cv_id nor name are provided in the insert statement, hilarity is bound to ensue.

The responsible bit of the input file go.oboxml, around line 67, is

  <cvterm id="IAO:0000589">
    <dbxref_id>dbxref__IAO:0000589</dbxref_id>
  </cvterm>

That dbxref_id does seem to be missing a great deal of information when compared to the other dbxref_id’s in the neighborhood.

That bit of XML seems to have been driven by this hunk in go.obo

[Term]
id: GO:0005623
name: cell
namespace: cellular_component
def: "The basic structural and functional unit of all organisms. Includes the plasma membrane and any external encapsulating structures such as the cell wall and cell envelope." [GOC:go_curators]
subset: goslim_generic
subset: goslim_plant
subset: gosubset_prok
xref_analog: NIF_Subcellular:sao1813327414
xref_analog: Wikipedia:Cell_(biology)
is_a: GO:0005575 ! cellular_component
property_value: IAO:0000589 "cell and encapsulating structures" xsd:string

The go.obo file that I am using begins thusly

format-version: 1.0
version: releases/2014-07-16
date: 16:07:2014 00:43
saved-by: jenkins-slave

phew.


------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel
Reply | Threaded
Open this post in threaded view
|

Re: Unable to load GO into Chado, GO bug or GMOD tools bug or ....

Scott Cain
Hi George,

It sounds to me like it is a problem with the GO file (or at least, a impedence mismatch between the GO obo file and stag-storenode), but I'd like to hear thoughts from Chris.

Chris: I know that stag-storenode.pl is your favorite piece of software, but do you have thoughts on George's very thorough bug report?

In the mean time, I can at least try to replicate it on a cloud instance.

Scott



On Thu, Jul 17, 2014 at 12:55 PM, George Hartzell <[hidden email]> wrote:

Hi Everybody,

I’m trying to install Chado following the instructions in the ‘Chado from SVN’ section of the Getting Started page. I have everything working successfully until the make ontologies step. I can successfully use the make ontologies command to install the Relationship Ontology and the Sequence Ontology but am unable to successfully load the Gene Ontology.

It appears that the loading process is trying to insert a row into the cvterm table without values for several fields that are required to be “not null”.

After a long and winding road, the problem seems to be caused by this portion:

property_value: IAO:0000589 "cell and encapsulating structures" xsd:string

of the entry for the GO term GO:0005623 in the go.obo file. [Is IAO the “Information Artifact Ontology?] Is it possible that go.obo needs to provide should some additional information about the IAO, e.g. a ‘name’ and ‘definition’?

What can I do to help make this work?

Thanks,

g.


Here are the details:

When you run make ontologies and select ontologies “1,2,3,4” (for example), this occurs:

fetching files for Relationship Ontology
  +http://www.obofoundry.org/ro/ro.obo
    updated
    loading...done!
fetching files for Sequence Ontology
  +http://song.cvs.sourceforge.net/*checkout*/song/ontology/so.obo
    updated
    loading...done!
fetching files for Gene Ontology
  +http://www.geneontology.org/ontology/gene_ontology.obo
    updated
    loading...System call 'stag-storenode.pl -d 'dbi:Pg:dbname=gmod;host=localhost;port=5432' --user gmod  --password 'secret'  tmp/go/go.oboxml' failed: 256

FATAL: all hosts have already failed -- aborting

If one digs down a bit into what’s happening, one discovers (beyond how annoying the use of “one” as a third person pronoun is…) that the process is dying within stag-storenode.pl, illustrated here:

gmod@precise32:~/src/gmod-schema-chado$ stag-storenode.pl -d 'dbi:Pg:dbname=gmod;host=localhost;port=5432' --user gmod  --password 'secret'  tmp/go/go.oboxml
DBD::Pg::st execute failed: ERROR:  null value in column "cv_id" violates not-null constraint
DETAIL:  Failing row contains (2268, null, null, null, 3088, 0, 0). [for Statement "INSERT INTO cvterm (dbxref_id) VALUES (?)" with ParamValues: 1='3088'] at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 3322.
DBD::Pg::st execute failed: ERROR:  null value in column "cv_id" violates not-null constraint
DETAIL:  Failing row contains (2268, null, null, null, 3088, 0, 0). [for Statement "INSERT INTO cvterm (dbxref_id) VALUES (?)" with ParamValues: 1='3088'] at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 3322.
 at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 3332
    DBIx::DBStag::insertrow('DBIx::DBStag=HASH(0x8f839a4)', 'cvterm', 'HASH(0x9493148)', 'cvterm_id') called at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 1928
    DBIx::DBStag::_storenode('DBIx::DBStag=HASH(0x8f839a4)', 'Data::Stag::StagImpl=ARRAY(0x9918b78)') called at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 1180
    DBIx::DBStag::storenode('DBIx::DBStag=HASH(0x8f839a4)', 'Data::Stag::StagImpl=ARRAY(0x993a4f0)') called at /usr/local/gmod/bin/stag-storenode.pl line 88
    eval {...} called at /usr/local/gmod/bin/stag-storenode.pl line 87
    main::store('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'Data::Stag::StagImpl=ARRAY(0x993a4f0)') called at /usr/local/gmod/bin/stag-storenode.pl line 137
    main::__ANON__('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'Data::Stag::StagImpl=ARRAY(0x993a4f0)') called at /usr/local/gmod/lib/perl5/Data/Stag/BaseHandler.pm line 594
    Data::Stag::BaseHandler::end_event('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'cvterm') called at /usr/local/gmod/lib/perl5/Data/Stag/BaseHandler.pm line 742
    Data::Stag::BaseHandler::end_element('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'HASH(0x980d2a8)') called at /usr/local/gmod/lib/perl5/XML/Parser/PerlSAX.pm line 239
    XML::Parser::PerlSAX::_handle_end('XML::Parser::PerlSAX=HASH(0x974ce40)', 'XML::Parser::Expat=HASH(0x96edf24)', 'cvterm') called at /usr/local/gmod/lib/perl5/XML/Parser/PerlSAX.pm line 79
    XML::Parser::PerlSAX::__ANON__('XML::Parser::Expat=HASH(0x96edf24)', 'cvterm') called at /usr/local/gmod/lib/perl5/i686-linux-gnu-thread-multi-64int/XML/Parser/Expat.pm line 470
    XML::Parser::Expat::parse('XML::Parser::Expat=HASH(0x96edf24)', 'FileHandle=GLOB(0x9777a34)') called at /usr/local/gmod/lib/perl5/i686-linux-gnu-thread-multi-64int/XML/Parser.pm line 187
    eval {...} called at /usr/local/gmod/lib/perl5/i686-linux-gnu-thread-multi-64int/XML/Parser.pm line 186
    XML::Parser::parse('XML::Parser=HASH(0x96396b0)', 'FileHandle=GLOB(0x9777a34)') called at /usr/local/gmod/lib/perl5/XML/Parser/PerlSAX.pm line 146
    XML::Parser::PerlSAX::parse('XML::Parser::PerlSAX=HASH(0x974ce40)', 'Handler', 'Data::Stag::BaseHandler=HASH(0x961c6a4)', 'Source', 'HASH(0x974d124)') called at /usr/local/gmod/lib/perl5/Data/Stag/XMLParser.pm line 69
    Data::Stag::XMLParser::parse_fh('Data::Stag::XMLParser=HASH(0x961c9b0)', 'FileHandle=GLOB(0x9777a34)') called at /usr/local/gmod/lib/perl5/Data/Stag/BaseGenerator.pm line 476
    Data::Stag::BaseGenerator::parse('Data::Stag::XMLParser=HASH(0x961c9b0)', '-file', 'tmp/go/go.oboxml', '-str', undef, '-fh', undef) called at /usr/local/gmod/lib/perl5/Data/Stag/XMLParser.pm line 58
    Data::Stag::XMLParser::parse('Data::Stag::XMLParser=HASH(0x961c9b0)', '-file', 'tmp/go/go.oboxml', '-str', undef, '-fh', undef) called at /usr/local/gmod/lib/perl5/Data/Stag/StagImpl.pm line 275
    Data::Stag::StagImpl::parse('Data::Stag', '-format', undef, '-file', 'tmp/go/go.oboxml', '-handler', 'Data::Stag::BaseHandler=HASH(0x961c6a4)') called at /usr/local/gmod/lib/perl5/Data/Stag.pm line 181
    Data::Stag::AUTOLOAD('Data::Stag', '-format', undef, '-file', 'tmp/go/go.oboxml', '-handler', 'Data::Stag::BaseHandler=HASH(0x961c6a4)') called at /usr/local/gmod/bin/stag-storenode.pl line 143

Drilling down into DBIx::DBStag, one hits a call to _storenode with this value for node, leading to a call to insertrow with these arguments:

  DB<6> c
DBIx::DBStag::_storenode(/usr/local/gmod/lib/perl5/DBIx/DBStag.pm:1188):
1188:        if (!$node) {
$VAR1 = bless( [
                 'cvterm',
                 [
                   bless( [
                            '@',
                            [
                              bless( [
                                       'id',
                                       'IAO:0000589'
                                     ], 'Data::Stag::StagImpl' )
                            ]
                          ], 'Data::Stag::StagImpl' ),
                   bless( [
                            'dbxref_id',
                            'dbxref__IAO:0000589'
                          ], 'Data::Stag::StagImpl' )
                 ]
               ], 'Data::Stag::StagImpl' );
  DB<6> c
DBIx::DBStag::_storenode(/usr/local/gmod/lib/perl5/DBIx/DBStag.pm:1930):
1930:                $id =
1931:                  $self->insertrow($element,
1932:                                   \%store_hash,
1933:                                   $pkcol);
$VAR1 = bless( [
                 'cvterm',
                 [
                   bless( [
                            'dbxref_id',
                            3088
                          ], 'Data::Stag::StagImpl' )
                 ]
               ], 'Data::Stag::StagImpl' );
  DB<6> c

which leads to this SQL being generated:

  DB<5> x $sql
0  'INSERT INTO cvterm (dbxref_id) VALUES (?)'
  DB<6>

and run against the cvterm table, which is defined thusly:

                                           Table "public.cvterm"
       Column        |          Type           |                         Modifiers
---------------------+-------------------------+------------------------------------------------------------
 cvterm_id           | integer                 | not null default nextval('cvterm_cvterm_id_seq'::regclass)
 cv_id               | integer                 | not null
 name                | character varying(1024) | not null
 definition          | text                    |
 dbxref_id           | integer                 | not null
 is_obsolete         | integer                 | not null default 0
 is_relationshiptype | integer                 | not null default 0

Given that neither cv_id nor name are provided in the insert statement, hilarity is bound to ensue.

The responsible bit of the input file go.oboxml, around line 67, is

  <cvterm id="IAO:0000589">
    <dbxref_id>dbxref__IAO:0000589</dbxref_id>
  </cvterm>

That dbxref_id does seem to be missing a great deal of information when compared to the other dbxref_id’s in the neighborhood.

That bit of XML seems to have been driven by this hunk in go.obo

[Term]
id: GO:0005623
name: cell
namespace: cellular_component
def: "The basic structural and functional unit of all organisms. Includes the plasma membrane and any external encapsulating structures such as the cell wall and cell envelope." [GOC:go_curators]
subset: goslim_generic
subset: goslim_plant
subset: gosubset_prok
xref_analog: NIF_Subcellular:sao1813327414
xref_analog: Wikipedia:Cell_(biology)
is_a: GO:0005575 ! cellular_component
property_value: IAO:0000589 "cell and encapsulating structures" xsd:string

The go.obo file that I am using begins thusly

format-version: 1.0
version: releases/2014-07-16
date: 16:07:2014 00:43
saved-by: jenkins-slave

phew.


------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel




--
------------------------------------------------------------------------
Scott Cain, Ph. D.                                   scott at scottcain dot net
GMOD Coordinator (http://gmod.org/)                     216-392-3087
Ontario Institute for Cancer Research

------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-schema] Unable to load GO into Chado, GO bug or GMOD tools bug or ....

Eric Rasche
Hi all,

I'd like to jump in here too if that's ok. Looking over the logs for my automatic chado schema builder, I noticed an identical failure.

loading...DBD::Pg::st execute failed: ERROR:  null value in column "cv_id" violates not-null constraint [for Statement "INSERT INTO cvterm (dbxref_id) VALUES (?)" with ParamValues: 1='3088'] at /usr/share/perl5/DBIx/DBStag.pm line 3322.
DBD::Pg::st execute failed: ERROR:  null value in column "cv_id" violates not-null constraint [for Statement "INSERT INTO cvterm (dbxref_id) VALUES (?)" with ParamValues: 1='3088'] at /usr/share/perl5/DBIx/DBStag.pm line 3322.

<snip>
System call 'stag-storenode.pl -d 'dbi:Pg:dbname=csb;host=geiger;port=5432' --user schema_builder \
 --password 'secret'  tmp/go/go.oboxml' failed: 256
I can absolutely say it happened between June 25th and July 2nd, based on when the jenkins job started failing

Hope that helps!

Cheers,
Eric



On 07/17/2014 01:26 PM, Scott Cain wrote:
Hi George,

It sounds to me like it is a problem with the GO file (or at least, a impedence mismatch between the GO obo file and stag-storenode), but I'd like to hear thoughts from Chris.

Chris: I know that stag-storenode.pl is your favorite piece of software, but do you have thoughts on George's very thorough bug report?

In the mean time, I can at least try to replicate it on a cloud instance.

Scott



On Thu, Jul 17, 2014 at 12:55 PM, George Hartzell <[hidden email]> wrote:

Hi Everybody,

I’m trying to install Chado following the instructions in the ‘Chado from SVN’ section of the Getting Started page. I have everything working successfully until the make ontologies step. I can successfully use the make ontologies command to install the Relationship Ontology and the Sequence Ontology but am unable to successfully load the Gene Ontology.

It appears that the loading process is trying to insert a row into the cvterm table without values for several fields that are required to be “not null”.

After a long and winding road, the problem seems to be caused by this portion:

property_value: IAO:0000589 "cell and encapsulating structures" xsd:string

of the entry for the GO term GO:0005623 in the go.obo file. [Is IAO the “Information Artifact Ontology?] Is it possible that go.obo needs to provide should some additional information about the IAO, e.g. a ‘name’ and ‘definition’?

What can I do to help make this work?

Thanks,

g.


Here are the details:

When you run make ontologies and select ontologies “1,2,3,4” (for example), this occurs:

fetching files for Relationship Ontology
  +http://www.obofoundry.org/ro/ro.obo
    updated
    loading...done!
fetching files for Sequence Ontology
  +http://song.cvs.sourceforge.net/*checkout*/song/ontology/so.obo
    updated
    loading...done!
fetching files for Gene Ontology
  +http://www.geneontology.org/ontology/gene_ontology.obo
    updated
    loading...System call 'stag-storenode.pl -d 'dbi:Pg:dbname=gmod;host=localhost;port=5432' --user gmod  --password 'secret'  tmp/go/go.oboxml' failed: 256

FATAL: all hosts have already failed -- aborting

If one digs down a bit into what’s happening, one discovers (beyond how annoying the use of “one” as a third person pronoun is…) that the process is dying within stag-storenode.pl, illustrated here:

gmod@precise32:~/src/gmod-schema-chado$ stag-storenode.pl -d 'dbi:Pg:dbname=gmod;host=localhost;port=5432' --user gmod  --pa
 ssword 's
ecret'  tmp/go/go.oboxml
DBD::Pg::st execute failed: ERROR:  null value in column "cv_id" violates not-null constraint
DETAIL:  Failing row contains (2268, null, null, null, 3088, 0, 0). [for Statement "INSERT INTO cvterm (dbxref_id) VALUES (?)" with ParamValues: 1='3088'] at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 3322.
DBD::Pg::st execute failed: ERROR:  null value in column "cv_id" violates not-null constraint
DETAIL:  Failing row contains (2268, null, null, null, 3088, 0, 0). [for Statement "INSERT INTO cvterm (dbxref_id) VALUES (?)" with ParamValues: 1='3088'] at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 3322.
 at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 3332
    DBIx::DBStag::insertrow('DBIx::DBStag=HASH(0x8f839a4)', 'cvterm', 'HASH(0x9493148)', 'cvterm_id') called at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 1928
    DBIx::DBStag::_storenode('DBIx::DBStag=HASH(0x8f839a4)', 'Data::Stag::StagImpl=ARRAY(0x9918b78)') called at /usr/local/gmod/lib/perl5/DBIx/DBStag.pm line 1180
    DBIx::DBStag::storenode('DBIx::DBStag=HASH(0x8f839a4)', 'Data::Stag::StagImpl=ARRAY(0x993a4f0)') called at /usr/local/gmod/bin/stag-storenode.pl line 88
    eval {...} called at /usr/local/gmod/bin/stag-storenode.pl line 87
    main::store('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'Data::Stag::StagImpl=ARRAY(0x993a4f0)') called at /usr/local/gmod/bin/stag-storenode.pl line 137
    main::__ANON__('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'Data::Stag::StagImpl=ARRAY(0x993a4f0)') called at /usr/local/gmod/lib/perl5/Data/Stag/BaseHandler.pm line 594
    Data::Stag::BaseHandler::end_event('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'cvterm') called at /usr/local/gmod/lib/perl5/Data/Stag/BaseHandler.pm line 742
    Data::Stag::BaseHandler::end_element('Data::Stag::BaseHandler=HASH(0x961c6a4)', 'HASH(0x980d2a8)') called at /usr/local/gmod/lib/perl5/XML/Parser/PerlSAX.pm line 239
    XML::Parser::PerlSAX::_handle_end('XML::Parser::PerlSAX=HASH(0x974ce40)', 'XML::Parser::Expat=HASH(0x96edf24)', 'cvterm') called at /usr/local/gmod/lib/perl5/XML/Parser/PerlSAX.pm line 79
    XML::Parser::PerlSAX::__ANON__('XML::Parser::Expat=HASH(0x96edf24)', 'cvterm') called at /usr/local/gmod/lib/perl5/i686-linux-gnu-thread-multi-64int/XML/Parser/Expat.pm line 470
    XML::Parser::Expat::parse('XML::Parser::Expat=HASH(0x96edf24)', 'FileHandle=GLOB(0x9777a34)') called at /usr/local/gmod/lib/perl5/i686-linux-gnu-thread-multi-64int/XML/Parser.pm line 187
    eval {...} called at /usr/local/gmod/lib/perl5/i686-linux-gnu-thread-multi-64int/XML/Parser.pm line 186
    XML::Parser::parse('XML::Parser=HASH(0x96396b0)', 'FileHandle=GLOB(0x9777a34)') called at /usr/local/gmod/lib/perl5/XML/Parser/PerlSAX.pm line 146
    XML::Parser::PerlSAX::parse('XML::Parser::PerlSAX=HASH(0x974ce40)', 'Handler', 'Data::Stag::BaseHandler=HASH(0x961c6a4)', 'Source', 'HASH(0x974d124)') called at /usr/local/gmod/lib/perl5/Data/Stag/XMLParser.pm line 69
    Data::Stag::XMLParser::parse_fh('Data::Stag::XMLParser=HASH(0x961c9b0)', 'FileHandle=GLOB(0x9777a34)') called at /usr/local/gmod/lib/perl5/Data/Stag/BaseGenerator.pm line 476
    Data::Stag::BaseGenerator::parse('Data::Stag::XMLParser=HASH(0x961c9b0)', '-file', 'tmp/go/go.oboxml', '-str', undef, '-fh', undef) called at /usr/local/gmod/lib/perl5/Data/Stag/XMLParser.pm line 58
    Data::Stag::XMLParser::parse('Data::Stag::XMLParser=HASH(0x961c9b0)', '-file', 'tmp/go/go.oboxml', '-str', undef, '-fh', undef) called at /usr/local/gmod/lib/perl5/Data/Stag/StagImpl.pm line 275
    Data::Stag::StagImpl::parse('Data::Stag', '-format', undef, '-file', 'tmp/go/go.oboxml', '-handler', 'Data::Stag::BaseHandler=HASH(0x961c6a4)') called at /usr/local/gmod/lib/perl5/Data/Stag.pm line 181
    Data::Stag::AUTOLOAD('Data::Stag', '-format', undef, '-file', 'tmp/go/go.oboxml', '-handler', 'Data::Stag::BaseHandler=HASH(0x961c6a4)') called at /usr/local/gmod/bin/stag-storenode.pl line 143

Drilling down into DBIx::DBStag, one hits a call to _storenode with this value for node, leading to a call to insertrow with these arguments:

  DB<6> c
DBIx::DBStag::_storenode(/usr/local/gmod/lib/perl5/DBIx/DBStag.pm:1188):
1188:        if (!$node) {
$VAR1 = bless( [
                 'cvterm',
                 [
                   bless( [
                            '@',
                            [
                              bless( [
                                       'id',
                                       'IAO:0000589'
                                     ], 'Data::Stag::StagImpl' )
                            ]
                          ], 'Data::Stag::StagImpl' ),
                   bless( [
                            'dbxref_id',
                            'dbxref__IAO:0000589'
                          ], 'Data::Stag::StagImpl' )
                 ]
               ], 'Data::Stag::StagImpl' );
  DB<6> c
DBIx::DBStag::_storenode(/usr/local/gmod/lib/perl5/DBIx/DBStag.pm:1930):
1930:                $id =
1931:                  $self->insertrow($element,
1932:                                   \%store_hash,
1933:                                   $pkcol);
$VAR1 = bless( [
                 'cvterm',
                 [
                   bless( [
                            'dbxref_id',
                            3088
                          ], 'Data::Stag::StagImpl' )
                 ]
               ], 'Data::Stag::StagImpl' );
  DB<6> c

which leads to this SQL being generated:

  DB<5> x $sql
0  'INSERT INTO cvterm (dbxref_id) VALUES (?)'
  DB<6>

and run against the cvterm table, which is defined thusly:

                                           Table "public.cvterm"
       Column        |          Type           |                         Modifiers
---------------------+-------------------------+------------------------------------------------------------
 cvterm_id           | integer                 | not null default nextval('cvterm_cvterm_id_seq'::regclass)
 cv_id               | integer                 | not null
 name                | character varying(1024) | not null
 definition          | text                    |
 dbxref_id           | integer                 | not null
 is_obsolete         | integer                 | not null default 0
 is_relationshiptype | integer                 | not null default 0

Given that neither cv_id nor name are provided in the insert statement, hilarity is bound to ensue.

The responsible bit of the input file go.oboxml, around line 67, is

  <cvterm id="IAO:0000589">
    <dbxref_id>dbxref__IAO:0000589</dbxref_id>
  </cvterm>

That dbxref_id does seem to be missing a great deal of information when compared to the other dbxref_id’s in the neighborhood.

That bit of XML seems to have been driven by this hunk in go.obo

[Term]
id: GO:0005623
name: cell
namespace: cellular_component
def: "The basic structural and functional unit of all organisms. Includes the plasma membrane and any external encapsulating structures such as the cell wall and cell envelope." [GOC:go_curators]
subset: goslim_generic
subset: goslim_plant
subset: gosubset_prok
xref_analog: NIF_Subcellular:sao1813327414
xref_analog: Wikipedia:Cell_(biology)
is_a: GO:0005575 ! cellular_component
property_value: IAO:0000589 "cell and encapsulating structures" xsd:string

The go.obo file that I am using begins thusly

format-version: 1.0
version: releases/2014-07-16
date: 16:07:2014 00:43
saved-by: jenkins-slave

phew.


------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel




--
------------------------------------------------------------------------
Scott Cain, Ph. D.                                   scott at scottcain dot net
GMOD Coordinator (http://gmod.org/)                     216-392-3087
Ontario Institute for Cancer Research


------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds


_______________________________________________
Gmod-schema mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-schema

------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-schema] Unable to load GO into Chado, GO bug or GMOD tools bug or ....

George Hartzell

TL;DR, it was a bug in the public release of go. Fixed and should be in tomorrow/Sat’s file.


I received the following reply from the folks behind the gene ontology helpdesk:

Hi George,

Sorry about this. IAO is indeed the information artefact ontology. This line is doing important work in the editor’s version of GO, but it looks like we need to strip it out from our public release. We also need to put some checks in place to prevent something similar from happening again.

Or is build nightly, so this should be fixed by tomorrow or Saturday at the latest.

David
Dr David Osumi-Sutherland
[…]


------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-schema] Unable to load GO into Chado, GO bug or GMOD tools bug or ....

Chris Mungall
Hi George!

Technically not a bug, as it's conformant to the obo format spec (
http://oboformat.org ).

I would recommend that for loading Chado people use go-basic.obo,
download details here:

http://geneontology.org/page/download-ontology

The notion of a "basic" OBO file is elucidated in more detail in the
spec, it essentially removes anything that might trip up older ontology
software. This advice goes for other ontologies too. A basic obo file
should always load into Chado without difficulties.

If anyone is particularly interested in capturing the additional
information filtered out from a basic file, I can give advice on how to
modify the XSLT used by the Chado loader, or better yet, rewrite the
whole thing using Bio::Chado or GOBL or whatever.

On 17 Jul 2014, at 13:09, George Hartzell wrote:

> TL;DR, it was a bug in the public release of go. Fixed and should be
> in
> tomorrow/Sat’s file.
> ------------------------------
>
> I received the following reply from the folks behind the gene ontology
> helpdesk:
>
> Hi George,
>
> Sorry about this. IAO is indeed the information artefact ontology.
> This
> line is doing important work in the editor’s version of GO, but it
> looks
> like we need to strip it out from our public release. We also need to
> put
> some checks in place to prevent something similar from happening
> again.
>
> Or is build nightly, so this should be fixed by tomorrow or Saturday
> at the
> latest.
>
> David
> Dr David Osumi-Sutherland
> […]
>
> ​

------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-schema] Unable to load GO into Chado, GO bug or GMOD tools bug or ....

George Hartzell
On Thu, Jul 17, 2014 at 5:55 PM, Chris Mungall <[hidden email]> wrote:
Hi George!

Technically not a bug, as it's conformant to the obo format spec ( http://oboformat.org ).

Oops.  Didn't mean to cast aspersions.  I was just happy to figure out what had changed where.


I would recommend that for loading Chado people use go-basic.obo, download details here:

http://geneontology.org/page/download-ontology

The notion of a "basic" OBO file is elucidated in more detail in the spec, it essentially removes anything that might trip up older ontology software. This advice goes for other ontologies too. A basic obo file should always load into Chado without difficulties.

If anyone is particularly interested in capturing the additional information filtered out from a basic file, I can give advice on how to modify the XSLT used by the Chado loader, or better yet, rewrite the whole thing using Bio::Chado or GOBL or whatever.

I'm just following the cookbook recipes here, but I'd be interested in reading along if folks who are more familiar with the issue were to chime in with whether the installation instructions should be updated and/or what the impact might be of changing the set of GO information that was loaded out of the box.

In other words, does anyone see any incompatibilities if I switch to the file Chris recommends?

g.

------------------------------------------------------------------------------
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
_______________________________________________
Gmod-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-devel