Sync phenotype data

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Sync phenotype data

Hugo-pH

Hello all for the first time,

We're starting a project for the creation of a phenotypic database and an associated web site for a transgenic strawberry collection. We're using Drupal/Tripal and Chado, specially the Natural Diversity Module.

First I must to say that Tripal is a great tool, really easy to use, so my sincerely congrats and thanks to the developers.

And now the problems. I have installed and configured Drupal-Tripal-Chado without problems, and now I'm trying to upload and retrieve phetotypic data. I have created new organisms and stocks, and have been able to upload data to some of the ND tables, using the bulk loader function (great tool!). But now I get stuck in the phenotype search page. I'm sure that the phenotype table is populated since I can see the data through pgphpadmin and the psql console, but I don't get any results when I run the search in the page. I have seen in the tutorial that, for organism, stock and feature data, it is necessary to sync the database and the drupal site but I can't find anything similar for phenotype data. Can anyone give us some clue for that?


For the record, the OS where the system is installed is Ubuntu 12.04, postgres version is 9.1.11, drupal version is 6.30, and tripal is 1.1.

Many thanks,

Hugo.


------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal
Reply | Threaded
Open this post in threaded view
|

Re: Sync phenotype data

Stephen Ficklin-2
Hi Hugo,

Thanks for the compliment.  We are actively working on Tripal 2.0 for Drupal 7 which has some nice improvements so it will continue to improve :-) 

In response to the search form not working.... You are using the Phenotype search tool under the 'Search biological data' menu item at http://your.site/chado/phenotypes?    That search uses a Drupal View and will try to execute this query for searching:

SELECT
  phenotype.uniquename AS phenotype_uniquename,
  cvterm_phenotype_1.name AS cvterm_phenotype_1_name,
  cvterm_phenotype_2.name AS cvterm_phenotype_2_name,
  phenotype.value AS phenotype_value,
  cvterm_phenotype_3.name AS cvterm_phenotype_3_name,
  cvterm_phenotype.name AS cvterm_phenotype_name
FROM phenotype phenotype
  LEFT JOIN cvterm cvterm_phenotype ON phenotype.assay_id = cvterm_phenotype.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_1 ON phenotype.attr_id = cvterm_phenotype_1.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_2 ON phenotype.cvalue_id = cvterm_phenotype_2.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_3 ON phenotype.observable_id = cvterm_phenotype_3.cvterm_id
ORDER BY phenotype_uniquename ASC


If you run that query manually in phppgadmin do you see results?  Are there any messages in the Drupal recent log ('Administer' -> 'Reports' -> 'Recent log entries') after you try to perform a search?

That search form will show phenotypes and observed values, but it doesn't link to the stocks or features that may be associated with those phenotypes.  Are you looking for this type of behavior?

Right now, the only way to see phenotype data for features or stocks is if you have it associated with a feature (via feature_phenotype table) or a stock (via nd_experiment tables).  For instance, if you have phenotypes associated with particular stock then when you visit the stock page you should see phenotype information (e.g. http://www.cottongen.org/node/49712 (see Trait Score link--that page has been customized from the default but shows phenotypes and observed values for the stock)).   Once you sync features and stocks the phenotype data, if present, should automatically appear on the appropriate feature and stock pages.  Do you envision that you would need to search or view phenotype data differently?  If so, let me know and perhaps we can help you get what you need.

Stephen


On 1/22/2014 4:50 AM, Hugo Pineda wrote:

Hello all for the first time,

We're starting a project for the creation of a phenotypic database and an associated web site for a transgenic strawberry collection. We're using Drupal/Tripal and Chado, specially the Natural Diversity Module.

First I must to say that Tripal is a great tool, really easy to use, so my sincerely congrats and thanks to the developers.

And now the problems. I have installed and configured Drupal-Tripal-Chado without problems, and now I'm trying to upload and retrieve phetotypic data. I have created new organisms and stocks, and have been able to upload data to some of the ND tables, using the bulk loader function (great tool!). But now I get stuck in the phenotype search page. I'm sure that the phenotype table is populated since I can see the data through pgphpadmin and the psql console, but I don't get any results when I run the search in the page. I have seen in the tutorial that, for organism, stock and feature data, it is necessary to sync the database and the drupal site but I can't find anything similar for phenotype data. Can anyone give us some clue for that?


For the record, the OS where the system is installed is Ubuntu 12.04, postgres version is 9.1.11, drupal version is 6.30, and tripal is 1.1.

Many thanks,

Hugo.



------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today. 
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk


_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal


------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal
Reply | Threaded
Open this post in threaded view
|

Re: Sync phenotype data

Hugo-pH
Hi Stephen,

Thank for your quick response.
I'll answer your questions bellow:

On 01/22/2014 03:18 PM, Stephen Ficklin wrote:
Hi Hugo,

Thanks for the compliment.  We are actively working on Tripal 2.0 for Drupal 7 which has some nice improvements so it will continue to improve

In response to the search form not working.... You are using the Phenotype search tool under the 'Search biological data' menu item at http://your.site/chado/phenotypes?    That search uses a Drupal View and will try to execute this query for searching:

Yes, I'm running the search at my.site/chado/phenotypes. I have tried entering 'Phenotypic Attribute', 'Unique Name Contains' and leaving all fields blank, but I always get the message: 'No phenotypes match the supplied criteria'.

SELECT
  phenotype.uniquename AS phenotype_uniquename,
  cvterm_phenotype_1.name AS cvterm_phenotype_1_name,
  cvterm_phenotype_2.name AS cvterm_phenotype_2_name,
  phenotype.value AS phenotype_value,
  cvterm_phenotype_3.name AS cvterm_phenotype_3_name,
  cvterm_phenotype.name AS cvterm_phenotype_name
FROM phenotype phenotype
  LEFT JOIN cvterm cvterm_phenotype ON phenotype.assay_id = cvterm_phenotype.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_1 ON phenotype.attr_id = cvterm_phenotype_1.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_2 ON phenotype.cvalue_id = cvterm_phenotype_2.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_3 ON phenotype.observable_id = cvterm_phenotype_3.cvterm_id
ORDER BY phenotype_uniquename ASC

If you run that query manually in phppgadmin do you see results?  Are there any messages in the Drupal recent log ('Administer' -> 'Reports' -> 'Recent log entries') after you try to perform a search?

About the query in phppgadmin, if I run the code at the drupal level (PostgreSQL -> drupal), I get the following error:

SQL error:
ERROR:  relation "phenotype" does not exist
LINE 8: FROM phenotype phenotype
             ^
In statement:
SELECT
  phenotype.uniquename AS phenotype_uniquename,
  cvterm_phenotype_1.name AS cvterm_phenotype_1_name,
  cvterm_phenotype_2.name AS cvterm_phenotype_2_name,
  phenotype.value AS phenotype_value,
  cvterm_phenotype_3.name AS cvterm_phenotype_3_name,
  cvterm_phenotype.name AS cvterm_phenotype_name
FROM phenotype phenotype
  LEFT JOIN cvterm cvterm_phenotype ON phenotype.assay_id = cvterm_phenotype.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_1 ON phenotype.attr_id = cvterm_phenotype_1.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_2 ON phenotype.cvalue_id = cvterm_phenotype_2.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_3 ON phenotype.observable_id = cvterm_phenotype_3.cvterm_id
ORDER BY phenotype_uniquename ASC

Nevertheless, when I run the query at chado level (PostgreSQL -> drupal -> chado) the data is shown correctly except for the phenotype.name field (cvterm_phenotype_name in the results) that it appears as NULL but it's populated in the table (SELECT * FROM phenotype shows that field). Any idea for this behavior?

Another strange thing. In the 'Edit view phenotype_listing', when I click in the 'Preview' button at the bottom of the page, the form is showed but also, all the phenotipic data! if I try to search using the form (still in the edit -> preview) again it returns the 'No phenotypes match the supplied criteria' message.

About the Drupal recent log, when the search is performed, no new messages appears (neither in the postgres log) but there are some warnings and errors in the drupal log that might be related:

warning: include(): Failed opening './sites/all/modules/tripal/tripal_stock/tripal_stock_phenotypes.tpl.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/includes/theme.inc on line 1079.
warning: include(): Failed opening './sites/all/modules/tripal/tripal_stock/tripal_stock_phenotypes.tpl.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/includes/theme.inc on line 1079.

Both messages appears when I access a stock page (my.site/node/24) through my.site/chado/stocks page (Search Biological Data -> Stocks).

That search form will show phenotypes and observed values, but it doesn't link to the stocks or features that may be associated with those phenotypes.  Are you looking for this type of behavior?

I'm really more interested in the next option (access phenotypes through stocks or feature) but it would be great if this kind of search could work too.

Right now, the only way to see phenotype data for features or stocks is if you have it associated with a feature (via feature_phenotype table) or a stock (via nd_experiment tables).  For instance, if you have phenotypes associated with particular stock then when you visit the stock page you should see phenotype information (e.g. http://www.cottongen.org/node/49712 (see Trait Score link--that page has been customized from the default but shows phenotypes and observed values for the stock)).   Once you sync features and stocks the phenotype data, if present, should automatically appear on the appropriate feature and stock pages.  Do you envision that you would need to search or view phenotype data differently?  If so, let me know and perhaps we can help you get what you need.

Ok, I have seen the phenotype data in the stock page, thank you! I think we will work with that for the moment. If I have some troubles editing the view or need anything more specific I would let you know, but thank you for your help offer.
 
On 1/22/2014 4:50 AM, Hugo Pineda wrote:

Hello all for the first time,

We're starting a project for the creation of a phenotypic database and an associated web site for a transgenic strawberry collection. We're using Drupal/Tripal and Chado, specially the Natural Diversity Module.

First I must to say that Tripal is a great tool, really easy to use, so my sincerely congrats and thanks to the developers.

And now the problems. I have installed and configured Drupal-Tripal-Chado without problems, and now I'm trying to upload and retrieve phetotypic data. I have created new organisms and stocks, and have been able to upload data to some of the ND tables, using the bulk loader function (great tool!). But now I get stuck in the phenotype search page. I'm sure that the phenotype table is populated since I can see the data through pgphpadmin and the psql console, but I don't get any results when I run the search in the page. I have seen in the tutorial that, for organism, stock and feature data, it is necessary to sync the database and the drupal site but I can't find anything similar for phenotype data. Can anyone give us some clue for that?


For the record, the OS where the system is installed is Ubuntu 12.04, postgres version is 9.1.11, drupal version is 6.30, and tripal is 1.1.

Many thanks,

Hugo.



------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today. 
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk


_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal



------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal
Reply | Threaded
Open this post in threaded view
|

Re: Sync phenotype data

Lacey-Anne Sanderson
Hi Hugo,
The phenotypic view provided with Tripal is unfortunately a bit buggy -this is fixed in the Drupal 7 Tripal 2 version! Drupal Views 2 (the version you're using right now) doesn't handle many joins to the same table very well and the phenotype view does this the most so unfortunately the bug is in Drupal rather than Tripal making it impossible for us to fix. It is fixed in Drupal 7 Views 3 though.

I do have a few suggestions for you that might work with the current version. The fact that the phenotypic data appears in the preview but not on the page makes me think some of the problem might be with the "Search Results" Filter. You might want to try removing this from the phenotype view (go to admin/build/view and click the edit link beside the phenotype_listing view. Then under the filter heading on the far right, click on the "Global: Search Results" filter. An options pane will open below via ajax; click the remove button. Then save the view. This will cause you view to show results immediately rather than waiting for the user to click search. They will still have the ability to filter the results through the select/text boxes, they just won't have to do that first. This should get you the same behaviour you see in the preview.

See additional comments below:

Hope this helps explain what is happening a bit more! If the display of phenotypes on your stock page is sufficient for the moment, I might wait for the search functionality until you upgrade to Tripal 2.
~Lacey

------------------------------------------------------
Lacey-Anne Sanderson
Bioinformaticist
Pulse Crop Breeding and Genetics
Phone: (306) 966-4975
Room 3C06 Agriculture
Department of Plant Sciences
University of Saskatchewan

On 2014-01-23, at 7:28 AM, Hugo Pineda <[hidden email]> wrote:

Hi Stephen,

Thank for your quick response.
I'll answer your questions bellow:

On 01/22/2014 03:18 PM, Stephen Ficklin wrote:
Hi Hugo,

Thanks for the compliment.  We are actively working on Tripal 2.0 for Drupal 7 which has some nice improvements so it will continue to improve

In response to the search form not working.... You are using the Phenotype search tool under the 'Search biological data' menu item at http://your.site/chado/phenotypes?    That search uses a Drupal View and will try to execute this query for searching:

Yes, I'm running the search at my.site/chado/phenotypes. I have tried entering 'Phenotypic Attribute', 'Unique Name Contains' and leaving all fields blank, but I always get the message: 'No phenotypes match the supplied criteria'.

SELECT
  phenotype.uniquename AS phenotype_uniquename,
  cvterm_phenotype_1.name AS cvterm_phenotype_1_name,
  cvterm_phenotype_2.name AS cvterm_phenotype_2_name,
  phenotype.value AS phenotype_value,
  cvterm_phenotype_3.name AS cvterm_phenotype_3_name,
  cvterm_phenotype.name AS cvterm_phenotype_name
FROM phenotype phenotype
  LEFT JOIN cvterm cvterm_phenotype ON phenotype.assay_id = cvterm_phenotype.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_1 ON phenotype.attr_id = cvterm_phenotype_1.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_2 ON phenotype.cvalue_id = cvterm_phenotype_2.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_3 ON phenotype.observable_id = cvterm_phenotype_3.cvterm_id
ORDER BY phenotype_uniquename ASC

If you run that query manually in phppgadmin do you see results?  Are there any messages in the Drupal recent log ('Administer' -> 'Reports' -> 'Recent log entries') after you try to perform a search?

About the query in phppgadmin, if I run the code at the drupal level (PostgreSQL -> drupal), I get the following error:

SQL error:
ERROR:  relation "phenotype" does not exist
LINE 8: FROM phenotype phenotype
             ^
In statement:
SELECT
  phenotype.uniquename AS phenotype_uniquename,
  cvterm_phenotype_1.name AS cvterm_phenotype_1_name,
  cvterm_phenotype_2.name AS cvterm_phenotype_2_name,
  phenotype.value AS phenotype_value,
  cvterm_phenotype_3.name AS cvterm_phenotype_3_name,
  cvterm_phenotype.name AS cvterm_phenotype_name
FROM phenotype phenotype
  LEFT JOIN cvterm cvterm_phenotype ON phenotype.assay_id = cvterm_phenotype.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_1 ON phenotype.attr_id = cvterm_phenotype_1.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_2 ON phenotype.cvalue_id = cvterm_phenotype_2.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_3 ON phenotype.observable_id = cvterm_phenotype_3.cvterm_id
ORDER BY phenotype_uniquename ASC

This is due to chado being in a separate schema/namespace than the Drupal tables. Drupal Views already knows that so it looks at the chado tables directly.


Nevertheless, when I run the query at chado level (PostgreSQL -> drupal -> chado) the data is shown correctly except for the phenotype.name field (cvterm_phenotype_name in the results) that it appears as NULL but it's populated in the table (SELECT * FROM phenotype shows that field). Any idea for this behavior?

So the cvterm_phenotype_name is actually the cvterm name referenced by your phenotype.assay_id. So when you select from the phenotype table is your assay_id set?


Another strange thing. In the 'Edit view phenotype_listing', when I click in the 'Preview' button at the bottom of the page, the form is showed but also, all the phenotipic data! if I try to search using the form (still in the edit -> preview) again it returns the 'No phenotypes match the supplied criteria' message.

This is what I addressed above.


About the Drupal recent log, when the search is performed, no new messages appears (neither in the postgres log) but there are some warnings and errors in the drupal log that might be related:

warning: include(): Failed opening './sites/all/modules/tripal/tripal_stock/tripal_stock_phenotypes.tpl.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/includes/theme.inc on line 1079.
warning: include(): Failed opening './sites/all/modules/tripal/tripal_stock/tripal_stock_phenotypes.tpl.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/includes/theme.inc on line 1079. 

Both messages appears when I access a stock page (my.site/node/24) through my.site/chado/stocks page (Search Biological Data -> Stocks).

This is the template that displays your phenotypes on any stock page which is why it appears when you go to a stock page. Do you have the tripal_theme installed as instructed in the tutorial? Maybe try clearing your Drupal cache a couple of times (the best way is using drush cache-clear all but if you don't have crush you can click the "clear Cached Data"  button at admin/settings/performance though this is a less thorough clear).


That search form will show phenotypes and observed values, but it doesn't link to the stocks or features that may be associated with those phenotypes.  Are you looking for this type of behavior?

I'm really more interested in the next option (access phenotypes through stocks or feature) but it would be great if this kind of search could work too.

Right now, the only way to see phenotype data for features or stocks is if you have it associated with a feature (via feature_phenotype table) or a stock (via nd_experiment tables).  For instance, if you have phenotypes associated with particular stock then when you visit the stock page you should see phenotype information (e.g. http://www.cottongen.org/node/49712 (see Trait Score link--that page has been customized from the default but shows phenotypes and observed values for the stock)).   Once you sync features and stocks the phenotype data, if present, should automatically appear on the appropriate feature and stock pages.  Do you envision that you would need to search or view phenotype data differently?  If so, let me know and perhaps we can help you get what you need.

Ok, I have seen the phenotype data in the stock page, thank you! I think we will work with that for the moment. If I have some troubles editing the view or need anything more specific I would let you know, but thank you for your help offer.
 
On 1/22/2014 4:50 AM, Hugo Pineda wrote:

Hello all for the first time,

We're starting a project for the creation of a phenotypic database and an associated web site for a transgenic strawberry collection. We're using Drupal/Tripal and Chado, specially the Natural Diversity Module.

First I must to say that Tripal is a great tool, really easy to use, so my sincerely congrats and thanks to the developers.

And now the problems. I have installed and configured Drupal-Tripal-Chado without problems, and now I'm trying to upload and retrieve phetotypic data. I have created new organisms and stocks, and have been able to upload data to some of the ND tables, using the bulk loader function (great tool!). But now I get stuck in the phenotype search page. I'm sure that the phenotype table is populated since I can see the data through pgphpadmin and the psql console, but I don't get any results when I run the search in the page. I have seen in the tutorial that, for organism, stock and feature data, it is necessary to sync the database and the drupal site but I can't find anything similar for phenotype data. Can anyone give us some clue for that?


For the record, the OS where the system is installed is Ubuntu 12.04, postgres version is 9.1.11, drupal version is 6.30, and tripal is 1.1.

Many thanks,

Hugo.



------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today. 
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk


_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal


------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal


------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal
Reply | Threaded
Open this post in threaded view
|

Re: Sync phenotype data

Stephen Ficklin-2
In reply to this post by Hugo-pH
Hi Hugo,

Can you tell me what version of Tripal you are using?  Is it v1.1 or is it the development branch 6.x-1.x?  

You mentioned you saw this error message:

warning: include(): Failed opening './sites/all/modules/tripal/tripal_stock/tripal_stock_phenotypes.tpl.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/includes/theme.inc on line 1079.

That error will occur if the file 'tripal_stock_phenotypes.tpl.php' cannot be found in the theme directory.  Drupal searches for theme files first in the default theme, then the tripal theme and then the module.  As you can see in the error message, it's looking for the file in the module directory which means it couldn't find it in the theme directories.  Is the file in your theme directory?  I have found though, that if you customize the node template file (e.g. node--chado_stock.tpl.php) by copying it into the default theme folder that you must also copy all of the other stock template files as well.  If this is the case, you'll find a chado_stock folder in the tripal theme directory which you should also copy to your default theme directory.   After that clear the cache and see if that helps.

The behavior you saw with running that query failing in the 'drupal' schema was correct.   I should have asked you to set the search_path in phppgadmin (in the box just above the query) to be 'chado,public'.   The cvterm_phenotype_name field was NULL because it is the cvterm name of the assay_id.  You probably don't have a value in the phenotype.assay_id field?  That SQL statement was generated by Drupal Views so the naming of the result fields isn't always intuitive.  The uniquename field of the phenotype is shown in the View results but not the name (it can be easily added).

But, that doesn't explain why you couldn't see the results in the phenotype search page...  Do the other search forms under the 'Search Biological Data' work for you?

If you do decide you need a custom search form to find phenotypes associated with stocks or features, let us know and we can provide some pointers.

Stephen


On 1/23/2014 8:28 AM, Hugo Pineda wrote:
Hi Stephen,

Thank for your quick response.
I'll answer your questions bellow:

On 01/22/2014 03:18 PM, Stephen Ficklin wrote:
Hi Hugo,

Thanks for the compliment.  We are actively working on Tripal 2.0 for Drupal 7 which has some nice improvements so it will continue to improve

In response to the search form not working.... You are using the Phenotype search tool under the 'Search biological data' menu item at http://your.site/chado/phenotypes?    That search uses a Drupal View and will try to execute this query for searching:

Yes, I'm running the search at my.site/chado/phenotypes. I have tried entering 'Phenotypic Attribute', 'Unique Name Contains' and leaving all fields blank, but I always get the message: 'No phenotypes match the supplied criteria'.

SELECT
  phenotype.uniquename AS phenotype_uniquename,
  cvterm_phenotype_1.name AS cvterm_phenotype_1_name,
  cvterm_phenotype_2.name AS cvterm_phenotype_2_name,
  phenotype.value AS phenotype_value,
  cvterm_phenotype_3.name AS cvterm_phenotype_3_name,
  cvterm_phenotype.name AS cvterm_phenotype_name
FROM phenotype phenotype
  LEFT JOIN cvterm cvterm_phenotype ON phenotype.assay_id = cvterm_phenotype.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_1 ON phenotype.attr_id = cvterm_phenotype_1.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_2 ON phenotype.cvalue_id = cvterm_phenotype_2.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_3 ON phenotype.observable_id = cvterm_phenotype_3.cvterm_id
ORDER BY phenotype_uniquename ASC

If you run that query manually in phppgadmin do you see results?  Are there any messages in the Drupal recent log ('Administer' -> 'Reports' -> 'Recent log entries') after you try to perform a search?

About the query in phppgadmin, if I run the code at the drupal level (PostgreSQL -> drupal), I get the following error:

SQL error:
ERROR:  relation "phenotype" does not exist
LINE 8: FROM phenotype phenotype
             ^
In statement:
SELECT
  phenotype.uniquename AS phenotype_uniquename,
  cvterm_phenotype_1.name AS cvterm_phenotype_1_name,
  cvterm_phenotype_2.name AS cvterm_phenotype_2_name,
  phenotype.value AS phenotype_value,
  cvterm_phenotype_3.name AS cvterm_phenotype_3_name,
  cvterm_phenotype.name AS cvterm_phenotype_name
FROM phenotype phenotype
  LEFT JOIN cvterm cvterm_phenotype ON phenotype.assay_id = cvterm_phenotype.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_1 ON phenotype.attr_id = cvterm_phenotype_1.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_2 ON phenotype.cvalue_id = cvterm_phenotype_2.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_3 ON phenotype.observable_id = cvterm_phenotype_3.cvterm_id
ORDER BY phenotype_uniquename ASC

Nevertheless, when I run the query at chado level (PostgreSQL -> drupal -> chado) the data is shown correctly except for the phenotype.name field (cvterm_phenotype_name in the results) that it appears as NULL but it's populated in the table (SELECT * FROM phenotype shows that field). Any idea for this behavior?

Another strange thing. In the 'Edit view phenotype_listing', when I click in the 'Preview' button at the bottom of the page, the form is showed but also, all the phenotipic data! if I try to search using the form (still in the edit -> preview) again it returns the 'No phenotypes match the supplied criteria' message.

About the Drupal recent log, when the search is performed, no new messages appears (neither in the postgres log) but there are some warnings and errors in the drupal log that might be related:

warning: include(): Failed opening './sites/all/modules/tripal/tripal_stock/tripal_stock_phenotypes.tpl.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/includes/theme.inc on line 1079.
warning: include(): Failed opening './sites/all/modules/tripal/tripal_stock/tripal_stock_phenotypes.tpl.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/includes/theme.inc on line 1079.

Both messages appears when I access a stock page (my.site/node/24) through my.site/chado/stocks page (Search Biological Data -> Stocks).

That search form will show phenotypes and observed values, but it doesn't link to the stocks or features that may be associated with those phenotypes.  Are you looking for this type of behavior?

I'm really more interested in the next option (access phenotypes through stocks or feature) but it would be great if this kind of search could work too.

Right now, the only way to see phenotype data for features or stocks is if you have it associated with a feature (via feature_phenotype table) or a stock (via nd_experiment tables).  For instance, if you have phenotypes associated with particular stock then when you visit the stock page you should see phenotype information (e.g. http://www.cottongen.org/node/49712 (see Trait Score link--that page has been customized from the default but shows phenotypes and observed values for the stock)).   Once you sync features and stocks the phenotype data, if present, should automatically appear on the appropriate feature and stock pages.  Do you envision that you would need to search or view phenotype data differently?  If so, let me know and perhaps we can help you get what you need.

Ok, I have seen the phenotype data in the stock page, thank you! I think we will work with that for the moment. If I have some troubles editing the view or need anything more specific I would let you know, but thank you for your help offer.
 
On 1/22/2014 4:50 AM, Hugo Pineda wrote:

Hello all for the first time,

We're starting a project for the creation of a phenotypic database and an associated web site for a transgenic strawberry collection. We're using Drupal/Tripal and Chado, specially the Natural Diversity Module.

First I must to say that Tripal is a great tool, really easy to use, so my sincerely congrats and thanks to the developers.

And now the problems. I have installed and configured Drupal-Tripal-Chado without problems, and now I'm trying to upload and retrieve phetotypic data. I have created new organisms and stocks, and have been able to upload data to some of the ND tables, using the bulk loader function (great tool!). But now I get stuck in the phenotype search page. I'm sure that the phenotype table is populated since I can see the data through pgphpadmin and the psql console, but I don't get any results when I run the search in the page. I have seen in the tutorial that, for organism, stock and feature data, it is necessary to sync the database and the drupal site but I can't find anything similar for phenotype data. Can anyone give us some clue for that?


For the record, the OS where the system is installed is Ubuntu 12.04, postgres version is 9.1.11, drupal version is 6.30, and tripal is 1.1.

Many thanks,

Hugo.



------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today. 
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk


_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal




------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal
Reply | Threaded
Open this post in threaded view
|

Re: Sync phenotype data

Hugo-pH
In reply to this post by Lacey-Anne Sanderson
Hi Lacey,

thank you for your help, I'll answer your questions bellow:

On 01/23/2014 06:11 PM, Lacey-Anne Sanderson wrote:
Hi Hugo,
The phenotypic view provided with Tripal is unfortunately a bit buggy -this is fixed in the Drupal 7 Tripal 2 version! Drupal Views 2 (the version you're using right now) doesn't handle many joins to the same table very well and the phenotype view does this the most so unfortunately the bug is in Drupal rather than Tripal making it impossible for us to fix. It is fixed in Drupal 7 Views 3 though.

I do have a few suggestions for you that might work with the current version. The fact that the phenotypic data appears in the preview but not on the page makes me think some of the problem might be with the "Search Results" Filter. You might want to try removing this from the phenotype view (go to admin/build/view and click the edit link beside the phenotype_listing view. Then under the filter heading on the far right, click on the "Global: Search Results" filter. An options pane will open below via ajax; click the remove button. Then save the view. This will cause you view to show results immediately rather than waiting for the user to click search. They will still have the ability to filter the results through the select/text boxes, they just won't have to do that first. This should get you the same behaviour you see in the preview.
I have tried your suggestion and removed the 'Search Results" filter from phenotype_listing view but I still have no results. Now the 'No phenotypes match the supplied criteria' message appears directly when I access the 'chado/phenotype' page.

See additional comments below:

Hope this helps explain what is happening a bit more! If the display of phenotypes on your stock page is sufficient for the moment, I might wait for the search functionality until you upgrade to Tripal 2.
~Lacey

------------------------------------------------------
Lacey-Anne Sanderson
Bioinformaticist
Pulse Crop Breeding and Genetics
Phone: (306) 966-4975
Room 3C06 Agriculture
Department of Plant Sciences
University of Saskatchewan

On 2014-01-23, at 7:28 AM, Hugo Pineda <[hidden email]> wrote:

Hi Stephen,

Thank for your quick response.
I'll answer your questions bellow:

On 01/22/2014 03:18 PM, Stephen Ficklin wrote:
Hi Hugo,

Thanks for the compliment.  We are actively working on Tripal 2.0 for Drupal 7 which has some nice improvements so it will continue to improve

In response to the search form not working.... You are using the Phenotype search tool under the 'Search biological data' menu item at http://your.site/chado/phenotypes?    That search uses a Drupal View and will try to execute this query for searching:

Yes, I'm running the search at my.site/chado/phenotypes. I have tried entering 'Phenotypic Attribute', 'Unique Name Contains' and leaving all fields blank, but I always get the message: 'No phenotypes match the supplied criteria'.

SELECT
  phenotype.uniquename AS phenotype_uniquename,
  cvterm_phenotype_1.name AS cvterm_phenotype_1_name,
  cvterm_phenotype_2.name AS cvterm_phenotype_2_name,
  phenotype.value AS phenotype_value,
  cvterm_phenotype_3.name AS cvterm_phenotype_3_name,
  cvterm_phenotype.name AS cvterm_phenotype_name
FROM phenotype phenotype
  LEFT JOIN cvterm cvterm_phenotype ON phenotype.assay_id = cvterm_phenotype.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_1 ON phenotype.attr_id = cvterm_phenotype_1.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_2 ON phenotype.cvalue_id = cvterm_phenotype_2.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_3 ON phenotype.observable_id = cvterm_phenotype_3.cvterm_id
ORDER BY phenotype_uniquename ASC

If you run that query manually in phppgadmin do you see results?  Are there any messages in the Drupal recent log ('Administer' -> 'Reports' -> 'Recent log entries') after you try to perform a search?

About the query in phppgadmin, if I run the code at the drupal level (PostgreSQL -> drupal), I get the following error:

SQL error:
ERROR:  relation "phenotype" does not exist
LINE 8: FROM phenotype phenotype
             ^
In statement:
SELECT
  phenotype.uniquename AS phenotype_uniquename,
  cvterm_phenotype_1.name AS cvterm_phenotype_1_name,
  cvterm_phenotype_2.name AS cvterm_phenotype_2_name,
  phenotype.value AS phenotype_value,
  cvterm_phenotype_3.name AS cvterm_phenotype_3_name,
  cvterm_phenotype.name AS cvterm_phenotype_name
FROM phenotype phenotype
  LEFT JOIN cvterm cvterm_phenotype ON phenotype.assay_id = cvterm_phenotype.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_1 ON phenotype.attr_id = cvterm_phenotype_1.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_2 ON phenotype.cvalue_id = cvterm_phenotype_2.cvterm_id
  LEFT JOIN cvterm cvterm_phenotype_3 ON phenotype.observable_id = cvterm_phenotype_3.cvterm_id
ORDER BY phenotype_uniquename ASC

This is due to chado being in a separate schema/namespace than the Drupal tables. Drupal Views already knows that so it looks at the chado tables directly.
Thanks for tip, good to know the inner workings :).


Nevertheless, when I run the query at chado level (PostgreSQL -> drupal -> chado) the data is shown correctly except for the phenotype.name field (cvterm_phenotype_name in the results) that it appears as NULL but it's populated in the table (SELECT * FROM phenotype shows that field). Any idea for this behavior?

So the cvterm_phenotype_name is actually the cvterm name referenced by your phenotype.assay_id. So when you select from the phenotype table is your assay_id set?
No it is not, my assay_id column is empty, sorry for that, I misunderstood the query.


Another strange thing. In the 'Edit view phenotype_listing', when I click in the 'Preview' button at the bottom of the page, the form is showed but also, all the phenotipic data! if I try to search using the form (still in the edit -> preview) again it returns the 'No phenotypes match the supplied criteria' message.

This is what I addressed above.


About the Drupal recent log, when the search is performed, no new messages appears (neither in the postgres log) but there are some warnings and errors in the drupal log that might be related:

warning: include(): Failed opening './sites/all/modules/tripal/tripal_stock/tripal_stock_phenotypes.tpl.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/includes/theme.inc on line 1079.
warning: include(): Failed opening './sites/all/modules/tripal/tripal_stock/tripal_stock_phenotypes.tpl.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/includes/theme.inc on line 1079. 

Both messages appears when I access a stock page (my.site/node/24) through my.site/chado/stocks page (Search Biological Data -> Stocks).

This is the template that displays your phenotypes on any stock page which is why it appears when you go to a stock page. Do you have the tripal_theme installed as instructed in the tutorial? Maybe try clearing your Drupal cache a couple of times (the best way is using drush cache-clear all but if you don't have crush you can click the "clear Cached Data"  button at admin/settings/performance though this is a less thorough clear).

Yes, the tripal_theme is installed as in the tutorial. The warnings don't appear anymore. I created a block linking to stock page some days ago. This morning I removed it and the warnings have disappeared.  Nevertheless, now there are two new warnings in the my.site/chado/stocks page:
 
warning: mb_strlen() expects parameter 1 to be string, array given in /var/www/includes/unicode.inc on line 409.
warning: htmlspecialchars() expects parameter 1 to be string, array given in /var/www/includes/bootstrap.inc on line 860.

Any clue?

I'm also testing Stephen's suggestions, I'll follow up his mail.

Hugo.

------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal
Reply | Threaded
Open this post in threaded view
|

Re: Sync phenotype data

Hugo-pH
In reply to this post by Stephen Ficklin-2
Hi Stephen,

thank you again for your help, I'll answer bellow:

On 01/23/2014 09:58 PM, Stephen Ficklin wrote:
Hi Hugo,

Can you tell me what version of Tripal you are using?  Is it v1.1 or is it the development branch 6.x-1.x?  
The 'sites/all/modules/tripal/tripal_core/tripal_core.info' file includes this information:

name = Tripal Core                                                                                                                            
description = The core module for the Tripal package that integrates Drupal and GMOD chado. This module provides common support for all Tripal modules.
core = 6.x
project = tripal_core
package = Tripal
version = 6.x-1.1

You mentioned you saw this error message:

warning: include(): Failed opening './sites/all/modules/tripal/tripal_stock/tripal_stock_phenotypes.tpl.php' for inclusion (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/includes/theme.inc on line 1079.

That error will occur if the file 'tripal_stock_phenotypes.tpl.php' cannot be found in the theme directory.  Drupal searches for theme files first in the default theme, then the tripal theme and then the module.  As you can see in the error message, it's looking for the file in the module directory which means it couldn't find it in the theme directories.  Is the file in your theme directory?  I have found though, that if you customize the node template file (e.g. node--chado_stock.tpl.php) by copying it into the default theme folder that you must also copy all of the other stock template files as well.  If this is the case, you'll find a chado_stock folder in the tripal theme directory which you should also copy to your default theme directory.   After that clear the cache and see if that helps.
As you can see in my message to Lacey, after removing a block linking to stock that I created previously the warnings have disappeared.
Nevertheless, the 'tripal_stock_phenotypes.tpl.php' file is not in my actual default theme directory (sites/all/themes/sky) and neither is in the tripal_theme folder. If I make 'locate tripal_stock_phenotypes.tpl.php' I get no results, but there is a file called tripal_stock_nd_phenotypes.tpl.php in 'sites/all/themes/tripal_theme/tripal_stock/'. Is it this file that Drupal is searching for? For the record, this is the content of the 'sites/all/theme/tripal_theme' folder:

-rw-r--r--  1 root root   13 Jan  9 12:06 README.txt
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 css
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 images
drwxr-xr-x  5 root root 4.0K Jan  9 12:06 js
-rw-r--r--  1 root root  15K Jan  9 12:06 logo.png
-rw-r--r--  1 root root 4.8K Jan  9 12:06 node-chado_analysis.tpl.php
-rw-r--r--  1 root root 5.4K Jan  9 12:06 node-chado_contact.tpl.php
-rw-r--r--  1 root root 6.4K Jan  9 12:06 node-chado_feature.tpl.php
-rwxr-xr-x  1 root root 5.5K Jan  9 12:06 node-chado_featuremap.tpl.php
-rw-r--r--  1 root root 5.4K Jan  9 12:06 node-chado_library.tpl.php
-rw-r--r--  1 root root 4.9K Jan  9 12:06 node-chado_organism.tpl.php
-rw-r--r--  1 root root 5.4K Jan  9 12:06 node-chado_project.tpl.php
-rw-r--r--  1 root root 5.8K Jan  9 12:06 node-chado_pub.tpl.php
-rw-r--r--  1 root root 5.4K Jan  9 12:06 node-chado_stock.tpl.php
-rw-r--r--  1 root root 2.6K Jan  9 12:06 node-tripal_bulk_loader.tpl.php
-rw-r--r--  1 root root  357 Jan  9 12:06 node-tripal_pub.tpl.php
-rw-r--r--  1 root root  587 Jan  9 12:06 node.tpl.php
-rw-r--r--  1 root root 9.7K Jan  9 12:06 screenshot.png
-rw-r--r--  1 root root    7 Jan  9 12:06 template.php
-rw-r--r--  1 root root 1.5K Jan  9 12:06 theme-settings.php
-rw-r--r--  1 root root    0 Jan  9 12:06 tripal-node-templates.css
-rw-r--r--  1 root root  602 Jan  9 12:06 tripal.info
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 tripal_analysis
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 tripal_bulk_loader
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 tripal_contact
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 tripal_feature
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 tripal_featuremap
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 tripal_library
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 tripal_organism
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 tripal_project
drwxr-xr-x  3 root root 4.0K Jan  9 12:06 tripal_pub
drwxr-xr-x  2 root root 4.0K Jan  9 12:06 tripal_stock

And the content of the 'sites/all/theme/tripal_theme/tripal_stock' folder:

-rw-r--r--  1 root root 2.2K Jan  9 12:06 tripal_stock_base.tpl.php
-rw-r--r--  1 root root 1.9K Jan  9 12:06 tripal_stock_collections.tpl.php
-rw-r--r--  1 root root 4.9K Jan  9 12:06 tripal_stock_genotypes.tpl.php
-rw-r--r--  1 root root 4.2K Jan  9 12:06 tripal_stock_nd_genotypes.tpl.php
-rw-r--r--  1 root root 4.7K Jan  9 12:06 tripal_stock_nd_phenotypes.tpl.php
-rw-r--r--  1 root root 1.9K Jan  9 12:06 tripal_stock_properties.tpl.php
-rw-r--r--  1 root root 2.1K Jan  9 12:06 tripal_stock_references.tpl.php
-rw-r--r--  1 root root 4.5K Jan  9 12:06 tripal_stock_relationships.tpl.php
-rw-r--r--  1 root root 1.6K Jan  9 12:06 tripal_stock_synonyms.tpl.php
-rw-r--r--  1 root root 3.6K Jan  9 12:06 tripal_stock_teaser.tpl.php


I'm interested in understanding how I have to proceed when the Drupal theme is changed since I'm also trying to set one of them for our site. So when I change the default theme, I add the 'base theme = tripal" line to the 'theme.info' file how the tutorial said, but is it also necessary to copy the 'tripal_theme/tripal_stock' folder to the default theme directory?


The behavior you saw with running that query failing in the 'drupal' schema was correct.   I should have asked you to set the search_path in phppgadmin (in the box just above the query) to be 'chado,public'.   The cvterm_phenotype_name field was NULL because it is the cvterm name of the assay_id.  You probably don't have a value in the phenotype.assay_id field?  That SQL statement was generated by Drupal Views so the naming of the result fields isn't always intuitive.  The uniquename field of the phenotype is shown in the View results but not the name (it can be easily added).
Yes, that is, the phetotype.assay_id field is empty, thank you for the explanation :)

But, that doesn't explain why you couldn't see the results in the phenotype search page...  Do the other search forms under the 'Search Biological Data' work for you?
Well, 'Stocks', 'Organisms' and 'Features' search forms work fine, but I can't test the other forms since I still don't have any data for 'Genotypes', 'Projects' or 'Ananlyses'. I'll introduce some data from the tutorial and I'll let you know what happen.


Thanks again,
Hugo.





------------------------------------------------------------------------------
CenturyLink Cloud: The Leader in Enterprise Cloud Services.
Learn Why More Businesses Are Choosing CenturyLink Cloud For
Critical Workloads, Development Environments & Everything In Between.
Get a Quote or Start a Free Trial Today.
http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
_______________________________________________
Gmod-tripal mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-tripal