Re: [Gmod-tripal] Natural Diversity -Drupal Stock & Natural Diversity Modules

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: [Gmod-tripal] Natural Diversity -Drupal Stock & Natural Diversity Modules

Bob MacCallum
Tripal looks like a really viable option for the UI for the natural
diversity db and I've just signed up for the gmod-tripal mailing list.

We have some concerns though, for example, there seems to be a lot of
bare SQL in the Tripal modules (although we like the modular design).
Using a schema as flexible as Chado across many physical sites (as
VectorBase will be doing), we would prefer to have all data go in/out
of the database via an API; this means loader scripts, web UI
read/write, BioMart dumper scripts, everything, to ensure that the
data is consistently represented in the db.  I'm not a PHP expert, but
have read about doctrine 1.2 and 2.0, and it seems as if some
higher-level code could be developed with a doctrine ORM layer.
(www.doctrine-project.org)

So I have a couple of questions:

1. What API development plans do people have for the NatDiv schema (in
whatever language)?
2. Could an API be made to work across sites (e.g. apples, rice,
mosquitoes, ...) or are we using the schema in too different ways?
3. Is there any Tripal/ORM development going on at the moment?

cheers,
Bob.

On Wed, May 12, 2010 at 3:16 PM, Stephen Ficklin <[hidden email]> wrote:

> Hi Lacey,
>
>
>
> Chado currently does have some recommendations for table auditing (history
> keeping) at http://gmod.org/wiki/Chado_Audit_Module.  These recommendations
> are not implemented in Chado, just recommendations.  I do, however, like
> your idea of only storing the edited column as this will reduce database
> bloat over time.
>
>
>
> Also, this past January at the GMOD annual meeting the folks from GNPannot
> at CIRAD showed a very nice trigger based wrapper for Chado called Chado
> controller:  http://www.gnpannot.org/content/chado-controller .  It helps
> control user access and auditing of a chado database.  It’s all based on
> Postgres and requires no other software.  I can contact them and see if they
> would mind sharing with us.
>
>
>
> Stephen
>
>
>
>
>
> From: Lacey-Anne Sanderson [mailto:[hidden email]]
> Sent: Monday, May 10, 2010 6:27 PM
> To: GMOD Tripal; GMOD Schema; Meg Staton; Stephen Ficklin
> Subject: Re: [Gmod-tripal] Natural Diversity -Drupal Stock & Natural
> Diversity Modules
>
>
>
> PS. Stephen -See my comments below #2 in regards to the delete issue. This
> way I can implement marking as obsolete separately from deleting which is
> likely a more general use case. What do you think?
>
>
>
> ------------------------------------------------------
>
> Lacey-Anne Sanderson
>
> Bioinformaticist
>
> Pulse Crop Breeding and Genetics
>
> Phone: (306) 966-2430
>
> Room 3D10 Argriculture
>
> Department of Plant Sciences
>
> University of Saskatchewan
>
>
>
> On 2010-05-10, at 10:36 AM, Meg Staton wrote:
>
>
> 2.  Drupal does have a versioning system, but we don't use it.  Chado just
> isn't built to handle versioning of records, so we haven't tackled it.  We
> want to enable community annotation, but backing out to old versions is
> tough.
>
> The first idea that comes to mind is to create a new chado stock record each
> time a node is changed, and have each drupal node revision point to a
> different stock record in chado.  You'd have to copy over any related data
> (props, cvterms, pubs) as well.  You'd have to think through every possible
> relationship in the database.  And this solution seems like a "hack" of
> chado - if you handed your database to someone else, it wouldn't make sense
> to them, because the stock table would have many revisions of what should be
> a single stock record.
>
> My second idea would be to somehow store the stock record details in a
> drupal field that is parseable later.  If you needed to roll back, you would
> just pick up the old fields in drupal, and restore them to chado.  This has
> the problem of being redundant and bloating your drupal nodes.  Basically,
> if we implemented this for all the modules, we might as well get rid of
> chado entirely and just store everything in drupal.
>
> I'm not a fan of either of these solutions, but I think the first one is
> better.  Are there any other ideas out there?
>
>
>
> I think this needs to be handled by a versioning chado schema module. What
> comes to mind is adding a stock_revision table which has a stock_id, date,
> wwwuser, column_name old_value, new_value & just updating the stock
> everytime. You could even implement triggers which automatically add to the
> stock_revision table on update. Then when you need to rollback you can just
> select all old_values from the date period you want to roll back to and
> update the stock table to have these values.
>
>
>
> This would at least keep the stock table uncluttered & unchanged... Also it
> should be pretty generic.
>
>
> Thanks!
> Meg
>
>
> --
> Margaret E. Staton
> Clemson University Genomics Institute
> [hidden email]
> 864-656-4643
>
>
>
>
>
> ________________________________
>
> From: Stephen Ficklin <[hidden email]>
> To: Lacey-Anne Sanderson <[hidden email]>; GMOD Schema
> <[hidden email]>; GMOD Tripal
> <[hidden email]>
> Sent: Fri, May 7, 2010 7:00:33 PM
> Subject: Re: [Gmod-tripal] Natural Diversity -Drupal Stock & Natural
> Diversity Modules
>
> Hi Lacey,
>
> Sounds like you've done a great job with a new Drupal stock module for
> Chado!
>
> In response to your first question below, perhaps you could maintain the
> 'is_obsolete' column of chado as an option in your Drupal module as well.
> Therefore when a user really wants to delete the stock it is really
> deleted.  But, if the user wants to make a stock obsolete then it gets
> flagged as obsolete in Chado and reflected as such in Drupal.  The GMOD
> folks can correct me if I'm wrong but I believe the is_obsolete field is
> intended to keep data around for historical purposes so it doesn't have to
> be deleted.  In which case you would still want to maintain and show the
> Drupal node for stocks that have the is_obsolete set to true.
>
> In response to your third question.  I know the folks at WSU are planning to
> do some Drupal work that would plug into Tripal for the natural diversity
> tables that are being developed.  I have not followed that discussion very
> closely on the mailing list, but I don't think it was decided how that would
> be distributed.  My opinion is that I would like to see Drupal modules that
> are more generic in nature that correspond to data management of chado
> modules be incorporated as "core" modules in Tripal.  Other modules that are
> highly tailored, for example for different analysis methods should be
> offered separate from Tripal. We will probably take the analysis modules
> that we developed and pull them out of the "core" as well.  But for now it's
> just easier to distribute them all together.  We are willing to house a
> repository of user-contributed Tripal/Drupal compatible modules, and give
> appropriate credit to the authors.  This will create a single place to find
> them all.  But, ultimately where a module goes should be up to the
> developers.  I'd be happy to discuss with you the possibility of
> incorporating your stock module into Tripal if you would like it to go that
> route.
>
> Cheers,
> Stephen
> ________________________________________
> From: Lacey-Anne Sanderson [[hidden email]]
> Sent: Friday, May 07, 2010 5:51 PM
> To: GMOD Schema; GMOD Tripal
> Subject: [Gmod-tripal] Natural Diversity -Drupal Stock & Natural Diversity
>     Modules
>
> I'm creating a chado_stock drupal module that is almost ready to be released
> as an alpha version (sometime next week).
>
> Details:
> - only stores stock_id mapped to a drupal node in drupal
> - all other data is queried from chado when the node is loaded
> - data from chado that is supported includes all fields from the stock
> table, those from stockprop (including a separate listing of those with
> cvterm 'synonym'), from stock_dbxref and from stock_relationship
> -the fields from stock are listed in a table format at the top of the node
> and data from the other three tables are shown below each inside its own
> signature tripal expandable box.
> -syncing of data between drupal & chado (essentially creating drupal nodes
> for all chado stock records & storing the stock_id) is supported
> -as is entering stocks through the web interface (currently need to finish
> this for the other 3 tables) which writes directly to chado.
>
> Features not yet supported (Still in the works):
> A number of admin features including cleanup, reindexing and taxonomy
> tracking of who entered the data in chado -some type of versioning?
> More user friendly selecting of stocks for relationships and cvterms
> (currently is a Very long drop-down select)
>
> What needs to be done next:
> 2. Create a generic nd_assay module
> 3. Create genotype, phenotype, crossexperiment and fieldcollection modules
> each will all the features from nd_assay but with added required fields and
> customized display
> 4. Create a project module which groups together data into projects. This
> one is not specific to the natural diversity project but one of the types of
> data that can be added to a project with be a nd_assay.
>
> A couple of Questions:
> 1. Deleting a stock node in drupal -> how should this be reflected in drupal
> - although stock.is_obsolete can be set to True there doesn't seem to be a
> similar way to indicate that stock relationships or properties have been
> deleted...
> - stocks which are marked as obsolete won't be added back into drupal during
> syncing therefore does it matter if the relationships and properties aren't
> marked since they are only available by accessing chado directly?
>
> 2. This is a more chado-oriented question: How is versioning handled?
> -In drupal you can easily edit a stock which currently just writes over the
> original content in chado...
> - Is there a way to keep the original in case you want to rollback?
>
> 3. Should this stock module be part of tripal rather than the drupal natural
> diversity module?
> Should the Drupal Natural Diversity module be separate from Tripal
> (distributed separately as an add-on?
>
> Thanks for any input!
> Lacey
>
>
> ------------------------------------------------------
> Lacey-Anne Sanderson
> Bioinformaticist
> Pulse Crop Breeding and Genetics
> Phone: (306) 966-2430
> Room 3D10 Argriculture
> Department of Plant Sciences
> University of Saskatchewan
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Gmod-tripal mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gmod-tripal
>
>
>
>
>
>

------------------------------------------------------------------------------

_______________________________________________
Gmod-schema mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gmod-schema